Feedback
The Basics
|
Quickstart Download License Installation |
User's Manual
|
Overview Policies I/O Retargeting Deployment Formal Stuff Language Reference |
Language Reference
The language reference contains two parts: a syntactic reference, and a semantic reference. Read the syntactic section to learn exactly how to format your Policy file, and the semantic section to learn exactly how the Reflex engine processes each incoming event.
Our event specification language is expressive and flexible. The grammar in EBNF is as follows:
Policy ::= Pattern {Pattern}
|
<pattern-name>s are simply identifiers.
<expression>s are similar to C++ expressions. They may contain relational operators (<,>,<=,>=,==,!=), basic arithmetic operators (+,-,*,/), and boolean operators (&&,||). The expression itself should evaluate to true or false (whether to match the current event). The expression may also contain attributes from the current event (represented by identifiers), stored attributes for a partial match (identifiers in []s), and constants ("strings", integers, doubles, true/false).
<stored-attribute>s are identifiers in square brackets ([]s). The assignments are executed if the expression is evaluated as true. Attributes can be queried for existence by the isNull function.
Note how the event sets must be in this order: reset, always, ordered, unordered.
Datatypes
When the system reads in a name/value pair from the event stream, the element that is created to model the pair dynamically determines the datatype of the value. It uses this method for determining:
'true' or 'false' bool
|
Programming Constructs
Even though the system uses a pattern-specification language, there are some ways to implement familiar imperative language control structures like loops and conditionals.
Loops
Many times, one will want to see some number of events pass by and then trigger an alarm when that number exceeds some limit. In C, it may look like this:
int eventCount = 0;
|
This is also possible in our language using the always event set.
# Once we see a "start" event, count, and set an alarm 5 events after
|
Conditional Expressions
In the action part of an event, one might want to perform the action only if some condition qualifies. In C,
if( Action == "PiggyBack" )
|
This is possible in our language as well. Events already come in the form
( some-condition ? some-action )
Using the ( ? ) syntax, conditional actions can be created.
pattern( CheckAction
|
Recirculating Alarms
The system is capable of describing and detecting more complicated patterns that are composed of sub-patterns. This is because when it matches a pattern, it sends out a new event describing the alarm. The elements of the event consist of PatternName, and any stored variables that that pattern-matching procedure picked up.
pattern( Small
|
When the Small pattern is found, the system sends out an event with the attribute PatternName set as the name of the pattern "Small". When the Medium pattern is found, it also sends out an event with the PatternName as "Medium", but since the pattern keeps track of the SmallCount, that name/value pair is also part of the Medium event. The Big pattern only triggers when the SmallCount in the Medium event is greater than 5.
A sample event stream is:
|
Glossary
| policy | The collection of patterns. |
Keywords
| pattern | The start of a pattern. pattern( myPattern ... ) |
| reset | The start of a reset set. Reset events are events that when found reset the state of the partial match, i.e. erase the mark. reset( (action=="ResetPolicy" ) )
|
| always | The start of an always set. Always events are events that when found do not change the state of a partial match. Always events are used mainly for logging purposes, keeping track of the number of occurrences of a particular event to report later. always( (type=="Poke" ? [pokecount] = [pokecount] + 1; ) )
|
| ordered | The start of an ordered set. Ordered events are events that must be seen in order to continue matching the pattern. # If we see a Poke pattern and then a Probe pattern... |
| unordered | The start of an unordered set. Unordered events are events that can be seen in any order to continue matching the pattern. # If we ever see a Poke and a Probe... |
Operators
The operators used in our language are similar to the ones used in C++. Numbers are compared like normal, strings are compared lexically, and true and false are casted to 1 and 0 respectively, then compared. The same precedence rules apply, but parentheses may be used for clarification. Operators have been listed in order of precedence.
| * | Arithmetic multiply. [AttackRating] = damage * [AttackCount]; |
| / | Arithmetic divide. [Average] = ( first + second ) / 2; |
| + | Arithmetic plus. [AttackCount] + [SniffCount] < 20 |
| - | Arithmetic minus. [finishTime] - [startTime] < 5 |
| < | Less-than. port < 50 |
| <= | Less-than-or-equal. [tries] <= 20 |
| > | Greater-than. [AttackCount] > 5 |
| >= | Great-than-or-equal. [AttackCount] >= 5 |
| == | Equal. SourceIP == [StoredIP] |
| != | Not-equal. SourceIP != [StoredIP] |
| && | Boolean logical AND. Commonly used to link predicate comparisons. type=="Attack" && port==12121 |
| || | Boolean logical OR. Commonly used to link predicate comparisons. port == 80 || ( type == "FTP" && port == 25 ) |
| = | Assignment. A value can only be assigned to a stored attribute. [SourceIP] = attacker; |
Other
| ; | The end of an assignment. [this] = is + an + assignment;
|
| # | A comment. Applies for one line only. # This is a comment. |
| isNull( <Event or Stored Attribute> ) | This function checks if the attribute has been declared and initialized yet. # If we haven't logged this SIP yet, do so. |
Principles
Our first task in designing Reflex was to determine our philosophies for it, so that we could argue about how it should behave when we had more than once choice. The easiest way to understand how it works, therefore, is to start with those same philosophies in hand.
- Reflex should provide the minimum sensible output. Many event correlators are designed to monitor networks, and their main job is to turn a slew of symptoms into a single message indicating the true problem. Reflex should react whenever it sees the expected input, but not more often than a user would expect.
- Reflex should be easy to follow, even when distributed. We built Reflex so that the correlation work could be distributed over multiple machines. Reflex should give the same output distributed as it would running as a single process.
- Reflex should work in an environment with many things happening at the same time. We expect that Reflex engines will run in environments with many messages, most of which the engine should ignore. Also, the engine may need to react more than once to more than one problem. (As an example, if the reflex is to remove a finger from a hot stove, if two fingers are touch the stove the reflex had better be to remove both of them!)
Semantics -- Following an Event
A new event is passed to a Reflex engine via the Handle() function. Here's what happens:
The event goes from one pattern to the next, in the same order that the patterns were written in the policy. For each pattern, the event may be used only once.
First, the event is compared to each partial match for the pattern, starting with the oldest. For each partial match, the event is first examined as a possible 'reset' event. If it matches, the partial match is deleted and processing moves to the next pattern!
If not, the event is examined as an 'always' event. If it matches, the event may also match this partial match only as an ordered or unordered event. (This allows descriptions that examine only stored attributes to match, completing counting patterns and such). After the always match and possible ordered or unordered match, processing moves to the next pattern.
If not, the event is examined as an 'ordered' or 'unordered' event (ordered first). If it matches, the state of the partial match changes and processing moves to the next pattern. If the partial match was completed by this event, the partial match is deleted and sent out as an Alarm.
If this event has not matched this partial match at all, it attempts to match against the next partial match for this pattern.
If there are no more partial matches, the event is checked as an 'ordered' or 'unordered' event to create a new match (only the first ordered event, of course). If it matches, a new partial match is created. (If there are too many partial matches, the oldest for this pattern is discarded!) The limit defaults to 100 matches per pattern, and can be modified by editing the correlator engine code.
At this point, processing moves to the next pattern.
What just happened here? The event can be used by by each pattern once. It is checked against partial matches before getting the opportunity to create a new match. It is checked against the oldest partial match first.
The event is evaluated as a reset event, then an always event, then an ordered event, then an unordered event. (The same order as in the Policy file). If more than one description in a pattern can match the event, the first one written will match it.
If the event matches anything, the state of the partial match changes and all other partial matches are left alone. Processing resumes on the next pattern's partial match set.
If an event matches an always description, it is also allowed to match an ordered or unordered description for the same match. This allows patterns that count or collect events to have a final ordered or unordered description which will test for a completion condition and sent out the alarm when this happens.
This is how an Event is processed by the Reflex engine. To examine the exact code, check out the implementation of the Handle() function that the compiler produces for your Policy. If the behavior seems odd, run through the steps that the engine does and you'll often find why.
Copyright © 2001