Hi Carl, Thanks for the many constructive suggestions and opinions about ossec. I really like this kind of feedback, which will just make ossec better in the long term.
I will try to answer them, but if I forget about something, just ask :) -Yes, our rule/decoder documentation is not in the level I wanted it to be. I am trying to push the usage of the wiki to improve its quality, but we still have a long way to go. One document that might be helpful too is (my presentation at AusCERT): http://www.ossec.net/ossec-docs/conf2007-dcid.pdf -Regarding the terms, english is not my main language, so all these "if_matched", "if_sid", etc are bad named for sure :) You will also find lots of weird configuration option names, mainly because I couldn't name it properly. Maybe for version 2.0, we could put up a team to go though them all and name it better. -Classification is used to speed up our event processing. In the decoder you can set a "type" (like firewall, windows, squid, etc) and on the rule you can have a "category" (with the same names as in the decoder). What happens is that if you have an event that was decoded as type "firewall", for example, it will only be evaluated against rules in that category, so saving a lot of cycles in rules that we are not interested. This is organized in the "rules_config.xml", where we have the first points of the tree. -<match> is just a fast string matching. It allows for a few keywords and nothing else. More info: http://www.ossec.net/wiki/index.php/Know_How:Regex_Readme -On the if_sid you can provide more than one rule id, like <if_sid>123,234</if_sid> or as you said, you can use the group option. (<if_group>). -The if_match (either group or sid or regex), mainly look back in our event list for events that matched in the past. For performance reasons, we generally use <if_matched_sid> or <if_matched_group> because each has its own internal lists, making the search very quickly. On the other hand, with if_match_regex, you look at every single event that is stored in there... -What do you mean by temporal rules? On the ossec-logtest, if you type multiple entries, it will store them in the event list, so your if_matched_* will be tested... Hope it answers your concerns a bit. Thanks, -- Daniel B. Cid dcid ( at ) ossec.net On Sat, Aug 23, 2008 at 9:49 PM, <[EMAIL PROTECTED]> wrote: > > Hi, > I've recently been looking at programming a number of customized > rules for OSSEC. The following is a collection of *personnel* comments > regarding my OSSEC learning curve. Any mistakes/misunderstandings in > the following simply reflect my (potentially!?) faulty interpretation > of what I believe is going on (at this moment in time!) - YMMV. > > Firstly, I've read your online documentation and the OSSEC book. > Sadly, you can't really proceed without both information sources being > at your fingertips (along with various other unspecified occult > devices!) [ps. The online manual has a great summary of rule, etc. > options - something the book does not readily provide access to (IMHO, > this is mainly due to the fact that the books index has been compiled > for the wrong use-case scenarios (eg. think about trying to access/ > find information regarding rule groups in the book) - the eBook > version is the easy way forward here).]. > > One of your biggest failings in *all* of your documentation is the > lack of any real description of the rule processing engine. Sure you > have high level views of it, but nothing that's meaningful to a > programmer (ie. order of evaluation; meaning of an evaluation goal; > how groups work; etc.). For myself, what has really helped in my > understanding of rule processing are uses of: > - the tool ossec-logtest > - the group thread http://www.ossec.net/ossec-list/2007-January/msg00029.html > (backed up my own views as to how a rule-tree should be efficiently > evaluated!) and even then, I've had major problems understanding what > is *actually* meant by terms such as: > - group > - matched > I've yet to understand what is meant by match(!) and category (and no > doubt there's a series of other terms also awaiting to be > deciphered!?). > > In terms of structuring my rules, it eventually became clear that I > should have 2 sets of local rules: > - classification rules: these are often atomic rules and their sole > purpose is to note and report certain types of events (they're often > the rule counterpart of the log lines the decoder is actioning. These > rules tend to be more brittle in nature across different OSes. > - logical rules: these are typically a composite rule and are often > temporal in nature - typically these rules are highly portable (if > written correctly!). > I now organize my rules so that all logical rules occur last (that way > back references will work as I expect them to work - see below). It's > also worth pointing out that there are a few standard/shipped rules > that will not operate as expected (since they make use of if_group > conditions too early (ie. whilst no rules have yet become members of > the group) in the rule tree (eg. the attack, policy and 1 syslog > rules)). > > If we focus on processing atomic rules, then everything (IMHO) works > great - well, ok, up to a point. You have to realize that, for a given > rule: > - the rule can have 3 categories of tags: > + conditions (eg. if_group; if_sid; etc.) > + meta-tags (eg. comments; description; etc.) > + state changes (eg. group; options; etc.) > - conditions such as if_sid or if_group are back references to other > rules *from* the point you currently are at within the rule tree. It's > a shame groups don't actually reference forward as well - however, > there may well be a good reason for not doing this!? > > When you wish to encode disjunctive rule conditions (ie. an action > states that event1 or event2 has occurred), it becomes clear that > groups are the way forward here. What isn't initially clear is that > conditions within rules are evaluated in a conjunctive manner (ie. > condition1 and condition2 and...). > > The fact that rule processing stops at a goal (ie. an action will > occur at a rule node with no (back) references to it) is fine for > atomic-type rules. With composite rules, I think you'll have real > problems here with expressivity (here I believe we need to process all > rules - I'd need to build a formal model to work this out > definitively). > > As an example: with composite terms, we have real problems if we wish > to capture fast and slow events. Imagine a scenario where we have some > firewall drops (from a common IP address) that occur to a number of > honey-ports in a short period of time. Also imagine that we have a > number of firewall drops (from a common IP address) that occur slowly > over a much larger period of time. FWIU, the slower events will never > get triggered because the rule itself is never considered (it has no > back reference into the fast rule and the fast rule has precedence > over the slow one). If you swap the rule order, you end up with an > analogous problem. > > Whilst documented, it is not immediately obvious that terms starting > if_matched are actually future tense terms and not (as the name > suggests) past tense terms. It's also not sufficiently clear as to > what is meant by the term 'frequency' - experimentation seems to > indicate that the term refers to how many events have been > *previously* seen (ie. we don't include the current event). > > To be honest, if it wasn't for the recent release of your ossec- > logtest tool, I wouldn't have got this far in the time I've had to > evaluate OSSEC - at least not without trawling through the source code > - so great work and thanks (we just need temporal testing now)! > > For myself, my current limitation/lack of knowedge is how you deal > with temporal based rules. Do you have any formal logical models (eg. > modal or temporal) that attempt to classify the types of logging > events you may capture? Is there any chance of developing ossec- > logtest further to help debug such issues? > > Many thanks on such a great product, > > Carl Pulley. > >
