Hi, I've recently been looking at programming a number of customized rules for OSSEC. The following is a collection of *personnel* comments regarding my OSSEC learning curve. Any mistakes/misunderstandings in the following simply reflect my (potentially!?) faulty interpretation of what I believe is going on (at this moment in time!) - YMMV.
Firstly, I've read your online documentation and the OSSEC book. Sadly, you can't really proceed without both information sources being at your fingertips (along with various other unspecified occult devices!) [ps. The online manual has a great summary of rule, etc. options - something the book does not readily provide access to (IMHO, this is mainly due to the fact that the books index has been compiled for the wrong use-case scenarios (eg. think about trying to access/ find information regarding rule groups in the book) - the eBook version is the easy way forward here).]. One of your biggest failings in *all* of your documentation is the lack of any real description of the rule processing engine. Sure you have high level views of it, but nothing that's meaningful to a programmer (ie. order of evaluation; meaning of an evaluation goal; how groups work; etc.). For myself, what has really helped in my understanding of rule processing are uses of: - the tool ossec-logtest - the group thread http://www.ossec.net/ossec-list/2007-January/msg00029.html (backed up my own views as to how a rule-tree should be efficiently evaluated!) and even then, I've had major problems understanding what is *actually* meant by terms such as: - group - matched I've yet to understand what is meant by match(!) and category (and no doubt there's a series of other terms also awaiting to be deciphered!?). In terms of structuring my rules, it eventually became clear that I should have 2 sets of local rules: - classification rules: these are often atomic rules and their sole purpose is to note and report certain types of events (they're often the rule counterpart of the log lines the decoder is actioning. These rules tend to be more brittle in nature across different OSes. - logical rules: these are typically a composite rule and are often temporal in nature - typically these rules are highly portable (if written correctly!). I now organize my rules so that all logical rules occur last (that way back references will work as I expect them to work - see below). It's also worth pointing out that there are a few standard/shipped rules that will not operate as expected (since they make use of if_group conditions too early (ie. whilst no rules have yet become members of the group) in the rule tree (eg. the attack, policy and 1 syslog rules)). If we focus on processing atomic rules, then everything (IMHO) works great - well, ok, up to a point. You have to realize that, for a given rule: - the rule can have 3 categories of tags: + conditions (eg. if_group; if_sid; etc.) + meta-tags (eg. comments; description; etc.) + state changes (eg. group; options; etc.) - conditions such as if_sid or if_group are back references to other rules *from* the point you currently are at within the rule tree. It's a shame groups don't actually reference forward as well - however, there may well be a good reason for not doing this!? When you wish to encode disjunctive rule conditions (ie. an action states that event1 or event2 has occurred), it becomes clear that groups are the way forward here. What isn't initially clear is that conditions within rules are evaluated in a conjunctive manner (ie. condition1 and condition2 and...). The fact that rule processing stops at a goal (ie. an action will occur at a rule node with no (back) references to it) is fine for atomic-type rules. With composite rules, I think you'll have real problems here with expressivity (here I believe we need to process all rules - I'd need to build a formal model to work this out definitively). As an example: with composite terms, we have real problems if we wish to capture fast and slow events. Imagine a scenario where we have some firewall drops (from a common IP address) that occur to a number of honey-ports in a short period of time. Also imagine that we have a number of firewall drops (from a common IP address) that occur slowly over a much larger period of time. FWIU, the slower events will never get triggered because the rule itself is never considered (it has no back reference into the fast rule and the fast rule has precedence over the slow one). If you swap the rule order, you end up with an analogous problem. Whilst documented, it is not immediately obvious that terms starting if_matched are actually future tense terms and not (as the name suggests) past tense terms. It's also not sufficiently clear as to what is meant by the term 'frequency' - experimentation seems to indicate that the term refers to how many events have been *previously* seen (ie. we don't include the current event). To be honest, if it wasn't for the recent release of your ossec- logtest tool, I wouldn't have got this far in the time I've had to evaluate OSSEC - at least not without trawling through the source code - so great work and thanks (we just need temporal testing now)! For myself, my current limitation/lack of knowedge is how you deal with temporal based rules. Do you have any formal logical models (eg. modal or temporal) that attempt to classify the types of logging events you may capture? Is there any chance of developing ossec- logtest further to help debug such issues? Many thanks on such a great product, Carl Pulley.
