On Tue, 12 Oct 2010, Morris, Christopher wrote:
I use SEC to monitor UNIX syslog messages on a central loghost server.
When I started using SEC a single config file was sufficient. But as time went
on and number of rules grew it became clear that my single config file was a
maintenance problem.
To make matters worse, I use a philosophy whereby I create a SEC rule for every known
message/issue -- and then any unknown message "falls through" all the rules and
is reported as an unknown event so that I can examine it and of course make a new rule.
(The reasoning behind this process is that I don't necessarily know what kinds of events
I am looking for and I need to be aware of new or unknown events. Our environment is
highly dynamic and so have to be able to adapt to changing conditions.)
I do the same thing, but instead of generating the resulting 'unknown'
messages in real time, I do it as a nightly report (my log volume is high
enough that doing it in real time would kill me when a new app started
spewing logs)
As you can guess -- besides a maintenance problem, there is also a performance
problem as each message may be examined by lots of rules. I've mitigated this
problem to some degree by creating sections in our configuration file... There
is a section at the top called 'high-volume' that intercepts and handles events
that are known to be voluminous in nature. And then there's a 'temporary'
section to deal with short term or unusual conditions. Finally, the remaining
rules are ordered by the syslog 'tag'. I sometimes use SIGUSR1 to dump the
rule-usage-frequency and may will reorder the rules to reduce the load on the
system and to improve message processing. But of course this adds to the
maintenance burden... (BTW, our loghost is a Sun R240 system.)
I've experimented with creating rules in separate config files - but this has
two problems from my perspective: One is that it doesn't help the problem of
unnecessary examination of events by rules that don't really apply to that
ruleset -- and (2) is that my 'philosophy/strategy' of using a set of rules to
filter out known events and reporting exceptions breaks down when I use
multiple config files... Or does it?
I guess I'm asking if there are some ideas/suggestions for rules/patters/strategies
that I might be able to adopt to make our SEC configuration better and to make
management & maintenance easier given the style and goals we have.
if you put the rules in multiple files, group them by something and then
have the first rule in the file be to match that something, otherwise
skip checking all other rules in that file
has anyone thought about making a variation of SEC that could compile the
rule matches down to a parse tree instead of doing a series of regex
matches? at low traffic volumes it doesn't matter, but for high traffic
volumes and large numbers of rules (like Christopher has) it could make a
huge difference.
David Lang
------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3.
Spend less time writing and rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3.
Spend less time writing and rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
