Hi all:
I know this should be doable, and I know my current thinking is
barking up the wrong tree, but my brain is locked into doing something
and it's not quite working.
Here's what happened. In order to get the output from postgres into
the proper form to be used by an performance analysis application, I
have to change the prefix for every log message.
Needless to say that will require some changes to the exiting parsing
rules in SEC.
My inital thought was that I could set up a single rule to parse the
new prefix and then use the results of that parsing in additional
rules via $+{named_variables}. This would let me do 3 things:
1) limit the number of places I need to write the prefix
regexp to extract fields (it may change again in the future)
2) only apply the regexp for the prefix extraction once
rather than dozens of times for each rule in the ruleset
3) as a result of 2 eliminate a performance bottleneck caused
by applying the somewhat complex prefix regexp for every rule
My initial setup was:
type = single
ptype = regexp
pattern = .....
varmap = pg_prefix; date=1; host=2; db=3; user=4; level=5; event=6; eorder=7
action = none
continue=takenext
type = single
pattern = LOG: duration: (\d+\.\d+)
context = !pg_log_event_$+{host}_$+{event} && =( $1 > 2 )
rem = here is an action specific to the duration log lines.
action add pg_log_event_$+{host}_$+{event} $0
and so forth. For a varmap without a pg_prefix; argument, the $+{}
variables aren't defined except in the scope of the rule with the
varmap. This makes sense as their underlying variable $1...$N only
have the scope of a single rule.
However, it turns out you can't access the $+{} variables in the
second rule even with the pg_prefix; argument unless the second
rule is of the form:
type = single
ptype = cached
pattern = pg_prefix
....
which doesn't let me perform additional matches to the event to detect
that it is a duration log event. I suppose I could add a context
=(return 1 if "LOG duration: (\d+\.\d+)" =~ $0 && $$1 > 2 )
and do the pattern matching/data extraction using a perl expression in
the context but eeek. (IIRC $$1 should be $1 from the =~ in the perl
expression as opposed to a $1 stored by sec.)
I suppose I could create contexts for each named variable using the
action field in the first rule. Then in the second rule copy the
contexts to variables and use them in further actions. But that seems
to be asking for performance trouble and bugs caused by inaccurate
housekeeping.
As I said at the start I am pretty sure there is a mechanism to
incrementally parse an event so each rule can apply a regexp to a part
of the event line and use the parsed results in other rules, but my
brain is just fixated on this method at the moment and not seeing the
proper way to do this.
Thanks for any clue by 4's.
--
-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
