On Mon, 13 Aug 2012, John P. Rouillard wrote:
> Hi all:
>
> I know this should be doable, and I know my current thinking is
> barking up the wrong tree, but my brain is locked into doing something
> and it's not quite working.
>
> Here's what happened. In order to get the output from postgres into
> the proper form to be used by an performance analysis application, I
> have to change the prefix for every log message.
>
> Needless to say that will require some changes to the exiting parsing
> rules in SEC.
>
> My inital thought was that I could set up a single rule to parse the
> new prefix and then use the results of that parsing in additional
> rules via $+{named_variables}. This would let me do 3 things:
>
> 1) limit the number of places I need to write the prefix
> regexp to extract fields (it may change again in the future)
> 2) only apply the regexp for the prefix extraction once
> rather than dozens of times for each rule in the ruleset
> 3) as a result of 2 eliminate a performance bottleneck caused
> by applying the somewhat complex prefix regexp for every rule
>
> My initial setup was:
>
> type = single
> ptype = regexp
> pattern = .....
> varmap = pg_prefix; date=1; host=2; db=3; user=4; level=5; event=6; eorder=7
> action = none
> continue=takenext
>
> type = single
> pattern = LOG: duration: (\d+\.\d+)
> context = !pg_log_event_$+{host}_$+{event} && =( $1 > 2 )
> rem = here is an action specific to the duration log lines.
> action add pg_log_event_$+{host}_$+{event} $0
>
> and so forth. For a varmap without a pg_prefix; argument, the $+{}
> variables aren't defined except in the scope of the rule with the
> varmap. This makes sense as their underlying variable $1...$N only
> have the scope of a single rule.
>
> However, it turns out you can't access the $+{} variables in the
> second rule even with the pg_prefix; argument unless the second
> rule is of the form:
>
> type = single
> ptype = cached
> pattern = pg_prefix
> ....
>
> which doesn't let me perform additional matches to the event to detect
> that it is a duration log event. I suppose I could add a context
>
> =(return 1 if "LOG duration: (\d+\.\d+)" =~ $0 && $$1 > 2 )
>
> and do the pattern matching/data extraction using a perl expression in
> the context but eeek. (IIRC $$1 should be $1 from the =~ in the perl
> expression as opposed to a $1 stored by sec.)
>
> I suppose I could create contexts for each named variable using the
> action field in the first rule. Then in the second rule copy the
> contexts to variables and use them in further actions. But that seems
> to be asking for performance trouble and bugs caused by inaccurate
> housekeeping.
>
> As I said at the start I am pretty sure there is a mechanism to
> incrementally parse an event so each rule can apply a regexp to a part
> of the event line and use the parsed results in other rules, but my
> brain is just fixated on this method at the moment and not seeing the
> proper way to do this.
>
> Thanks for any clue by 4's.
Why are you trying to use $+{} instead of just regular variables?
you can create a variable/hash $event{} and just tinker with it through
multiple rules (just be sure that it's always cleared before some other
rule tries to use it)
David Lang
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
