On 11/26/2012 06:36 PM, John P. Rouillard wrote:
>
> In message<50b33ca3.7050...@seb.ee>,
> Risto Vaarandi writes:
>> the first alpha version of SEC-2.7 has been released and is available at
>> http://sourceforge.net/projects/simple-evcorr/files/sec/2.7.alpha1/sec-2.7.a
>> lpha1.tar.gz/download
>
> Good news.
>
>> It has quite many new features, including 14 new actions which allow for
>> more advanced operations on contexts, but also conditional execution
>> with if-action and loops with while-action.
>
> Ooh nice.
>
>> PerlFunc patterns have been augmented with an opportunity to set named
>> match variables. For all pattern types which set match variables,
>> previously cached matches can be accessed with $:{entry:var} syntax.
>
> Will this allow the incremental parsing I was discussing on the list?
>
> type = single
> cont = takenext
> desc = parse standard syslog info
> rem = 2011-11-06T00:29:07.388514+00:00 rtr01 sshd[3456]:
> ptype = regexp
> pattern = '^.{32} (\w+) (\w+)\[(\d+)\]:
> varmap = syslog; host=1; service=2; pid=3; line=0
> action = none
>
>
> type = single
> desc = log all rtr1 failing sshd entries
> ptype = substr
> pattern = fail
> context = =($:{syslog:host} eq "rtr01")
> action = write sshd_failure_log $:{syslog:line}
>
> will do the "right thing" 8-). Also will this work if they are in two
> different files (e.g. if the first rule resulted in a jump to say
> $2_rules.sr).
It will almost work in this way -- you only have to change ptype of the
second rule from 'substr' to 'regexp'.
For the reasons of efficiency, substring and truth value patterns do not
set match variables. On the other hand, substitution of regular match
variables ($N and $+{name}) and exported match variables $:{entry:name}
is done in one go. (If it was to done separately, substituted values
would have to go through a separate masking procedure for $ and %, in
order to prevent side effects during substitutions, and this would add
another layer of complexity and impose additional computing overhead.)
Fortunately enough, it is trivial to write any substr pattern as regexp,
if match variables need to be set.
Also, the second rule would be much more efficient if context expression
is written as
context= $:{syslog:host} -> sub { $_[0] eq "rtr01" }
or
context=syslog :> sub { $_[0]->{"host"} eq "rtr01" }
since now we are avoiding the compilation before each execution which is
very inefficient. The last alternative with :> is the fastest, since now
we are avoiding parameter substitution altogether, and are passing a
reference to the set of cached variables to the matching function.
If we would like to verify that the 'syslog' entry really exists in the
cache, we could rewrite the context expression as
context=varset syslog && syslog :> sub { $_[0]->{"host"} eq "rtr01" }
(the 'varset' operator checks if the entry with the following name is
present in the cache.)
>
>> Not all features which I had in mind or which were discussed in the
>> mailing list were implemented. In fact, when writing the first alpha
>> release, I created 3-4 different implementations with different feature
>> sets enabled, and benchmarked them against larger event volumes. The
>> testing clearly revealed that some features would decrease the
>> performance of the tool by 10-15%, and enabling them all would impose a
>> hard penalty on all users. Therefore I opted a more conservative
>> approach and implemented only those things which add true value to the
>> end user without sacrificing performance.
>
> Is there a chnagelog and examples available separately from the
> download so I know what new things to look for?
>
There is a ChangeLog file both in the tarball and in the SourceForge
download repository.
However, the most notable changes in context expressions are the ones
which I outlined above. In addition to that, most notable changes are a
number of actions for getting and setting the context parameters, adding
data to the *beginning* of context event store, and popping and shifting
data from context event stores. Finally, you have the ability for if and
while statements.
To summarize many of new actions into one list, consider an example from
updates man page:
action=getsize %size C1; while %size ( shift C1 %event; \
lcall %nocomment %event -> ( sub { $_[0] !~ /^#/ } ); \
if %nocomment ( add C1 %event ); \
lcall %size %size -> ( sub { $_[0]-1; } ) )
This action list will take the context C1 and remove all comment lines
from its event store.
The following action list creates the context C if it does not exist,
and writes a warning to to stdout otherwise:
action=exists %o C; if %o ( write - context C already exists ) else (
create C )
For making if and while actions efficient, they operate on action list
variables which makes the condition evaluation very fast (you only need
to fetch the variable value from a hash table). Having arbitrary strings
in that rule would make evaluation slower and more complex, since
'while' would require a separate substitution procedure for each loop
iteration. The introduction of 'lcall' action into the previous release
was done with 'if' and 'while' in mind, since with 'lcall' you can
efficiently evaluate any expression, which can then be provided to 'if'
or 'while' via an action list variable.
kind regards,
risto
------------------------------------------------------------------------------
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
