Risto-
Thank you for taking time to respond so thoroughly. Your examples were quite
clear. I would not have thought to try to use global variables. Ultimately I
chose your cached/context approach, which worked great.
I ended up taking a monolithic config file of nearly 400 hard to
maintain/organize rules and spread them more logically across 29 (and counting)
configuration files. The impetus was needing to re-use some (but not all) of
the rules in a second network that I admin.
In an attempt to give back, I included below my general approach in case others
find it useful.
Thanks again,
-Michael
==============/=============
# 000-main.config
# this file will be the same on each network
type=Single
ptype=SubStr
pattern=SEC_STARTUP
desc=Started SEC
action=assign %a /destinationfile.log
type=Single
ptype=RegExp
pattern=^\w+\s+[0-9]+ [0-9]+:[0-9]+:[0-9]+ (.+?) .+
varmap=SYSLOG; hostname=1
desc=hostname
action=none
continue=TakeNext
type=Jump
ptype=cached
pattern=SYSLOG
context=$+{hostname} -> ( sub { return 1 if $_[0]; return 0 } )
cfset=001-all
continue=EndMatch
# This is a catch-all rule to dump to the logfile anything that didn't match
above..
type=Single
ptype=RegExp
pattern=.*
desc=$0
action=write %a $0
---------/--------
#001-all.config
# this file will be different between networks due to differences in vendors,
device naming standards, etc
type=Options
joincfset=001-all
procallin=no
type=Jump
ptype=cached
pattern=SYSLOG
context=$+{hostname} -> ( sub { return 1 if $_[0] =~ m/^r-/ } )
cfset=050-juniper
continue=EndMatch
type=Jump
ptype=cached
pattern=SYSLOG
context=$+{hostname} -> ( sub { return 1 if $_[0] =~ m/^[ts]-/ } )
# This is a catch-all rule to dump to the logfile anything that doesn't match
above..
type=Single
ptype=RegExp
pattern=.*
desc=$0
action=write %a $0
---------/--------
# 050-juniper.config
# these files will be the same between networks, and where most of the rules
and re-use will come in.
type=Options
joincfset=050-juniper
procallin=no
# near top due to frequency..
type=Jump
ptype=RegExp
pattern=.+
cfset=110-juniper-mx104
continue=TakeNext
# two examples, I have many stanzas like this for individual JunOS daemons
[hence many files]
type=Jump
ptype=RegExp
pattern= mgd\[[0-9]+\]:
cfset=150-juniper-mgd
continue=EndMatch
type=Jump
ptype=RegExp
pattern= rpd\[[0-9]+\]:
cfset=150-juniper-rpd
continue=EndMatch
# things that don’t match specific daemons
type=Jump
ptype=RegExp
pattern=.+
cfset=100-juniper-nodaemon
continue=TakeNext
# This is a catch-all rule to dump to the logfile anything that survived
type=Single
ptype=RegExp
pattern=.*
desc=$0
action=write %a $0
[FIN]
From: risto.vaara...@gmail.com <risto.vaara...@gmail.com>
Sent: Saturday, October 17, 2020 5:13 PM
To: Michael Hare <michael.h...@wisc.edu>
Cc: simple-evcorr-users@lists.sourceforge.net
Subject: Re: [Simple-evcorr-users] using variables learned in rule A in rule
B's perlfunc: possible?
hi Michael,
there are a couple of ways to address this problem. Firstly, instead of using
sec match variables, one can set up Perl's native variables for sharing data
between rules. For example, the regular expression pattern of the first rule
can be easily converted into perlfunc pattern, so that the pattern would assign
the hostname to Perl global variable $hostname. This global variable can then
be accessed in perfunc patterns of other rules. Here is an example that
illustrates the idea:
type=Single
ptype=perlfunc
pattern=sub { if ($_[0] =~ /^\w+\s+[0-9]+ [0-9]+:[0-9]+:[0-9]+ (.+?) /) \
{ $hostname = $1; } else { $hostname = ""; } return 1; }
desc=hostname
action=none
continue=TakeNext
type=Jump
ptype=perlfunc
pattern=sub { return 1 if $hostname =~ m/^first-use-case/ }
cfset=rules-for-this-match-1
type=Jump
ptype=perlfunc
pattern=sub { return 1 if $hostname =~ m/^second-use-case/ }
cfset=rules-for-this-match-2
Since SEC supports caching the results of pattern matching, one could also
store the matching result from the first rule into cache, and then retrieve the
result from cache with the 'cached' pattern type. Since this pattern type
assumes that the name of the cached entry is provided in the 'pattern' field,
the hostname check with a perl function has to be implemented in a context
expression (the expression is evaluated after the pattern match).
Here is an example which creates an entry named SYSLOG in a pattern match
cache, so that all match variables created in the first rule can be retrieved
in later rules. Note that the entry is created in the 'varmap' field which also
sets up named match variable $+{hostname}. In further rules, the 'cached'
pattern type is used for retrieving the SYSLOG entry from cache, and creating
all match variables from this entry. In order to check the hostname, the
$+{hostname} variable that was originally set in the first rule is passed into
perlfunc patterns in the second and third rule. Also, if you need to check more
than just few match variables in perlfunc pattern, it is more efficient to pass
a reference to the whole cache entry into the Perl function, so that individual
cached match variables can be accessed through the reference (I have added an
additional fourth rule into the example which illustrates this idea):
type=Single
ptype=RegExp
pattern=^\w+\s+[0-9]+ [0-9]+:[0-9]+:[0-9]+ (.+?) .+
varmap=SYSLOG; hostname=1
desc=hostname
action=none
continue=TakeNext
type=Jump
ptype=cached
pattern=SYSLOG
context=$+{hostname} -> ( sub { return 1 if $_[0] =~ m/^first-use-case/ } )
cfset=rules-for-this-match-1
type=Jump
ptype=cached
pattern=SYSLOG
context=$+{hostname} -> ( sub { return 1 if $_[0] =~ m/^second-use-case/ } )
cfset=rules-for-this-match-2
type=Jump
ptype=cached
pattern=SYSLOG
context=SYSLOG :> ( sub { return 1 if $_[0]->{"hostname"} =~ m/^third-use-case/
} )
cfset=rules-for-this-match-3
A small side-note about the first rule -- you can also create the $+{hostname}
variable in the regular expression itself by rewriting it as follows:
^\w+\s+[0-9]+ [0-9]+:[0-9]+:[0-9]+ (?<hostname>.+?) .+
And in that case, you can rewrite the 'varmap=SYSLOG; hostname=1' statement in
the first rule simply as 'varmap=SYSLOG'.
Hopefully these examples are useful and help you to tackle the problem.
kind regards,
risto
Hi-
I'm sorry to ask what is probably very basic question, but I have struggling
with this for awhile (I have perused the manual a lot and the mailing list a
bit) and could use some guidance.
The short version is: is there a way to take the results of a pattern match in
one rule and use that value in a perlfunc in another?
More verbosely, at this time I use SEC for network syslog exclusion; nothing
fancy. I would like to start using Jump rules based on hostname. Hostname is
derived from the incoming log line.
I thought I would be clever and use a single rule to determine if there was a
hostname or not, save it somewhere reusable, and then launch jump rules based
on that.
something like
type=Single
ptype=RegExp
pattern=^\w+\s+[0-9]+ [0-9]+:[0-9]+:[0-9]+ (.+?) .+
varmap= hostname=1
desc=hostname
action=assign %r $+{hostname}
continue=TakeNext
type=Jump
ptype=perlfunc
pattern=sub { return 1 if $+{hostname} =~ m/^first-use-case/ }
cfset=rules-for-this-match-1
type=Jump
ptype=perlfunc
pattern=sub { return 1 if $+{hostname} =~ m/^second-use-case/ }
cfset=rules-for-this-match-2
I know this doesn't work. I understand that '%r' is not a perl hash, and is an
action list variable, and that $+{hostname} is undef inside the type=Jump rule
perlfunc. I also know that %r is being set correctly, I see it in "variables
-> r" if I do SIGUSR1 dump.
So is it possible stash away a variable from one rule and use it in a Jump rule
like above? I can work around this easily by using a single rule like below,
but if I have for example 20 jump permutations, it seems quite redundant to
keep recalculating the hostname for comparison.
type=Jump
ptype=perlfunc
pattern=sub { return 0 unless (defined($_[1]) && $_[0] =~ /^\w+\s+[0-9]+
[0-9]+:[0-9]+:[0-9]+ (.+?) .+/); return 1 if $1 =~ m/^first-use-case/}
cfset=all-rules
Thanks in advance,
-Michael
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net<mailto:Simple-evcorr-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
