hi Dusan,
I am glad the ruleset is working now :)
Also, I wish you and all list members a happy new year!
kind regards,
risto
2017-01-01 14:44 GMT+02:00 Dusan Sovic <dusan.so...@hotmail.sk>:
> Hi Risto,
>
>
> Thank you very much for your answer!
>
> The use of the [ ] operator is the key and make my complex rule-set to
> work in way I want.
>
>
> Thanks & Best Regards,
>
> Dusan
>
> ------------------------------
> *Od:* Risto Vaarandi <risto.vaara...@gmail.com>
> *Odoslané:* 31. decembra 2016 10:51
> *Komu:* Dusan Sovic
> *Kópia:* simple-evcorr-users@lists.sourceforge.net
> *Predmet:* Re: [Simple-evcorr-users] Content of pattern match cache after
> synthetic event injection
>
> hi Dusan,
>
> you have asked an excellent question. Behavior you are seeing is
> actually something expected, since pattern match caching is done after
> a successful RegExp pattern match, but *before* the 'context' field of
> the rule definition is evaluated. It is also discussed in the
> documentation of the varmap statement, and in the man page section
> INPUT PROCESSING AND TIMING there is also a relevant rule example. In
> the case of the example that you provided, the following will happen
> after the synthetic event has been injected:
>
> 1) the 'pattern' field of the first rule will match the synthetic
> event, since the regular expression (?<EVENT>\S+)
> (?<TYPE>Problem|Resolution) produces a match
> 2) the 'varmap' statement creates a pattern match cache entry MY_EVENT
> 3) the 'context' field of the rule evaluates false, and therefore the
> action "write - R1: Parsing my event" is not executed
> 4) since the rule 'continue' field is set to TakeNext, the synthetic
> event is passed to following rules for further matching, and therefore
> another pattern match cache entry SYNTHETIC_EVENT will be created
>
> This behavior of sec is not a bug but intentional, since it allows to
> cache successful pattern matches at the earliest possible opportunity.
> If you have several rules with identical patterns but different
> context expressions, you can cache the match result in the first rule
> and reuse it in all following rules.
>
> Fortunately, there is also a fix for your ruleset and it involves the
> use of the [ ] operator -- if the context expression is enclosed in
> square brackets [ ], then it is evaluated *before* the pattern match.
> Therefore, if the context expression evaluates false, the 'pattern'
> field of the rule will not be tried, and consequently the 'varmap'
> statement will not be triggered. If we modify the first two rules of
> your example accordingly:
>
>
> rem=Rule 1
> type=Single
> ptype=RegExp
> pattern=(?<EVENT>\S+) (?<TYPE>Problem|Resolution)
> varmap=MY_EVENT
> context=[ !_INTERNAL_EVENT ]
> continue=TakeNext
> desc=Parse My Event
> action=write - R1: Parsing my event
>
> rem=Rule 2
> type=Single
> ptype=RegExp
> pattern=SYNTHETIC (?<EVENT>\S+) (?<TYPE>Problem|Resolution)
> varmap=SYNTHETIC_EVENT
> context=[ _INTERNAL_EVENT ]
> continue=TakeNext
> desc=Parse Synthetic Event
> action=write - R2: Parsing synthetic event
>
>
> then the ruleset starts to work in the way we want:
>
> R1: Parsing my event
> R4: Injecting synthetic event
> R2: Parsing synthetic event
> R5: SYNTHETIC Event1 Resolution
>
> I hope my answer was helpful, and thanks for posting such an
> interesting question to the mailing list!
>
> kind regards,
> risto
>
> 2016-12-30 18:20 GMT+02:00 Dusan Sovic <dusan.so...@hotmail.sk>:
> > Hello,
> > In my SEC rules I using pattern match cache. I would like to know is the
> pattern match cache content after injection of synthetics event. Is there
> any possibility to clear record from pattern match cache on demand?
> >
> > Consider the following SEC rule config (t.sec) :
> > ----------------------------------------------------
> > rem=Rule 1
> > type=Single
> > ptype=RegExp
> > pattern=(?<EVENT>\S+) (?<TYPE>Problem|Resolution)
> > varmap=MY_EVENT
> > context=!_INTERNAL_EVENT
> > continue=TakeNext
> > desc=Parse My Event
> > action=write - R1: Parsing my event
> >
> > rem=Rule 2
> > type=Single
> > ptype=RegExp
> > pattern=SYNTHETIC (?<EVENT>\S+) (?<TYPE>Problem|Resolution)
> > varmap=SYNTHETIC_EVENT
> > context=_INTERNAL_EVENT
> > continue=TakeNext
> > desc=Parse Synthetic Event
> > action=write - R2: Parsing synthetic event
> >
> > rem=Rule 3
> > type=Single
> > ptype=Cached
> > pattern=MY_EVENT
> > context=MY_EVENT :> ( sub { return $_[0]->{"TYPE"} eq "Problem"; } )
> > desc=Problem_$+{EVENT}
> > action=write - R3: Problem: $+{EVENT}
> >
> > rem=Rule 4
> > type=Single
> > ptype=Cached
> > pattern=MY_EVENT
> > context=MY_EVENT :> ( sub { return $_[0]->{"TYPE"} eq "Resolution"; } )
> > desc=Resolution_$+{EVENT}
> > action=event 0 SYNTHETIC $0; write - R4: Injecting synthetic event
> >
> > rem=Rule 5
> > type=Single
> > ptype=Cached
> > pattern=SYNTHETIC_EVENT
> > context=SYNTHETIC_EVENT :> ( sub { return $_[0]->{"TYPE"} eq
> "Resolution"; } )
> > desc=Resolution_$+{EVENT}
> > action=write - R5: $0
> >
> > Run the sec instance:
> > sec -input=- -conf=./t.sec -intevents -intcontexts
> >
> >
> > and put this input event:
> > Event1 Resolution
> >
> > SEC will match:
> > * Rule 1
> > * Rule 4 -> inject synthetic event
> > * Rule 2
> > * Rule 4
> > * Rule 2
> > * Rule 4
> > * etc.
> >
> > I would expect that after synthetic event injection (2nd rule), sec will
> match 5th rule.
> > As from doc: "Note that before processing each new input line, previous
> content of the pattern match cache is cleared."
> >
> > Instead of, it will match 2-4-2-4..etc rules.
> > This means, that after first synthetics event injection (4th rule) and
> match by 2nd rule, pattern match cache must contains two match records:
> "MY_EVENT" and "SYNTHETIC_EVENT".
> > Therefore rule 5 never match.
> >
> > One solution what I see is to add additional context test in rules
> 3-4-5 for presence of "_INTERNAL_EVENT" context.
> > Question is, if is possible to "somehow" clear the content or given
> record from pattern cache after synthetics event injection to allow rule5
> match?
> >
> > Thanks,
> > Dusan
> >
> > ------------------------------------------------------------
> ------------------
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> > _______________________________________________
> > Simple-evcorr-users mailing list
> > Simple-evcorr-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
> Simple-evcorr-users Info Page - SourceForge
> <https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users>
> lists.sourceforge.net
> To see the collection of prior postings to the list, visit the
> Simple-evcorr-users Archives. Using Simple-evcorr-users: To post a message
> to all the ...
>
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Simple-evcorr-users mailing list
> Simple-evcorr-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
