Thank you Risto!. Fits Good and you gave me great ideas to program more
code for the future. I´m still studying sec possibilities and learning
everyday SEC.
Regards.
2016-10-04 17:33 GMT+02:00 Risto Vaarandi <risto.vaara...@gmail.com>:
> ...just one additional suggestion (I know, it is a very different
> approach from yours) -- in the case *all* relevant messages appear in
> sec input at some point of time, you could also address the problem
> with following sec rules:
>
> # This rule associates the Subject-line and the sender-line with the
> same MID value,
> # and generates a synthetic event that represents both events, for example:
> #
> # MID:52366 Domain:sec.com Subject:Hello World
> #
> # The rule assumes that Subject-line and sender-line for the same MID
> value can
> # appear in any order and are separated by at most 60 seconds
>
> type=EventGroup2
> ptype=RegExp
> pattern=email:Mid (\d+) sender:'\S+?@(\S+)'
> count=fill MID_$1_DOMAIN $2
> ptype2=RegExp
> pattern2=email:Mid (\d+) Subject:'(.+)'
> count2=fill MID_$1_SUBJECT $2
> desc=Associate the subject with the sender domain based on mid $1
> action=copy MID_$1_SUBJECT %subject; copy MID_$1_DOMAIN %domain; \
> event MID:$1 Domain:%domain Subject:%subject
> init=create MID_$1_DOMAIN; create MID_$1_SUBJECT
> end=delete MID_$1_DOMAIN; delete MID_$1_SUBJECT
> window=60
>
> # This rule counts synthetic events generated by counting operations
> of the previous rule,
> # and generates an alert if three messages are seen with the same
> subject lines and
> # the same sender domain within 1 hour (if the MID values have to be
> unique for counted
> # synthetic events, this rule can be easily elaborated into EventGroup
> rule with this functionality)
>
> type=SingleWithThreshold
> ptype=RegExp
> pattern=MID:\d+ Domain:(\S+) Subject:(.+)
> desc=Three messages from the same domain $1 with the same subject $2
> action=write - %s
> window=3600
> thresh=3
>
>
> I am not sure if the above suggestion fits into your environment, and
> if not, maybe it is useful for tackling some future event correlation
> task :)
>
> kind regards,
> risto
>
> 2016-10-04 15:11 GMT+03:00 Jaren Peich <burkol...@gmail.com>:
> > Sorry i missed to add. The idea is to find the first 3 mids code with
> the
> > same subject and then compare domain email senders of 3 codes searched.
> > If every domain from the sender is equal, print a message on the screen.
> In
> > the case that 1 of them is different just not to show the message.
> >
> > I find mid numbers with the rules and keep in a context and in the perl
> > code. Then extract mid numbers and find domain sender sender. Keep domain
> > sender in an array to compare after the search.
> >
> > I erased this part just to test:
> > if($arraymids[0] eq $arraymids[1] && $arraymids[0] eq $arraymids[2]){\
> > print "\n=======>SAME DOMAIN\n";\
> > }else{\
> > print "\n=======>Different DOMAIN\n";\
> > }\
> >
> > Regards Risto.
> >
> > 2016-10-04 13:55 GMT+02:00 Jaren Peich <burkol...@gmail.com>:
> >>
> >> Hi,
> >>
> >> In this case the rules are ordered but there are some cases, i´ve seen
> >> that there are disordered data and in other files. I´ve seen today. I
> have
> >> to do perl search.
> >>
> >> It can be as input also like this, its quite a pity:
> >>
> >> email:Mid 52365 sender:'ja...@sec.com'
> >> email:Mid 52366 sender:'s...@sec.com'
> >> email:Mid 52364 Subject:'Hello World'
> >> email:Mid 52366 Subject:'Hello World'
> >> email:Mid 52364 sender:'pe...@sec.com'
> >> email:Mid 52365 Subject:'Hello World'
> >>
> >>
> >> I´m trying to access the context data, i just only copy the action:
> >>
> >> action = copy SM_$+{subject} %hashmids; lcall %o %hashmids -> ( sub{\
> >> $i=0;\
> >> my(@arraymids)=();\
> >> %pmid=%{ $_[0] };\
> >> @midcont = values %pmid;\
> >> foreach my $lmid (@midcont) {\
> >> print "\nlmid===>$lmid\n" if defined($mid);\
> >> }\
> >> @files = <C:\\files\\m*.log>;\
> >> print "@files";\
> >> if(@files){\
> >> foreach $file (@files) {\
> >> open (FILE, "$file");\
> >> while($line= <FILE> ){\
> >> my ($mid, $email)= $line=~ m/email:Mid\s(\d+)\ssender\:\'
> ([^\']*)\'/gi;\
> >> print "\n====>Vuelta\n";\
> >> print "$midcont[0]";\
> >> foreach my $lmid (@midcont) {\
> >> print "\nMid fichero===>$mid\tlmid===>$lmid\n" if defined($mid);\
> >> if ($mid == $lmid){\
> >> my ($username, $domain) = $email =~ /(.*)@(.*)/;\
> >> print "\nDOMAIN FINDED===>$domain\n";\
> >> unshift(@arraymids, $domain);\
> >> }\
> >> }\
> >> }\
> >> }\
> >> }else{\
> >> print "========>NO FILES TO SEARCH";\
> >> }\
> >> }\
> >> );reset 0;
> >>
> >> Output is not as expected:
> >>
> >> Stdin connected to terminal, SIGINT can't be used for changing the
> logging
> >> level
> >> Creating SEC internal context 'SEC_INTERNAL_EVENT'
> >> Creating SEC internal event 'SEC_STARTUP'
> >> Deleting SEC internal context 'SEC_INTERNAL_EVENT'
> >> Creating context 'SM_Hello World'
> >> Adding event(s) '52364' to context 'SM_Hello World'
> >> Adding event(s) '52365' to context 'SM_Hello World'
> >> Adding event(s) '52366' to context 'SM_Hello World'
> >> Copying context 'SM_Hello World' to variable '%hashmids'
> >> Variable '%hashmids' set to '52364
> >> 52365
> >> 52366'
> >> Calling code 'CODE(0x2927f34)' and setting variable '%o'
> >> C:\log.log
> >> ====>Vuelta
> >> Use of uninitialized value $main::SEC::midcont[0] in string at (eval 4)
> >> line 1, <FILE> line 1.
> >>
> >> ====>Vuelta
> >> Use of uninitialized value $main::SEC::midcont[0] in string at (eval 4)
> >> line 1, <FILE> line 2.
> >>
> >> ====>Vuelta
> >> Use of uninitialized value $main::SEC::midcont[0] in string at (eval 4)
> >> line 1, <FILE> line 3.
> >> Variable '%o' set to ''
> >> Terminating event correlation operation 'C:\prueba.conf | 1 | create
> >> context by subject_Hello World'
> >> Creating SEC internal context 'SEC_INTERNAL_EVENT'
> >> Creating SEC internal event 'SEC_SHUTDOWN'
> >> Deleting SEC internal context 'SEC_INTERNAL_EVENT'
> >>
> >> I don´t know why i can´t access to the %hashmids that I copied before
> and
> >> added as a parameter. How can i reference it?
> >>
> >>
> >> Thank you for your help Risto. Regards.
> >>
> >>
> >> 2016-10-03 15:33 GMT+02:00 Risto Vaarandi <risto.vaara...@gmail.com>:
> >>>
> >>> hi Jaren,
> >>> let me add few suggestions below:
> >>>
> >>> >
> >>> > rem=create context and store first mid-count 2 and 3
> >>> > type=EventGroup1
> >>> > ptype=Cached
> >>> > pattern=SubjectMID
> >>> > context=SM_$+{subject}
> >>> > thresh=2
> >>> > window=60
> >>> > count=add SM_$+{subject} $+{mid}
> >>> > desc=create context by subject_$+{subject}
> >>> > end=delete SM_$+{subject}
> >>> > action = copy SM_$+{subject} %hashmids; lcall %o %hashmids -> ( \
> >>> > print "\nArrive!!!!!";\
> >>>
> >>> The Perl code that has been provided with 'lcall' action is not a
> >>> valid function definition, and you need to enclose the code in sub { }
> >>> construct. For example:
> >>>
> >>> lcall %o %hashmids -> ( sub { print "\nArrive!!!!!"; } )
> >>>
> >>> Also, the value of the %hashmids variable is not used in the function
> >>> (in the perl function, $_[0] variable refers to the first input
> >>> parameters.
> >>>
> >>> > Output:
> >>> >
> >>> > SEC (Simple Event Correlator) 2.6.2
> >>> > Reading configuration from C:\prueba.conf
> >>> >
> >>> > Rule in C:\prueba.conf at line 10: Eval 'print "\nArrive!!!!!";'
> didn't
> >>> > return a code reference: 1
> >>> > Rule in C:\prueba.conf at line 10: Invalid action list ' copy
> >>> > SM_$+{subject}
> >>> > %hashmids; lcall %o %hashmids -> ( print "\nArrive!!!!!";);reset 0; '
> >>>
> >>> you are seeing this error message since 'lcall' does not have a
> >>> correct function definition (see my previous remark).
> >>>
> >>> >
> >>> > Is it possible to pass to the perl function a changing context name
> >>> > depending on the $+{subject} variable and process this data as i told
> >>> > you
> >>> > before?
> >>>
> >>> you can do that, but then you would have to access SEC's internal
> >>> context data structure, in order to process the context event store
> >>> (the context name serves as a key into %main::context_list hash table,
> >>> and in this hash table). In my opinion, that is not a very clean way,
> >>> and passing a newline separated string of numerals into the function
> >>> is both simpler and more readable. Also, you wouldn't have any
> >>> dependencies on sec internals.
> >>>
> >>> Just out of curiosity -- is my understanding correct that the lines
> >>> you want to process are all appearing in the same input file:
> >>>
> >>> email:Mid 52364 Subject:'Hello World'
> >>> email:Mid 52365 Subject:'Hello World'
> >>> email:Mid 52366 Subject:'Hello World'
> >>> email:Mid 52366 sender:'s...@sec.com'
> >>> email:Mid 52365 sender:'ja...@sec.com'
> >>> email:Mid 52364 sender:'pe...@sec.com'
> >>>
> >>> If that is the case, it might be cheaper not to repeatedly search the
> >>> input file with Perl code which is expensive, but rather implement the
> >>> same logic with sec rules. What is your actual goal for extracting the
> >>> e-mail addresses based on subject lines? There might be a much shorter
> >>> and more elegant way for addressing this problem.
> >>>
> >>> kind regards,
> >>> risto
> >>>
> >>>
> >>> > One create a 'SM_Hello World', other email context can be 'SM_Hello
> >>> > from
> >>> > SEC', other one 'SM_Hallo Frank', 'SM_Julius notes', etc...
> >>> >
> >>> > Finally in the first email i just added the perl function that i
> >>> > designed to
> >>> > search into the files this data.
> >>> >
> >>> > This one(Not working, because i add notes text between ##########):
> >>> >
> >>> > eval %o ( \
> >>> > $i=0;\
> >>> > my(@arraymids)=();\
> >>> > ####################################\
> >>> > #Here i want to read again the context and extract the values. Add to
> >>> > the
> >>> > array for processing.\
> >>> > @midcont=("52366","52365","52364");\
> >>> > ###################################
> >>> > @files = <C:\\files\\m*.log>;\
> >>> > print "@files";\
> >>> > if(@files){\
> >>> > foreach $file (@files) {\
> >>> > open (FILE, "$file");\
> >>> > while($line= <FILE> ){\
> >>> > my ($mid, $email)= $line=~
> >>> > m/email:Mid\s(\d+)\ssender\:\'([^\']*)\'/gi;\
> >>> > print "\n====>Vuelta\n";\
> >>> > foreach my $lmid (@midcont) {\
> >>> > print "\nMid fichero===>$mid\tlmid===>$lmid\n" if defined($mid);\
> >>> > if ($mid == $lmid){\
> >>> > my ($username, $domain) = $email =~ /(.*)@(.*)/;\
> >>> > print "\nDOMAIN FINDED===>$domain\n";\
> >>> > unshift(@arraymids, $domain);\
> >>> > }\
> >>> > }\
> >>> > }\
> >>> > }\
> >>> > if($arraymids[0] eq $arraymids[1] && $arraymids[0] eq
> $arraymids[2]){\
> >>> > print "\n=======>SAME DOMAIN\n";\
> >>> > }else{\
> >>> > print "\n=======>Different DOMAIN\n";\
> >>> > }\
> >>> > }else{\
> >>> > print "========>NO FILES TO SEARCH";\
> >>> > }\
> >>> > );reset 0;
> >>> >
> >>> >
> >>> >
> >>> > Thank you Risto again. Regards.
> >>> >
> >>> > 2016-10-01 12:50 GMT+02:00 Risto Vaarandi <risto.vaara...@gmail.com
> >:
> >>> >>
> >>> >> 2016-09-30 12:20 GMT+03:00 Jaren Peich <burkol...@gmail.com>:
> >>> >> > Hi Risto,
> >>> >> >
> >>> >> > I have a little doubt that i haven´t seen in the documentation.
> >>> >> > I want to detect and keep some data from the logs and then read
> the
> >>> >> > context
> >>> >> > again and extract this data to process this info throught a perl
> >>> >> > function.
> >>> >> >
> >>> >>
> >>> >> ...
> >>> >>
> >>> >> > action = eval %o ( \
> >>> >> > $i=0;\
> >>> >> > my(@arraymids)=();\
> >>> >> > ####################################\
> >>> >> > #Here i want to read again the context and extract the values. Add
> >>> >> > to
> >>> >> > the
> >>> >> > array for processing.\
> >>> >> > @midcont=("52366","52365","52364");\
> >>> >> > ###################################
> >>> >>
> >>> >> I am still not sure whether I have understood the problem correctly,
> >>> >> but if you would like to access the event store of the SEC context
> >>> >> from a Perl function/expression, I would recommend to use the 'copy'
> >>> >> action before the Perl code is invoked. For example, if the event
> >>> >> store of context TEST contains events "test1", "test2" and "test3",
> >>> >> then the following action
> >>> >>
> >>> >> copy TEST %events
> >>> >>
> >>> >> will set the action list variable %events to the following string
> >>> >> "test1<NEWLINE>test2<NEWLINE>test3".
> >>> >> After the %events variable has been set, you can pass it as an input
> >>> >> parameter to the Perl function invoked with 'lcall' action. If you
> are
> >>> >> using 'eval', the %events variable can be directly accessed from the
> >>> >> Perl code, since it is substituted before each compilation and
> >>> >> execution. (Again, I would take the opportunity and recommend the
> use
> >>> >> of 'lcall' instead of 'eval', since 'lcall' is much more efficient.)
> >>> >>
> >>> >> kind regards,
> >>> >> risto
> >>> >
> >>> >
> >>
> >>
> >
>
------------------------------------------------------------------------------
The Command Line: Reinvented for Modern Developers
Did the resurgence of CLI tooling catch you by surprise?
Reconnect with the command line and become more productive.
Learn the new .NET and ASP.NET CLI. Get your free copy!
http://sdm.link/telerik
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
