On 01/06/2014 04:43 PM, termvrl term wrote: > Dear Risto, > > If i use SingleWithSuppress rule, can i count how many logs has been > suppressed in the time window? > Thanks
In order to configure this behavior, you can take advantage of the EventGroup rule which allows for executing an action at each matching event, and executing another action at the end of the correlation window. On each matching event, you can increase a Perl counter, and in the end of the correlation window, you can report the reading of this counter. Also, you have to set the 'thresh' parameter to 1, since this will make EventGroup rule essentially identical to SingleWithSuppress. Since this particular question has been discussed in the mailing list couple of times already, I'll provide you the links to relevant posts: http://sourceforge.net/p/simple-evcorr/mailman/message/29984254/ http://sourceforge.net/p/simple-evcorr/mailman/message/27913498/ kind regards, risto > > > > > On Fri, Dec 27, 2013 at 11:48 PM, Risto Vaarandi <risto.vaara...@seb.ee > <mailto:risto.vaara...@seb.ee>> wrote: > > hi, > > from your log sample it appears that you have pfsense version which > generates multiline messages, and relevant messages span over 2 > consecutive lines. For matching such events, you would need RegExp2 > pattern type. Also, are you interested in matching *all* pfsense > messages or just those which concern blocked packets? If you want to > process events for blocked packets only, your regular expression might > look like follows: > > ptype=RegExp2 > pattern=pf: [\d.:]+ rule [\d/]+\(match\): block .* proto > (\w+).*\n.*pf:\s+([\d.]+) > ([\d.]+) > > This regular expression will set $1 to protocol, $2 to source IP and > port, and $3 to destination IP and port. Note that in the case of ICMP, > the $2 and $3 variables will just hold IP addresses without port > numbers. However, if you would like to assign port numbers to separate > variables, you could also try: > > ptype=RegExp2 > pattern=pf: [\d.:]+ rule [\d/]+\(match\): block .* proto > (\w+).*\n.*pf:\s+((?:\d+\.){3}\d+)(?:\.(\d+))? > > ((?:\d+\.){3}\d+)(?:\.(\d+))? > > Also, you have to keep in mind that when you are doing multiline > matching, it is recommended to start SEC with the --nojointbuf flag > which will set up a separate input buffer for each input file. For > example: > > sec --conf=/etc/sec/pfsense.rules --input=/var/log/pf.log --nojointbuf > > Without the --nojointbuf flag, lines from several input files and > synthetic events would end up in a single input buffer which would break > multi-line matching. > > Also, have I understood correctly that you would like to report only the > first event for given source and destination endpoint, and suppress the > following ones? If my understanding is correct, the best way would be to > use SingleWithSuppress rule: > > type=SingleWithSuppress > ptype=RegExp2 > pattern=pf: [\d.:]+ rule [\d/]+\(match\): block .* proto > (\w+).*\n.*pf:\s+((?:\d+\.){3}\d+)(?:\.(\d+))? > > ((?:\d+\.){3}\d+)(?:\.(\d+))? > desc=Blocked packets for proto $1 srcIP $2 srcport $3 destIP $4 > destport $5 > action=write - %s > window=60 > > The above rule reacts to the first event for some protocol ID, source > and destination IP, and source and destination port, and ignores during > 60 seconds the following events with the *same* protocol, > source/destination IP, and source/destination port. > > However, this means that if from some IP address the same destination > service is probed, using a different source port number, another output > event is triggered. To prevent this from happening, you need to change > the 'desc' field of the rule and exclude source port number from it: > > type=SingleWithSuppress > ptype=RegExp2 > pattern=pf: [\d.:]+ rule [\d/]+\(match\): block .* proto > (\w+).*\n.*pf:\s+((?:\d+\.){3}\d+)(?:\.(\d+))? > > ((?:\d+\.){3}\d+)(?:\.(\d+))? > desc=Blocked packets for proto $1 srcIP $2 destIP $4 destport $5 > action=write - %s > window=60 > > Also, if you would like to do repeated event suppression on IP address > basis only, you could set the 'desc' field as follows: > > desc=Blocked packets for srcIP $2 destIP $4 > > Now you would be getting just one output event for each unique pair of > (sourceIP, destinationIP), no matter if the transport protocol or port > numbers change during the 60 second period. > > Hope this helps, > risto > > > On 12/26/2013 02:53 PM, termvrl term wrote: > > HI Thanks for your reply... > > > > I have another question, i am wondering what is the best rules > type for > > this situation, > > > > This is firewall logs form pfsense, > > > > Dec 17 23:25:05 192.168.0.120 pf: 00:00:13.146869 rule 62/0(match): > > block in on em0: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], > proto UDP > > (17), length 92) > > Dec 17 23:25:05 192.168.0.120 pf: 10.0.0.2.57185 > > 192.168.0.112.53: > > 26220 updateMA+ [b2&3=0x6f6f] [25600q] Type0 (Class 0)? . Type0 > (Class > > 0)? . Type0 (Class 0)? . Type0 (Class 0)? . Type0 (Class 0)? . Type0 > > (Class 0)? . Type0 (Class 0)? . Type0 (Class 0)? . Type0 (Class 0)? . > > Type0 (Class 0)? .[|domain] > > Dec 17 23:25:05 192.168.0.120 pf: 00:00:00.000101 rule 62/0(match): > > block in on em0: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], > proto UDP > > (17), length 92) > > Dec 17 23:25:05 192.168.0.120 pf: 10.0.0.2.57185 > > 192.168.0.112.53: > > 26220 updateMA+ [b2&3=0x6f6f] [25600q] Type0 (Class 0)? . Type0 > (Class > > 0)? . Type0 (Class 0)? . Type0 (Class 0)? . Type0 (Class 0)? . Type0 > > (Class 0)? . Type0 (Class 0)? . Type0 (Class 0)? . Type0 (Class 0)? . > > Type0 (Class 0)? .[|domain] > > > > I want to already have the regex pattern to match with, but i not > sure > > to use which rule type. > > > > I want the final output would be like this, > > > > Firewall Alert - UDP Blocked. Source IP : 10.0.0.2 Port: 57185 > > Destination IP: 192.168.0.112 Destination Port: 53 > > > > and can it be suppressed ? > > > > > > > > On Fri, Dec 20, 2013 at 4:20 AM, Risto Vaarandi > > <risto.vaara...@gmail.com <mailto:risto.vaara...@gmail.com> > <mailto:risto.vaara...@gmail.com <mailto:risto.vaara...@gmail.com>>> > wrote: > > > > I had a look into your logs and to me the EventGroup2 rule is > working > > exactly it is supposed to. First, the rule will match the > following > > event: > > > > Dec 16 23:04:53 ubuntu apache-errors: [Mon Dec 16 23:04:50 2013] > > [error] [client 192.168.0.119] ModSecurity: Warning. Pattern > match > > > > "(?i:([\\\\s'\\"`\\xc2\\xb4\\xe2\\x80\\x99\\xe2\\x80\\x98\\\\(\\\\)]*)?([\\\\d\\\\w]+)([\\\\s'\\"`\\xc2\\xb4\\xe2\\x80\\x99\\xe2\\x80\\x98\\\\(\\\\)]*)?(?:=|<=>|r?like|sounds\\\\s+like|regexp)([\\\\s'\\"`\\xc2\\xb4\\xe2\\x80\\x99\\xe2\\x80\\x98\\\\(\\\\)]*)?\\\\2|([\\\\s'\\"`\\xc2\\xb4\\xe2\\x80\\x99\\xe2\\x80\\x98\\ > > ..." at ARGS:name. [file > > > > "/etc/modsecurity/activated_rules/modsecurity_crs_41_sql_injection_attacks.conf"] > > [line "77"] [id "950901"] [rev "2.2.5"] [msg "SQL Injection > Attack"] > > [data "script>alert"] [severity "CRITICAL"] [tag > > "WEB_ATTACK/SQL_INJECTION"] [tag "WASCTC/WASC-19"] [tag > > "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/CIE1"] [tag "PCI/6.5.2"] > > [hostname "192.168.0.112"] [uri "/dvwa/vulnerabilities/xss_r/"] > > [unique_id "Uq-3kn8AAQEAAANQBJ8AAAAE"] > > > > After seeing this event, the rule starts an event correlation > > operation for the client 192.168.0.119 and uniqueID > > Uq-3kn8AAQEAAANQBJ8AAAAE, with the lifetime of the operation > being 60 > > seconds. > > The rule will then match further SQL events for the same > client and > > unique ID, and they will therefore be all correlated by the > previously > > started operation. > > Finally, the rule will match an XSS event for client > 192.168.0.119 and > > uniqueID Uq-3kn8AAQEAAANQBJ8AAAAE, which will also handled by the > > operation. At this point, the operation has seen at least 1 > instance > > of both events (described by pattern and pattern2) which will > trigger > > the execution of the rule action. After this, the rule will > continue > > running for the remainder of the 60 second correlation window, > > terminating at 23:05:53. During this time frame, any further > events > > are silently consumed by the operation, without triggering > any action. > > > > A similar event processing flow will take place at 23:11:43. > This is > > the reason why you are seeing only two "High probability XSS > attacks" > > events in the log. > > (Also, it seems to me that your clocks are not very well > synchronized > > across systems, because at some points in time simultaneous > events at > > different systems are more than 2 minutes apart by their > timestamps.) > > > > It seems to me that you have somehow misunderstood how > EventGroup rule > > operates. Have you looked into its documentation at > > http://simple-evcorr.sourceforge.net/man.html#lbAR ? This section of > > the docs contains an example how EventGroup rule can reset the > > counting process immediately after triggering the action. If > this is > > what you want to do, I'd recommend to have a look into this > example. > > > > regards, risto > > > > > > > > > > > > ------------------------------------------------------------------------------ > > Rapidly troubleshoot problems before they affect your business. > Most IT > > organizations don't have a clear picture of how application > performance > > affects their revenue. With AppDynamics, you get 100% visibility > into your > > Java,.NET, & PHP application. Start your 15-day FREE TRIAL of > AppDynamics Pro! > > > > http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk > > > > > > > > _______________________________________________ > > Simple-evcorr-users mailing list > > Simple-evcorr-users@lists.sourceforge.net > <mailto:Simple-evcorr-users@lists.sourceforge.net> > > https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users > > > > > > ------------------------------------------------------------------------------ > Rapidly troubleshoot problems before they affect your business. Most IT > organizations don't have a clear picture of how application performance > affects their revenue. With AppDynamics, you get 100% visibility > into your > Java,.NET, & PHP application. Start your 15-day FREE TRIAL of > AppDynamics Pro! > > http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk > _______________________________________________ > Simple-evcorr-users mailing list > Simple-evcorr-users@lists.sourceforge.net > <mailto:Simple-evcorr-users@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users > > > > > ------------------------------------------------------------------------------ > Rapidly troubleshoot problems before they affect your business. Most IT > organizations don't have a clear picture of how application performance > affects their revenue. With AppDynamics, you get 100% visibility into your > Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! > http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk > > > > _______________________________________________ > Simple-evcorr-users mailing list > Simple-evcorr-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users > ------------------------------------------------------------------------------ Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk _______________________________________________ Simple-evcorr-users mailing list Simple-evcorr-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
