hi,
yes, SEC is using the Perl dialect of the regular expression language.
Also, from your letter it seems that you would like to lessen the number of
alerts, and somehow aggregate several alerts into one alert which could be
reported to the SIEM.
SEC allows to implement many different aggregation schemes, but in order to
implement them, there must be a clear understanding what exactly to
aggregate and what is the common identifier (or common identifiers) for the
aggregated events.
>From your log example it seems that there are several fields in messages
which could take this role:
1) client (or hostname) -- seems to contain the IP address of the remote
node
2) unique_id -- in your example, all the events have this field set to the
same value. Is this field somehow identifying the client request? if so,
you could take advantage of this.
Once you have identified field(s) of interest which can be used for
aggregation, there must be clear understanding what exactly you want to for
achieving aggregation. For example, it is fairly easy to write any of the
following:
1) a rule that reacts to the first event where field(s) have some values,
and suppresses the following events with same values during, say, 1 minute
2) a counting rule that reacts if N or more events have been seen during
some time window
3) a rule that aggregates repeated events during some time window, and
sends them into a SIEM in a single message
etc. etc.
In order to provide one example, you could have the following simplistic
rule:
type=SingleWithSuppress
ptype=RegExp
pattern=\[client ([\d.]+)\].+\[unique_id "(.+?)"\]
desc=client $1 unique_id $2
action=pipe 'detected alert(s) for client $1 id $2' /bin/mail root
window=60
This rule will match the client IP and unique_id from a message, send an
e-mail warning to the root, and then suppress during 60 seconds the
following events with the same IP address *and* unique_id. The event
suppression by distinct (clientIP, unique_id) value pairs is achieved by
extracting IP address and unique_id from each event, and setting the
suppression scope properly with the 'desc' field.
The above rule is just one example out of many possible rules -- but in
order to write them down, it must be known what fields are relevant in the
events, and how to use them for aggregation. After you have clarified this
question, it is much more easy to provide suggestions and advise.
hope this helps,
risto
2013/12/2 termvrl term <term...@gmail.com>
> Hi All, Thanks for your reply.
>
> ModSecurity is an open source web application firewall based on rules, so
> it generate logs each time match with its rules.
>
> For one type of attacks it will matched with multiple rules up to 4-5
> pattern matched.
>
> Example, SQL attacks, will match at least 4 rules from
> "sql_injestion.rules". So, i will receive 4 line of logs in my SIEM for the
> same event.
>
> Unfortunately, for a some type of attacks it will match multiple type of
> attacks rules, For example, when i try to do XSS attack, the rules matched
> is a lot, it combine rules from SQL and XSS. Futher checking, i found out
> that, the special character use in XSS attacks matched with some SQL rules.
>
> So, i want to use SEC to correlate it just to generate a single Alert send
> to SIEM.
>
> Hi Tim, i have try your conf and its work like i want.
> But i have one request, can we log it only one time?
>
> here is my output,
> root@ubuntu:/home/term/sec-2.7.2# perl sec -conf=tim.conf
> -input=/var/log/apache2/error.log
> SEC (Simple Event Correlator) 2.7.2
> Reading configuration from tim.conf
> 2 rules loaded from tim.conf
> Opening input file /var/log/apache2/error.log
> Stdin connected to terminal, SIGINT can't be used for changing the logging
> level
> Writing event 'SQL rule matched' to file -
> SQL rule matched
> Creating context 'SQL_INJECTION_ATTACK'
> Writing event 'SQL rule matched' to file -
> SQL rule matched
> Creating context 'SQL_INJECTION_ATTACK'
> Writing event 'SQL rule matched' to file -
> SQL rule matched
> Creating context 'SQL_INJECTION_ATTACK'
> Writing event 'SQL rule matched' to file -
> SQL rule matched
> Creating context 'SQL_INJECTION_ATTACK'
> Writing event 'XSS matched' to file secoutput.txt
> Writing event 'XSS matched' to file secoutput.txt
> Writing event 'XSS matched' to file secoutput.txt
> Writing event 'XSS matched' to file secoutput.txt
> Deleting stale context 'SQL_INJECTION_ATTACK'
> Stale context 'SQL_INJECTION_ATTACK' deleted
>
>
> Hi Risto,
> you correct with the regex, the regex its not detect it.
> And as for the regex use in SEC, may i know what type of regex style it
> use? is it Perl-style regex?
>
> Thanks. Sorry if this is too long.
>
> Term
>
>
> On Mon, Dec 2, 2013 at 3:28 AM, Risto Vaarandi
> <risto.vaara...@gmail.com>wrote:
>
>> 2013/12/1 termvrl term <term...@gmail.com>
>>
>>> Hi all,
>>>
>>> i have working on correlate the alert from modsecurity.
>>> when i simulate XSS attacks, modsec will generate alert and it will
>>> match with SQL rule and XSS rule. So, i want to use SEC to correlate if
>>> detect both signature then, use write action to log a new message. Here is
>>> conf file.
>>>
>>> # Rule to match XSS attack.
>>> # SQL + XSS
>>>
>>> type=Pair
>>> ptype=RegExp
>>> pattern=sql_injection_attacks
>>> desc=$0
>>> action=write - SQL rule matched
>>> ptype2=RegExp
>>> pattern2=xss_attacks\s*CRITICAL
>>> desc2=$0
>>> action2=write - XSS matched
>>> window=5
>>>
>>> The problem is it detect only the first pattern, and the second pattern
>>> never matched. Is it because the modsecurity generate it with same
>>> timestamp?
>>>
>>> Attach is the sample log that i want to correlate.
>>>
>>> Thanks
>>> Term
>>>
>>
>> hi,
>> the Pair rule works as follows -- if an event is observed which matches
>> the pattern given with the 'pattern' field, a waiting operation is started
>> which gets its ID from the 'desc' field, rule file name, and rule offset in
>> the file (the use of the rule file name and offset ensures that operation
>> IDs do not clash for different rules).
>> The waiting operation started by the Pair rule runs for the 'window'
>> seconds, and expects to see an event which is matched by 'pattern2'. Your
>> sample rule assumes the following:
>> 1) first an event which matches a regular expression
>> sql_injection_attacks must come in,
>> 2) the incoming event starts an operation which has the ID containing the
>> entire matching line (held by $0 variable),
>> 3) the operation runs for 5 seconds and waits an event matching the
>> regular expression xss_attacks\s*CRITICAL (this regular expression means
>> "the string xss_attacks, followed by 0 or more whitespace characters,
>> followed by the string CRITICAL").
>>
>> Generally, it's a bad idea to set desc to $0, since entire matching line
>> contains a timestamp, which starts a new waiting operation each time a
>> matching line with a newer timestamp is observed. Are you actually trying
>> to do processing on the client IP address basis? If so, your regular
>> expressions should match the IP address from incoming events and assign it
>> to a match variable, and this variable should be used in the 'desc' field.
>> In that way, you run separate event correlation operations for distinct IP
>> addresses (also, have a look into the relevant section in the official
>> documentation: http://simple-evcorr.sourceforge.net/man.html#lbAW)
>>
>> Also, the Pair rule assumes that events are ordered. If this is not the
>> case, I would recommend to consider the EventGroup2 rule. Finally, your
>> second regular expression xss_attacks\s*CRITICAL assumes that only
>> whitespace can appear in between "xss_attacks" and "CRITICAL", but your log
>> messages seem to be different. This is probably the reason why the second
>> pattern of your Pair rule fails to match incoming events.
>>
>> hope this helps,
>> risto
>>
>>
>>
>>
>>
>>>
>>> ------------------------------------------------------------------------------
>>> Rapidly troubleshoot problems before they affect your business. Most IT
>>> organizations don't have a clear picture of how application performance
>>> affects their revenue. With AppDynamics, you get 100% visibility into
>>> your
>>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of
>>> AppDynamics Pro!
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Simple-evcorr-users mailing list
>>> Simple-evcorr-users@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>>>
>>>
>>
>
------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
