On 12/10/2013 05:40 PM, termvrl term wrote:
Hi Risto,
Thanks for your helps,
I have tried with your suggestion with a bit changes to the sec rule.
type=SingleWithSuppress
ptype=RegExp
pattern=\[client ([\d.]+)\].+\[msg "(.*?)"\].+\[unique_id "(.+?)"\]
desc=client $1 unique_id $2
action= write secoutput.txt Alert - From $1 Type of Attack - $2 with
unique id $3
window=60
As i managed to get the output (as attached file, secoutput.txt), is it
possible if we aggregate the similar type of attacks alert with the same
host and unique id into a single alert? For example, This is for SQL
injection logs:-
Alert - From 192.168.0.119 Type of Attack - SQL Injection Attack with
unique id Uqcub38AAQEAAAUsQ5kAAAAG
Alert - From 192.168.0.119 Type of Attack - SQL Character Anomaly
Detection Alert - Repetative Non-Word Characters with unique id
Uqcub38AAQEAAAUsQ5kAAAAG
Alert - From 192.168.0.119 Type of Attack - Restricted SQL Character
Anomaly Detection Alert - Total # of special characters exceeded with
unique id Uqcub38AAQEAAAUsQ5kAAAAG
Alert - From 192.168.0.119 Type of Attack - Detects basic SQL
authentication bypass attempts 1/3 with unique id
Uqcub38AAQEAAAUsQ5kAAAAG
Alert - From 192.168.0.119 Type of Attack - Detects MSSQL code execution
and information gathering attempts with unique id
Uqcub38AAQEAAAUsQ5kAAAAG
Alert - From 192.168.0.119 Type of Attack - Detects basic SQL
authentication bypass attempts 2/3 with unique id
Uqcub38AAQEAAAUsQ5kAAAAG
Alert - From 192.168.0.119 Type of Attack - Detects classic SQL
injection probings 1/2 with unique id Uqcub38AAQEAAAUsQ5kAAAAG
Aggregate into a single alert,
Alert - From 192.168.0.119 Type of Attack - Possible SQL Injection
Attacks with unique id Uqcub38AAQEAAAUsQ5kAAAAG
It is easy to achieve with the already existing SEC rule if you change
the 'desc' field accordingly. Currently, the 'desc' field reads as:
desc=client $1 unique_id $2
but the $2 match variable does not correspond to unique_id, but rather
to the message text. As a consequence, each alarm from the same client
with a different text gets written to output file. In order to avoid
logging alarms for the same client and the same unique_id, write the
desc field as follows:
desc=client $1 unique_id $3
And Secondly,, For the XSS attacks,,it also match some rule from the SQL
, so i want to aggregate it to only a single alert.
Here the sample logs for XSS attacks.
Alert - From 192.168.0.111 Type of Attack - SQL Injection Attack with
unique id Uqcvg38AAQEAAANHCikAAAAE
Alert - From 192.168.0.111 Type of Attack - SQL Character Anomaly
Detection Alert - Repetative Non-Word Characters with unique id
Uqcvg38AAQEAAANHCikAAAAE
Alert - From 192.168.0.111 Type of Attack - Restricted SQL Character
Anomaly Detection Alert - Total # of special characters exceeded with
unique id Uqcvg38AAQEAAANHCikAAAAE
Alert - From 192.168.0.111 Type of Attack - Detects classic SQL
injection probings 2/2 with unique id Uqcvg38AAQEAAANHCikAAAAE
Alert - From 192.168.0.111 Type of Attack - Cross-site Scripting (XSS)
Attack with unique id Uqcvg38AAQEAAANHCikAAAAE
Alert - From 192.168.0.111 Type of Attack - Possible XSS Attack Detected
- HTML Tag Handler with unique id Uqcvg38AAQEAAANHCikAAAAE
Alert - From 192.168.0.111 Type of Attack - XSS Attack Detected with
unique id Uqcvg38AAQEAAANHCikAAAAE
Alert - From 192.168.0.111 Type of Attack - IE XSS Filters - Attack
Detected with unique id Uqcvg38AAQEAAANHCikAAAAE
Can we make a rule, if SQL+XSS logs with the same unique id, then make a
new alert, like this,
Alert - From 192.168.0.111 Type of Attack - High Possibility of XSS
attacks with unique id Uqcvg38AAQEAAANHCikAAAAE
since the XSS alarms and SQL alarms can happen in any order (you could
have SQL alarm first and then XSS, but also vice versa), you need to
correlate these two alarms with the EventGroup2 rule. This rule type
matches two different event types which can happen in arbitrary order.
If you would like to use this rule together with the previous rule, your
final ruleset might look like this:
type=SingleWithSuppress
ptype=RegExp
pattern=\[client ([\d.]+)\].+\[msg "(.*?)"\].+\[unique_id "(.+?)"\]
continue=TakeNext
desc=client $1 unique_id $3
action= write secoutput.txt Alert - From $1 Type of Attack - $2 with
unique id $3
window=60
type=EventGroup2
ptype=RegExp
pattern=\[client ([\d.]+)\].+\[msg "(.*?SQL.*?)"\].+\[unique_id "(.+?)"\]
ptype2=RegExp
pattern2=\[client ([\d.]+)\].+\[msg "(.*?XSS.*?)"\].+\[unique_id "(.+?)"\]
desc=client $1 unique_id $3
action= write secoutput.txt Alert - From $1 Type of Attack - High
Possibility of XSS attacks with unique id $3
window=60
Note that in the first rule the 'continue' parameter has been set to
TakeNext, so that matching events are passed to the following
Eventgroup2 rule.
Also, you might want to change the 'window' parameter in those two rules
to a smaller value than 60 seconds -- if the events of interest
generally happen during 2-3 seconds, you could set 'window' to 5 or 10,
for instance.
hope this helps,
risto
Thanks again,
Term
On Wed, Dec 4, 2013 at 1:20 AM, Risto Vaarandi <risto.vaara...@gmail.com
<mailto:risto.vaara...@gmail.com>> wrote:
hi,
yes, SEC is using the Perl dialect of the regular expression
language.
Also, from your letter it seems that you would like to lessen the
number of alerts, and somehow aggregate several alerts into one
alert which could be reported to the SIEM.
SEC allows to implement many different aggregation schemes, but in
order to implement them, there must be a clear understanding what
exactly to aggregate and what is the common identifier (or common
identifiers) for the aggregated events.
From your log example it seems that there are several fields in
messages which could take this role:
1) client (or hostname) -- seems to contain the IP address of the
remote node
2) unique_id -- in your example, all the events have this field set
to the same value. Is this field somehow identifying the client
request? if so, you could take advantage of this.
Once you have identified field(s) of interest which can be used for
aggregation, there must be clear understanding what exactly you want
to for achieving aggregation. For example, it is fairly easy to
write any of the following:
1) a rule that reacts to the first event where field(s) have some
values, and suppresses the following events with same values during,
say, 1 minute
2) a counting rule that reacts if N or more events have been seen
during some time window
3) a rule that aggregates repeated events during some time window,
and sends them into a SIEM in a single message
etc. etc.
In order to provide one example, you could have the following
simplistic rule:
type=SingleWithSuppress
ptype=RegExp
pattern=\[client ([\d.]+)\].+\[unique_id "(.+?)"\]
desc=client $1 unique_id $2
action=pipe 'detected alert(s) for client $1 id $2' /bin/mail root
window=60
This rule will match the client IP and unique_id from a message,
send an e-mail warning to the root, and then suppress during 60
seconds the following events with the same IP address *and*
unique_id. The event suppression by distinct (clientIP, unique_id)
value pairs is achieved by extracting IP address and unique_id from
each event, and setting the suppression scope properly with the
'desc' field.
The above rule is just one example out of many possible rules -- but
in order to write them down, it must be known what fields are
relevant in the events, and how to use them for aggregation. After
you have clarified this question, it is much more easy to provide
suggestions and advise.
hope this helps,
risto
2013/12/2 termvrl term <term...@gmail.com <mailto:term...@gmail.com
Hi All, Thanks for your reply.
ModSecurity is an open source web application firewall based on
rules, so it generate logs each time match with its rules.
For one type of attacks it will matched with multiple rules up
to 4-5 pattern matched.
Example, SQL attacks, will match at least 4 rules from
"sql_injestion.rules". So, i will receive 4 line of logs in my
SIEM for the same event.
Unfortunately, for a some type of attacks it will match multiple
type of attacks rules, For example, when i try to do XSS attack,
the rules matched is a lot, it combine rules from SQL and XSS.
Futher checking, i found out that, the special character use in
XSS attacks matched with some SQL rules.
So, i want to use SEC to correlate it just to generate a single
Alert send to SIEM.
Hi Tim, i have try your conf and its work like i want.
But i have one request, can we log it only one time?
here is my output,
root@ubuntu:/home/term/sec-2.7.2# perl sec -conf=tim.conf
-input=/var/log/apache2/error.log
SEC (Simple Event Correlator) 2.7.2
Reading configuration from tim.conf
2 rules loaded from tim.conf
Opening input file /var/log/apache2/error.log
Stdin connected to terminal, SIGINT can't be used for changing
the logging level
Writing event 'SQL rule matched' to file -
SQL rule matched
Creating context 'SQL_INJECTION_ATTACK'
Writing event 'SQL rule matched' to file -
SQL rule matched
Creating context 'SQL_INJECTION_ATTACK'
Writing event 'SQL rule matched' to file -
SQL rule matched
Creating context 'SQL_INJECTION_ATTACK'
Writing event 'SQL rule matched' to file -
SQL rule matched
Creating context 'SQL_INJECTION_ATTACK'
Writing event 'XSS matched' to file secoutput.txt
Writing event 'XSS matched' to file secoutput.txt
Writing event 'XSS matched' to file secoutput.txt
Writing event 'XSS matched' to file secoutput.txt
Deleting stale context 'SQL_INJECTION_ATTACK'
Stale context 'SQL_INJECTION_ATTACK' deleted
Hi Risto,
you correct with the regex, the regex its not detect it.
And as for the regex use in SEC, may i know what type of regex
style it use? is it Perl-style regex?
Thanks. Sorry if this is too long.
Term
On Mon, Dec 2, 2013 at 3:28 AM, Risto Vaarandi
<risto.vaara...@gmail.com <mailto:risto.vaara...@gmail.com>>
wrote:
2013/12/1 termvrl term <term...@gmail.com
<mailto:term...@gmail.com>>
Hi all,
i have working on correlate the alert from modsecurity.
when i simulate XSS attacks, modsec will generate alert
and it will match with SQL rule and XSS rule. So, i want
to use SEC to correlate if detect both signature then,
use write action to log a new message. Here is conf file.
# Rule to match XSS attack.
# SQL + XSS
type=Pair
ptype=RegExp
pattern=sql_injection_attacks
desc=$0
action=write - SQL rule matched
ptype2=RegExp
pattern2=xss_attacks\s*CRITICAL
desc2=$0
action2=write - XSS matched
window=5
The problem is it detect only the first pattern, and the
second pattern never matched. Is it because the
modsecurity generate it with same timestamp?
Attach is the sample log that i want to correlate.
Thanks
Term
hi,
the Pair rule works as follows -- if an event is observed
which matches the pattern given with the 'pattern' field, a
waiting operation is started which gets its ID from the
'desc' field, rule file name, and rule offset in the file
(the use of the rule file name and offset ensures that
operation IDs do not clash for different rules).
The waiting operation started by the Pair rule runs for the
'window' seconds, and expects to see an event which is
matched by 'pattern2'. Your sample rule assumes the
following:
1) first an event which matches a regular expression
sql_injection_attacks must come in,
2) the incoming event starts an operation which has the ID
containing the entire matching line (held by $0 variable),
3) the operation runs for 5 seconds and waits an event
matching the regular expression xss_attacks\s*CRITICAL (this
regular expression means "the string xss_attacks, followed
by 0 or more whitespace characters, followed by the string
CRITICAL").
Generally, it's a bad idea to set desc to $0, since entire
matching line contains a timestamp, which starts a new
waiting operation each time a matching line with a newer
timestamp is observed. Are you actually trying to do
processing on the client IP address basis? If so, your
regular expressions should match the IP address from
incoming events and assign it to a match variable, and this
variable should be used in the 'desc' field. In that way,
you run separate event correlation operations for distinct
IP addresses (also, have a look into the relevant section in
the official documentation:
http://simple-evcorr.sourceforge.net/man.html#lbAW)
Also, the Pair rule assumes that events are ordered. If this
is not the case, I would recommend to consider the
EventGroup2 rule. Finally, your second regular expression
xss_attacks\s*CRITICAL assumes that only whitespace can
appear in between "xss_attacks" and "CRITICAL", but your log
messages seem to be different. This is probably the reason
why the second pattern of your Pair rule fails to match
incoming events.
hope this helps,
risto
------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your
business. Most IT
organizations don't have a clear picture of how
application performance
affects their revenue. With AppDynamics, you get 100%
visibility into your
Java,.NET, & PHP application. Start your 15-day FREE
TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
<mailto:Simple-evcorr-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into
your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of
AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics
Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
