In message <20101201193119.ga18...@esri.com>,
Ray Van Dolson writes:
>As an exercise to learn SEC, I'm trying to create a ruleset that will
>report on clients sending more than X emails through our SMTP server in
>Y amount of seconds.
>
>Basically I'm looking for the same "from" address, same mail server and
>same client address (source IP) and if it occurs more than a certain
>number of times in 30 seconds or so, I want to know about it.
>
>Easy enough to set this up, but I also want to see all the
>corresponding log entries in the alert email.
>
>Based on this[1] thread, I've come up with:
>
> # Create our context on the original event trigger.
> type=SingleWithThreshold
> ptype=RegExp
> pattern=(esri3|vail)\ssendmail[^:]+.*from=([^,]+),\s.*relay=(\S+\s\[.*?\]).*$
> desc=Potential spam from $2 using $1 from relay $3
> window=30
> thresh=5
> action=create SPAM_$1_$2_$3 30 \
> (report SPAM_$1_$2_$3 /bin/mail -s "Test" rvandol...@esri.com); \
> add SPAM_$1_$2_$3 Spam attempt:; add SPAM_$1_$2_$3 $0
>
> # Add subsequent events to the context.
> type=Single
> ptype=RegExp
> pattern=(esri3|vail)\ssendmail[^:]+.*from=([^,]+),\s.*relay=(\S+\s\[.*?\]).*$
> desc=Potential spam from $2 using $1 from relay $3
> context=SPAM_$1_$2_$3
> action=add SPAM_$1_$2_$3 "Additional event: $0"; set SPAM_$1_$2_$3 30
>
>However, the email I receive includes only one event (which appears to
>be the first event seen), not the minimum of five I'd expect.
>
>Am I doing this wrong?
What's happening with your rules is that the event comes in and
triggers the threshold rule. This doesn't do anything till all 5
events have come in, then the action is executed. I don't remember at
the moment if $0 is the first or last event seen but from your comment
I assume it's the first.
Also since the threshold rule is first, it matches and consumes all of
the events and the single rule never sees an event. (If you send a
USR1 signal to sec you should see this in the rule execution counts
section of the sec dump.)
Try this ruleset:
# Add subsequent events to the context.
type=Single
ptype=RegExp
continue=takenext
pattern=(esri3|vail)\ssendmail[^:]+.*from=([^,]+),\s.*relay=(\S+\s\[.*?\]).*$
desc=Potential spam from $2 using $1 from relay $3 (rule 1)
context=SPAM_$1_$2_$3
action=add SPAM_$1_$2_$3 "Additional event: $0"
# Detect first event and add to context.
type=Single
ptype=RegExp
continue=takenext
pattern=(esri3|vail)\ssendmail[^:]+.*from=([^,]+),\s.*relay=(\S+\s\[.*?\]).*$
desc=Potential spam from $2 using $1 from relay $3 (rule 2)
context= ! SPAM_$1_$2_$3
action=create SPAM_$1_$2_$3 30 ; \
add SPAM_$1_$2_$3 Spam attempt:; \
add SPAM_$1_$2_$3 $0
# Report context when too many events arrive in 30 seconds
type=SingleWithThreshold
ptype=RegExp
pattern=(esri3|vail)\ssendmail[^:]+.*from=([^,]+),\s.*relay=(\S+\s\[.*?\]).*$
desc=Potential spam from $2 using $1 from relay $3 (rule 3)
window=30
thresh=5
action= report SPAM_$1_$2_$3 /bin/mail -s "Test" rvandol...@example.com;
This causes the following sequence (note $1, $2, $3 should be
interpreted as the values of those variables):
first event, bypasses rule 1 since the context SPAM_$1_$2_$3
doesn't yet exist. It hits rule 2 and triggers the creation of the
context SPAM_$1_$2_$3 and adds the event to the context. Then because
continue=takenext the event passes to rule 3 (threshold) and is
counted.
second event triggers rule 1 (since SPAM_$1_$2_$3 was created by
rule 1 for event 1) the event is added to the context,
because continue=takenext, the event is not consumed but is
passed to rule 2. Rule 2 ignores it since the context argument says
to run only if SPAM_$1_$2_$3 doesn't exist. Since rule 2 didn't
trigger, the event is passed to rule 3 that counts and consumes it.
3rd, 4th events are processed like the second.
The 5th event is processed by rules 1 and 2 the same as the second
event, but when it hits the threshold rule, the threshold rule fires
and reports the context with all 5 events.
Note that I don't set a new lifetime on SPAM_$1_$2_$3 when an event
matches. I set it to 30 seconds when the first event comes in and
that makes the context die at the same wall clock time that the 30
second threshold window will end.
As a result if only 4 events come in in the 30 second window, a new
context will be created for the next set of events. Setting the
lifetime of the context in rule 1 can result in additional events
being gathered that were not involved in triggering the threshold.
For example work through what happens if you have 1 event arriving
every 29 seconds for 3 minutes then you have 5 events arriving in 30
seconds. What does SPAM_$1_$2_$3 contain if you reset the lifetime in
rule 1?
I sometimes wish there were more actions for the threshold rule:
1) action when the correlation triggers (current action keyword)
A) action on the first match of a event (when the correlation starts)
B) action for each event that matches the correlation
2) action when window closes if 1 triggered (current action 2 I think)
That would make your use case a bit simpler to implement:
# Report context when too many events arrive in 30 seconds
# (wished for version)
type=SingleWithThreshold
ptype=RegExp
pattern=(esri3|vail)\ssendmail[^:]+.*from=([^,]+),\s.*relay=(\S+\s\[.*?\]).*$
desc=Potential spam from $2 using $1 from relay $3 (rule 3)
window=30
thresh=5
actionA= create SPAM_$1_$2_$3 30 ; \
add SPAM_$1_$2_$3 Spam attempt:; \
add SPAM_$1_$2_$3 $0
actionB= add SPAM_$1_$2_$3 "Additional event: $0"
action= report SPAM_$1_$2_$3 /bin/mail -s "Test" rvandol...@example.com;
action2 = delete SPAM_$1_$2_$3
action2 cleans up the context if it was used to trigger a report. You
would still have the issue of recording too many events under certain
conditions, but it is at least one rule.
Another possible mechanism to make this easier is for the threshold
rule to track/record the events as it processes them and to make that
list available to the user somehow. So you could use as your action:
action = geteventlist %E; add SPAM_$1_$2_$3 %E; \
report SPAM_$1_$2_$3 /bin/mail -s "Test" rvandol...@example.com;
this would have the drawback that you wouldn't have an easy way of
annotating the events ("Additional event: " for example), but it
would have the advantage that you could see exactly what events were
involved in triggering the correlation.
--
-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.
------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
