hi Przemysław,

for addressing this problem, you could divide it into several independent
subtasks:
1) memorize the mapping from msgid to username, so that each mapping would
exist for a certain amount of time
2) when a bounce event appears in the log, generate a synthetic event for
the bounce, so that msgid->username mapping is used for including the user
name in the synthetic event
3) count and threshold synthetic events, and generate an alarm if more than
10 events have been seen in 1 hour for the given user name

To illustrate how this could work, I've put together a small rulebase of
three rules, with each rule corresponding to one of the subtasks above:

# establish msgid->username mapping

type=single
ptype=regexp
pattern=postfix.*\]: (?<MSGID>\S+): .* sasl_username=(?<LOGIN>\S+)
desc=set up a context for message ID $+{MSGID}
action=create MSGID_$+{MSGID} 3600; fill MSGID_$+{MSGID} $+{LOGIN}

# generate a synthetic bounce event which contains the user  name
# (username is obtained through msgid->username mapping)

type=single
ptype=regexp
pattern=postfix\/smtp.*\]: (?<MSGID>\S+):.*dsn=5.*, status=bounced
context=MSGID_$+{MSGID}
desc=create a synthetic event for a bounce event with a username
action=copy MSGID_$+{MSGID} %user; event
BOUNCE_FOR_USER_%{user}_MSGID_$+{MSGID}

# count and threshold synthetic events generated above

type=singlewiththreshold
ptype=regexp
pattern=BOUNCE_FOR_USER_(?<LOGIN>\S+)_MSGID_(\S+)
desc=Too many bounces for user $+{LOGIN} during 1 hour
action=write - %s
thresh=10
window=3600

For recording the mappings, the first rule simply uses a context for each
msgid, where the message ID is used as a context name, and user name is
written into context's event store with the 'fill' action. Each context
exists for 1 hour, but you can increase the mapping lifetime by using a
larger value instead of 3600 seconds for the 'create' action.

The second rule converts the original bounce event into a synthetic event
which contains the user name. Note that the user name is taken from the
context for the given msgid with the 'copy' action (if the context does not
exist, the rule would not match the original bounce event due to 'context'
field, and the synthetic event is not created).

Finally, the third rule implements event counting which is done on user
name basis. If you would like to do it per each unique (username,msgid)
tuple, you can include the msgid into the 'desc' field (the regular
expression would also need to be modified for setting the $+{MSGID}
variable).

Hope this helps,
risto

2018-06-12 17:55 GMT+03:00 Przemysław Orzechowski <
przemek.orzechow...@makolab.com>:

> Hi
>
>
> I have successfully created 2 rules that mach lines i want to act on but
> I'm struggling in finding a way to correlate them.
>
> First description what i want to get
>
> Generate an event for a threshold ie 10 events in 1 hour the problem is i
> don't quite get how to count them correctly
>
> First is a line that has username and mailid for example:
>
> Jun  3 06:46:13 server1 postfix/smtpd[3268]: 626C5802295:
> client=unknown[8.8.8.8], sasl_method=LOGIN, sasl_username=login@some.domai
> n.com
>
> Im maching it with pattern
>
> pattern=postfix.*\]: (?<MSGID>\S+): .* sasl_username=(?<LOGIN>\S+)
>
> the threshold should be for unique LOGINS and should count events mached
> by the second rule ie lines like these:
>
>
> Jun  3 06:46:14 server1 postfix/smtp[23808]: 626C5802295: to=<
> somb...@somewhere.com>, relay=somewhere.com[9.9.9.9]:25, delay=1,
> delays=1/0.03/21/20, dsn=5.0.0, status=bounced (host somewhere.com[9.9.9.9]
> said: 550-The mail server could not deliver mail to somb...@somewhere.com.
> 550-The account or domain may not exist, they may be blacklisted, or
> missing 550 the proper dns entries. (in reply to RCPT TO command))
>
> Im maching this line with pattern
>
> pattern=postfix\/smtp.*\]: (?<MSGID>\S+):.*dsn=5.*
>
> the MSGID is identical in both lines
>
> Im trying to count the bounces and execute an action if there are too many
> of them for single sasl_username in a time period
>
> Btw MSGID can repeat themselvs in longer preiods of time but should be
> unique within the treshold period.
>
> I would be very thankfull for any help
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Simple-evcorr-users mailing list
> Simple-evcorr-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users

Reply via email to