RE: postscreen fail2ban filter

[Scott Techlist]

>There is no need to duplicate the threshold check.

I'm not duplicating the check.   I was just considering using the logged,
recorded checks (of a minimum value) and making use of those.  They could
trigger a ban of the IP via fail2ban's  respective jail's frequency
settings, based on those log entries.

But I understand other than getting the entries out of the logs, postscreen
can handle these without help from the firewall.

Re: postscreen fail2ban filter

[Allen Coates]

On 17/07/17 21:04, Scott Techlist wrote:
>> Postcreen logs DISCONNECT for clients that PASS the "after 220 greeting"
>> tests (bare newline, non-SMTP command, pipelining).
> Exactly what I was afraid of, thanks for the confirmation.
>
>> I don't think there is much to gain from parsing postscreen logging to
> produce
>> fail2ban rules. postscreen is designed to handle a lot of abuse with
> near-zero
>> resources.
> I understand and that's great.  But it would be nice to prevent at least
> some of connections and their ongoing log entries.  Without getting out of
> my comfort zone in solutions like Robert's and Allen's.

FWIW, I decided to implement iptables blocking after several bouts of 
hundreds of concurrent connect requests.  They created a weeks "worth"
of log entries in less than ten minutes  - which I didn't like !

These days I only see two or three such connect requests before they are
blocked.   From the IP table counters, some hosts keep trying to connect
for a month or more.

For some reason, I am also quite intolerant about AUTH probes, even
though there is nothing to find...

On balance, I would like to keep Fail2Ban, or something similar -  but
as a back-up, not a primary filter.

Allen C

Re: postscreen fail2ban filter

[Patrick Ben Koetter]

* Wietse Venema :
> Scott Techlist:
> > As I watch the bots and spammers hammer my server with connection attempts,
> > I figured I might as well stop them even closer to the front door when they
> > try repeatedly.
> > 
> > I have fail2ban running already and once I enabled postscreen it didn't seem
> > to have much to do anymore.
> > 
> > My primary question is: Can I filter on the DISCONNECT log line for bad
> > connections (and only bad connections), or do some "good" connections also
> > log a DISCONNECT.
> 
> Postcreen logs DISCONNECT for clients that PASS the "after 220
> greeting" tests (bare newline, non-SMTP command, pipelining).
> 
> I don't think there is much to gain from parsing postscreen logging
> to produce fail2ban rules. postscreen is designed to handle a lot
> of abuse with near-zero resources.

To add my 2ct: As long as it doesn't impose a problem on the application I
prefer to 'see' the disconnects in the application and not on some other
host (read: upstream firewall). This makes it easier for me to see
relationships etc.

p@rick

-- 
[*] sys4 AG
 
https://sys4.de, +49 (89) 30 90 46 64
Schleißheimer Straße 26/MG,80333 München
 
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer, Wolfgang Stief
Aufsichtsratsvorsitzender: Florian Kirstein
 

RE: postscreen fail2ban filter

[Scott Techlist]

>Postcreen logs DISCONNECT for clients that PASS the "after 220 greeting"
>tests (bare newline, non-SMTP command, pipelining).

Exactly what I was afraid of, thanks for the confirmation.

>I don't think there is much to gain from parsing postscreen logging to
produce
>fail2ban rules. postscreen is designed to handle a lot of abuse with
near-zero
>resources.

I understand and that's great.  But it would be nice to prevent at least
some of connections and their ongoing log entries.  Without getting out of
my comfort zone in solutions like Robert's and Allen's.

Alas (and for search posterity in case someone else tries the filter I
posted), I don't think the first line of my posted filter (copied from
someone else) is reliable after thinking about it some more.  Just because I
get a "listed by domain" log line, that won't mean the overall score was
above the threshold and going to fail necessarily.  So it's out.

failregex = ^%(__prefix_line)saddr  listed by domain .* as .*$
^%(__prefix_line)sNOQUEUE: reject: RCPT from
(.*)\[\]:([0-9]{4,5}:)? 550.*$

The second line (cleaned up and anchored) should be worthy counter I think
as it's a solid 550 reject.

Re this log line:
>Jul 17 14:23:36 tn3 postfix/postscreen[21915]: DNSBL rank 3 for
[46.102.230.94]:63564

Maybe it would be safe to filter on this line where the "DNSBL rank [n]" was
>= my threshold:

Assuming a threshold of 3, max less than 20, match with:
failregex = ^%(__prefix_line)sDNSBL rank (([3-9]|1[09])) for \[\]:.*$

I realize this is a little OT but the postfix question is:
Lines like the sample log lines like above, whose rank is at or above my
threshold, should represent a connection that's going to fail postscreen and
be dropped, right?

Won't pick all of them up, but will at least look at some of them.

Would be cool to have a log entry on overall postscreen pass (including
after 220) or fail.  That would be easy to watch.

Re: postscreen fail2ban filter

[Allen Coates]

On 17/07/17 16:43, Scott Techlist wrote:
> As I watch the bots and spammers hammer my server with connection attempts,
> I figured I might as well stop them even closer to the front door when they
> try repeatedly.
>
> I have fail2ban running already and once I enabled postscreen it didn't seem
> to have much to do anymore.
>
> My primary question is: Can I filter on the DISCONNECT log line for bad
> connections (and only bad connections), or do some "good" connections also
> log a DISCONNECT.  Like this:
>
> Jul 17 10:08:27 tn3 postfix/postscreen[19184]: DISCONNECT
> [110.175.112.118]:63862
>
> My server isn't "live" yet and my logs are just from bots and spammers
> already knocking at the door.  So I don't have a lot of "good" connections
> to look at.  I couldn't figure out if a "good" connection that went through
> after 220 tests, or any other pass, also got a DISCONNECT entry.  I fit
> does, I can't use it.  
>
> I've only found a couple of other's fail2ban filters related to postscreen
> logs:
>
> One from:
> https://github.com/jannickfahlbusch/fail2ban-rulez/blob/master/MailServer/Po
> stFix/postfix-dnsblog.conf
>
> That one picks up on the "listed by domain" string but because I may have
> multiple "hits" per connection due to multiple dnsbls, it throws off my
> banning thresholds.  Not a huge deal, but not the count I want.  This
> connection counted 4 "fails"
>
> Jul 17 10:01:40 tn3 postfix/postscreen[19136]: CONNECT from
> [105.174.2.98]:11607 to [45.63.111.83]:25
> Jul 17 10:01:40 tn3 postfix/dnsblog[19138]: addr 105.174.2.98 listed by
> domain b.barracudacentral.org as 127.0.0.2
> Jul 17 10:01:40 tn3 postfix/dnsblog[19142]: addr 105.174.2.98 listed by
> domain zen.spamhaus.org as 127.0.0.11
> Jul 17 10:01:40 tn3 postfix/dnsblog[19142]: addr 105.174.2.98 listed by
> domain zen.spamhaus.org as 127.0.0.4
> Jul 17 10:01:40 tn3 postfix/dnsblog[19143]: addr 105.174.2.98 listed by
> domain dnsbl.sorbs.net as 127.0.0.7
> Jul 17 10:01:46 tn3 postfix/postscreen[19136]: DNSBL rank 6 for
> [105.174.2.98]:11607
> Jul 17 10:01:48 tn3 postfix/postscreen[19136]: DISCONNECT
> [105.174.2.98]:11607
>
> >From searching this list I saw this filter:
> https://translate.google.com/translate?sl=auto=en=y=_t=en=U
> TF-8=https%3A%2F%2Fkupschke.net%2F2013%2F04%2F20%2Ffail2ban-und-postscreen
> %2F==url
>
> That one is picking up on 5xx reject codes like this one.  I don't' have
> many like this (yet):
>
> Jul 17 07:58:28 tn3 postfix/postscreen[8899]: CONNECT from
> [66.231.40.205]:64187 to [45.63.111.83]:25
> Jul 17 07:58:28 tn3 postfix/dnsblog[8904]: addr 66.231.40.205 listed by
> domain zen.spamhaus.org as 127.0.0.4
> Jul 17 07:58:34 tn3 postfix/postscreen[8899]: DNSBL rank 3 for
> [66.231.40.205]:64187
> Jul 17 07:58:35 tn3 postfix/postscreen[8899]: NOQUEUE: reject: RCPT from
> [66.231.40.205]:64187: 550 5.7.1 Service unavailable; client [66.231.40.205]
> blocked using zen.spamhaus.org;
> from=<36jr3j36jr36jr3.3625327...@superuser.com>,
> to=, proto=ESMTP, helo=<[192.168.1.5]>
> Jul 17 07:58:35 tn3 postfix/postscreen[8899]: DISCONNECT
> [66.231.40.205]:64187
>
> Anyone have any good postscreen fail2ban filters?
>
> Mine for now is:
> failregex =   ^%(__prefix_line)saddr  listed by domain .* as .*$
>   reject: RCPT from (.*)\[\]:([0-9]{4,5}:)? 550

My homespun iptables blocker works in two stages:

The log is swept over the last day or so, and -
a) Multiple transgressions puts the offender into a local blacklist.
b) Multiple blacklist refusals and they are added to an iptables DROP list.

and they stay there until they stop trying to connect ...

It is overkill for my (domestic) server,  but it keeps my hand in for
writing scripts.

(And I hate spam!!)

FYI  Below is today's attrition report:


Nuisance hosts currently blocked by firewall:91
Still active:24

POSTSCREEN  ATTRITION  REPORT  FOR  Today so far

Connections handled by Postscreen:189
White-listed:11
Black-listed Locally:23
Black-listed by DNSBL:75
Early hang-ups:9
Pre-Greets:3
Command pipelining:0
Non-SMTP commands:0
Multi-connect refusals:2

Grey-list Deferrals:1

Refusal Ratio:93 percent

Connections passed on to mail server:12
Early disconnects:0
Authorisation Probes:0
Client Host Refused:0
Bad HELO Command:0
Bad Sender address:0
Illegal Relay Attempts:0

Invalid Recipients:0

messages placed into Postfix queue:12
Sent to Dovecot:12
Outbound messages:0

Actual messages received in mailbox:12


Regards

Allen C











>
>

Re: postscreen fail2ban filter

[Robert Schetterer]

Am 17.07.2017 um 20:06 schrieb /dev/rob0:
> On Mon, Jul 17, 2017 at 01:33:24PM -0400, Wietse Venema wrote:
>> I don't think there is much to gain from parsing postscreen logging
>> to produce fail2ban rules. postscreen is designed to handle a lot
>> of abuse with near-zero resources.
> 
> Granted, not much benefit within Postfix.  But consider: these 
> botnets are also attacking other services: http, ssh, DNS, and more.  
> I think it's a reasonable goal to want to block botnets in the 
> firewall.
> 
> [ Linux-specific ]
> 
> We do it with ssh attacks here using the "recent" iptables module.
> (On my TODO is a plan to port those rules to the --match set and 
> --jump SET modules and ipset(8).)  These attacks, when exceeding 
> established maximum new connection rates, cause the attacker to be 
> entirely blocked in the firewall.
> 
> That obviously won't work for SMTP, where [FSVO] legitmate sites 
> might have a bunch of new connections in short periods.  For ssh, 
> we're using the assumption that these connections are humans who are 
> seeking shell access, although indeed a poorly-written script could 
> easily go beyond the limits.
> 
> So the move to ipset would allow broader participation in attack 
> deflection; fail2ban could help populate the firewall blocking with 
> input from httpd, named, and others (including Postfix.)
> 
> Another advantage of firewall blocking is at the human level: 
> decrease of noise in the logs, to potentially save time for the 
> admin.  I haven't had many systems which were vulnerable to the 
> brute-force ssh attacks, but I don't need to see that spam in the 
> system logs.
> 
> To be clear, I don't have an answer for the OP; I am just tossing 
> out a couple of coins in support of the goal.
> 

you may have a look here for ideas

https://sys4.de/de/blog/2015/11/07/abwehr-des-botnets-pushdo-cutwail-ehlo-ylmf-pc-mit-iptables-string-recent-smtp/

https://sys4.de/de/blog/2014/03/27/fighting-smtp-auth-brute-force-attacks/

https://sys4.de/de/blog/2012/12/28/botnets-mit-rsyslog-und-iptables-recent-modul-abwehren/


Best Regards
MfG Robert Schetterer

-- 
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64
Schleißheimer Straße 26/MG, 80333 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein

Re: postscreen fail2ban filter

[/dev/rob0]

On Mon, Jul 17, 2017 at 01:33:24PM -0400, Wietse Venema wrote:
> I don't think there is much to gain from parsing postscreen logging
> to produce fail2ban rules. postscreen is designed to handle a lot
> of abuse with near-zero resources.

Granted, not much benefit within Postfix.  But consider: these 
botnets are also attacking other services: http, ssh, DNS, and more.  
I think it's a reasonable goal to want to block botnets in the 
firewall.

[ Linux-specific ]

We do it with ssh attacks here using the "recent" iptables module.
(On my TODO is a plan to port those rules to the --match set and 
--jump SET modules and ipset(8).)  These attacks, when exceeding 
established maximum new connection rates, cause the attacker to be 
entirely blocked in the firewall.

That obviously won't work for SMTP, where [FSVO] legitmate sites 
might have a bunch of new connections in short periods.  For ssh, 
we're using the assumption that these connections are humans who are 
seeking shell access, although indeed a poorly-written script could 
easily go beyond the limits.

So the move to ipset would allow broader participation in attack 
deflection; fail2ban could help populate the firewall blocking with 
input from httpd, named, and others (including Postfix.)

Another advantage of firewall blocking is at the human level: 
decrease of noise in the logs, to potentially save time for the 
admin.  I haven't had many systems which were vulnerable to the 
brute-force ssh attacks, but I don't need to see that spam in the 
system logs.

To be clear, I don't have an answer for the OP; I am just tossing 
out a couple of coins in support of the goal.
-- 
  http://rob0.nodns4.us/
  Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:

Re: postscreen fail2ban filter

[Wietse Venema]

Scott Techlist:
> As I watch the bots and spammers hammer my server with connection attempts,
> I figured I might as well stop them even closer to the front door when they
> try repeatedly.
> 
> I have fail2ban running already and once I enabled postscreen it didn't seem
> to have much to do anymore.
> 
> My primary question is: Can I filter on the DISCONNECT log line for bad
> connections (and only bad connections), or do some "good" connections also
> log a DISCONNECT.

Postcreen logs DISCONNECT for clients that PASS the "after 220
greeting" tests (bare newline, non-SMTP command, pipelining).

I don't think there is much to gain from parsing postscreen logging
to produce fail2ban rules. postscreen is designed to handle a lot
of abuse with near-zero resources.

Wietse

postscreen fail2ban filter

[Scott Techlist]

As I watch the bots and spammers hammer my server with connection attempts,
I figured I might as well stop them even closer to the front door when they
try repeatedly.

I have fail2ban running already and once I enabled postscreen it didn't seem
to have much to do anymore.

My primary question is: Can I filter on the DISCONNECT log line for bad
connections (and only bad connections), or do some "good" connections also
log a DISCONNECT.  Like this:

Jul 17 10:08:27 tn3 postfix/postscreen[19184]: DISCONNECT
[110.175.112.118]:63862

My server isn't "live" yet and my logs are just from bots and spammers
already knocking at the door.  So I don't have a lot of "good" connections
to look at.  I couldn't figure out if a "good" connection that went through
after 220 tests, or any other pass, also got a DISCONNECT entry.  I fit
does, I can't use it.  

I've only found a couple of other's fail2ban filters related to postscreen
logs:

One from:
https://github.com/jannickfahlbusch/fail2ban-rulez/blob/master/MailServer/Po
stFix/postfix-dnsblog.conf

That one picks up on the "listed by domain" string but because I may have
multiple "hits" per connection due to multiple dnsbls, it throws off my
banning thresholds.  Not a huge deal, but not the count I want.  This
connection counted 4 "fails"

Jul 17 10:01:40 tn3 postfix/postscreen[19136]: CONNECT from
[105.174.2.98]:11607 to [45.63.111.83]:25
Jul 17 10:01:40 tn3 postfix/dnsblog[19138]: addr 105.174.2.98 listed by
domain b.barracudacentral.org as 127.0.0.2
Jul 17 10:01:40 tn3 postfix/dnsblog[19142]: addr 105.174.2.98 listed by
domain zen.spamhaus.org as 127.0.0.11
Jul 17 10:01:40 tn3 postfix/dnsblog[19142]: addr 105.174.2.98 listed by
domain zen.spamhaus.org as 127.0.0.4
Jul 17 10:01:40 tn3 postfix/dnsblog[19143]: addr 105.174.2.98 listed by
domain dnsbl.sorbs.net as 127.0.0.7
Jul 17 10:01:46 tn3 postfix/postscreen[19136]: DNSBL rank 6 for
[105.174.2.98]:11607
Jul 17 10:01:48 tn3 postfix/postscreen[19136]: DISCONNECT
[105.174.2.98]:11607

>From searching this list I saw this filter:
https://translate.google.com/translate?sl=auto=en=y=_t=en=U
TF-8=https%3A%2F%2Fkupschke.net%2F2013%2F04%2F20%2Ffail2ban-und-postscreen
%2F==url

That one is picking up on 5xx reject codes like this one.  I don't' have
many like this (yet):

Jul 17 07:58:28 tn3 postfix/postscreen[8899]: CONNECT from
[66.231.40.205]:64187 to [45.63.111.83]:25
Jul 17 07:58:28 tn3 postfix/dnsblog[8904]: addr 66.231.40.205 listed by
domain zen.spamhaus.org as 127.0.0.4
Jul 17 07:58:34 tn3 postfix/postscreen[8899]: DNSBL rank 3 for
[66.231.40.205]:64187
Jul 17 07:58:35 tn3 postfix/postscreen[8899]: NOQUEUE: reject: RCPT from
[66.231.40.205]:64187: 550 5.7.1 Service unavailable; client [66.231.40.205]
blocked using zen.spamhaus.org;
from=<36jr3j36jr36jr3.3625327...@superuser.com>,
to=, proto=ESMTP, helo=<[192.168.1.5]>
Jul 17 07:58:35 tn3 postfix/postscreen[8899]: DISCONNECT
[66.231.40.205]:64187

Anyone have any good postscreen fail2ban filters?

Mine for now is:
failregex = ^%(__prefix_line)saddr  listed by domain .* as .*$
reject: RCPT from (.*)\[\]:([0-9]{4,5}:)? 550