In your sample log lines, you have two dashes after the IP address; your regex
only expects one. Try:

failregex = ^<HOST>.+?"(GET|POST|HEAD) /.+?" 404 .+$

BTW, you don't escape / or -

Bill

On 3/13/2018 2:12 PM, Sophie Loewenthal wrote:
NGINX BOTCHECK

Debian 9.2

$ dpkg -l fail2ban
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                     Version           Architecture      Description
+++-========================-=================-=================-=====================================================
ii  fail2ban                 0.9.6-2           all               ban hosts that 
cause multiple authentication errors


-----------------------------------------------------
$ cat /etc/fail2ban/filter.d/nginx-botsearch.conf |grep -v ^#

[INCLUDES]

before = botsearch-common.conf

[Definition]

failregex = ^<HOST> \- \S+ \[\] \"(GET|POST|HEAD) \/<block> \S+\" 404 .+$
             ^ \[error\] \d+#\d+: \*\d+ (\S+ )?\"\S+\" (failed|is not found) \(2\: No such file or 
directory\), client\: <HOST>\, server\: \S*\, request: \"(GET|POST|HEAD) \/<block> 
\S+\"\, .*?$

ignoreregex =

——————————————————————————

Sample log lines:
# grep bot www.example.co.uk_access.log
66.249.75.148 - - [13/Mar/2018:12:33:58 +0000] TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384 "GET /robots.txt 
HTTP/1.1" 404 162 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; 
+http://www.google.com/bot.html)"
66.249.75.148 - - [13/Mar/2018:12:33:58 +0000] TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384 "GET /ads.txt 
HTTP/1.1" 404 162 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; 
+http://www.google.com/bot.html)"
157.55.39.236 - - [13/Mar/2018:15:04:19 +0000] TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384 "GET / HTTP/1.1" 
302 5 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"














On 13 Mar 2018, at 11:07, Tom Hendrikx <t...@whyscream.net> wrote:

Hi,

Please keep replies on-list, don't e-mail me privately.

Can you post:
- OS version you're running
- fail2ban version you're running
- contents of the /etc/fail2ban/filter.d/dovecot.conf file, so we can
extend the current regex

For nginx, please create a new thread and supply the same information,
along with some sample log lines.

Kind regards,

        Tom


On 12-03-18 21:03, Sophie Loewenthal wrote:
Hi,  Thanks for the fail2ban-regex checker. I checked nginx and this also 
seemed not to work.  Again I have the ciphers listed when they connect.



**** NGINX *****
# fail2ban-regex mx10.example.co.uk_access.log '^<HOST> \- \S+ \[\] \"(GET|POST|HEAD) 
\/<block> \S+\" 404 .+$'
Running tests
=============
Use   failregex line : ^<HOST> \- \S+ \[\] \"(GET|POST|HEAD) \/<block> \S...
Use         log file : mx10.example.co.uk_access.log
Use         encoding : UTF-8

Results
=======
Failregex: 0 total
Ignoreregex: 0 total
Date template hits:
|- [# of hits] date format
|  [10] Day(?P<_sep>[-/])MON(?P=_sep)Year[ 
:]?24hour:Minute:Second(?:\.Microseconds)?(?: Zone offset)?
`-

Lines: 10 lines, 0 ignored, 0 matched, 10 missed
[processed in 0.00 sec]

|- Missed line(s):
|  207.46.13.127 - - [12/Mar/2018:11:52:42 +0000] TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384 "GET / 
HTTP/1.1" 302 5 "-" "Mozilla/5.0 (compatible; bingbot/2.0; 
+http://www.bing.com/bingbot.htm)"
|  184.105.247.194 - - [12/Mar/2018:14:25:42 +0000] TLSv1.2/ECDHE-RSA-AES128-GCM-SHA256 "GET / 
HTTP/1.1" 302 5 "-" "-"
|  183.129.160.229 - - [12/Mar/2018:15:21:21 +0000] TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384 "GET 
/farm/libs/modules/tween/tween.min.js HTTP/1.1" 404 162 "-" "Mozilla/5.0 (Macintosh; 
Intel Mac OS X 10.11; rv:47.0) Gecko/20100101 Firefox/47.0"
|  207.46.13.104 - - [12/Mar/2018:15:48:45 +0000] TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384 "GET / 
HTTP/1.1" 302 5 "-" "Mozilla/5.0 (compatible; bingbot/2.0; 
+http://www.bing.com/bingbot.htm)"
|  207.46.13.127 - - [12/Mar/2018:16:15:41 +0000] TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384 "GET / 
HTTP/1.1" 302 5 "-" "Mozilla/5.0 (compatible; bingbot/2.0; 
+http://www.bing.com/bingbot.htm)"
|  66.249.75.148 - - [12/Mar/2018:16:37:47 +0000] TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384 "GET /robots.txt 
HTTP/1.1" 404 162 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; 
+http://www.google.com/bot.html)"
|  66.249.75.144 - - [12/Mar/2018:16:37:47 +0000] TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384 "GET /ads.txt 
HTTP/1.1" 404 162 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; 
+http://www.google.com/bot.html)"
|  207.46.13.45 - - [12/Mar/2018:19:01:28 +0000] TLSv1.2/DHE-RSA-AES256-GCM-SHA384 "GET /robots.txt 
HTTP/1.1" 404 162 "-" "Mozilla/5.0 (compatible; bingbot/2.0; 
+http://www.bing.com/bingbot.htm)"
|  207.46.13.45 - - [12/Mar/2018:19:01:29 +0000] TLSv1.2/DHE-RSA-AES256-GCM-SHA384 "GET /robots.txt 
HTTP/1.1" 404 162 "-" "Mozilla/5.0 (compatible; bingbot/2.0; 
+http://www.bing.com/bingbot.htm)"
|  40.77.167.54 - - [12/Mar/2018:19:01:34 +0000] TLSv1.2/ECDHE-RSA-AES256-GCM-SHA384 "GET / 
HTTP/1.1" 302 5 "-" "Mozilla/5.0 (compatible; bingbot/2.0; 
+http://www.bing.com/bingbot.htm)"
`-





***** DOVECOT ******
# fail2ban-regex /var/log/mail.log '^%(__prefix_line)s(?:pop3|imap)-login: (?:Info: )?(?:Aborted 
login|Disconnected)(?::(?: [^ \(]+)+)? \((?:auth failed, \d+ attempts( in \d+ secs)?|tried to use 
(disabled|disallowed) \S+ auth)\):( user=<[^>]+>,)?( method=\S+,)? rip=<HOST>(?:, 
lip=\S+)?(?:, TLS(?: handshaking(?:: SSL_accept\(\) failed: error:[\dA-F]+:SSL 
routines:[TLS\d]+_GET_CLIENT_HELLO:unknown protocol)?)?(: Disconnected)?)?(, 
session=<\S+>)?\s*$'

Running tests
=============
Use   failregex line : ^%(__prefix_line)s(?:pop3|imap)-login: (?:Info: )?...
Use         log file : /var/log/mail.log
Use         encoding : UTF-8

Results
=======
Failregex: 0 total
Ignoreregex: 0 total
Date template hits:
|- [# of hits] date format
|  [3014] (?:DAY )?MON Day 24hour:Minute:Second(?:\.Microseconds)?(?: Year)?
`-

Lines: 3014 lines, 0 ignored, 0 matched, 3014 missed
[processed in 0.38 sec]
Missed line(s): too many to print.  Use --print-all-missed to print all 3014 
lines



best,
Sophie





On 12 Mar 2018, at 10:47, Tom Hendrikx <t...@whyscream.net> wrote:

Hi,


you can test this using the fail2ban-regex tool. When I use one of your
example lines, it doesn't match on my setup (ubuntu 16.04, fail2ban
0.9.3). The similar logline from own setup doesn match:

Feb 19 03:02:33 alison dovecot: imap-login: Disconnected (auth failed, 1
attempts in 7 secs): user=<john...@example.net>, method=PLAIN,
rip=127.0.0.1, lip=127.0.0.1, TLS, session=<e1LxFYdlwKbes9bZ>

The latest config file for dovecot in github is completely different
from the one I'm using, but also lacks support for this AFAICS.

I guess we could come up with a regex that would support your log lines too.

Kind regards,
        Tom

On 12-03-18 10:02, Sophie Loewenthal wrote:
Hi,

Sorry for the delay. Flu.

Will fail2ban act on these example lines below with the extra cipher details?

I know the lines below would not trigger actions because there are not enough 
failures in the log. Normally dovecot does not have the TLS/cipher part logged. 
Will the regexes still matched correctly?


Mar 11 08:52:04 mx10 dovecot: imap-login: Disconnected (auth failed, 1 attempts in 
2 secs): hid...@example.co.uk>, method=PLAIN, rip=125.69.11.254, 
lip=10.1.1.100, TLS: Disconnected, TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 
bits)
Mar 11 10:18:24 mx10 dovecot: imap-login: Disconnected (auth failed, 1 attempts in 
2 secs): ju...@example.co.uk>, method=PLAIN, rip=37.59.8.29, lip=10.1.1.100, 
TLS: Disconnected, TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)
Mar 11 11:48:04 mx10 dovecot: imap-login: Disconnected (auth failed, 1 attempts in 
2 secs): neoc...@example.co.uk>, method=PLAIN, rip=178.216.98.75, 
lip=10.1.1.100, TLS: Disconnected, TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 
bits)
Mar 11 13:37:39 mx10 dovecot: imap-login: Aborted login (auth failed, 4 attempts 
in 26 secs): junk4>, method=PLAIN, rip=71.213.169.18, lip=10.1.1.100, TLS, 
TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)
Mar 11 13:37:40 mx10 dovecot: imap-login: Aborted login (auth failed, 4 attempts 
in 26 secs): junk4>, method=PLAIN, rip=187.67.197.100, lip=10.1.1.100, TLS, 
TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)
Mar 11 22:35:24 mx10 dovecot: imap-login: Disconnected (auth failed, 1 attempts in 
2 secs): hid...@example.co.uk>, method=PLAIN, rip=182.100.218.83, 
lip=10.1.1.100, TLS, TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)


The jails are enabled in the config. I’ve not see a match for 3 months since I 
installed the server.
[dovecot]
port    = imap,imaps,sieve
logpath = %(dovecot_log)s
backend = %(dovecot_backend)s

[sieve]
port   = smtp,465,submission
logpath = %(dovecot_log)s
backend = %(dovecot_backend)s




On 6 Mar 2018, at 10:50, Tom Hendrikx <t...@whyscream.net> wrote:



On 06-03-18 08:59, Sophie Loewenthal wrote:
Morning,

My logging from and postfix dovecot is in this format:

Mar  6 07:49:45 mx dovecot: imap-login: Login: sop...@example.com>, 
method=PLAIN, rip=94.19.2.3, lip=1.31.1.3, mpid=10655, TLS, TLSv1.2 with cipher 
ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)

Mar  6 07:55:36 mx postfix/smtpd[10793]: Anonymous TLS connection established 
from unknown[94.19.2.3]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 
(256/256 bits)

How can I adapt the filter to pick this up? I don’t think the regex in  
filter.d/postfix.conf|dovecot.conf will pick these changed lines up because 
they have the ciphers included, will they?
Lines that are not understood/matched by fail2ban are ignored.

I don't think these lines signify anything that fail2ban should act on,
but please explain what you would like fail2ban to do, based on those
log lines?

Best wishes,

Sophie






------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Fail2ban-users mailing list
Fail2ban-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fail2ban-users

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Fail2ban-users mailing list
Fail2ban-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fail2ban-users

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Fail2ban-users mailing list
Fail2ban-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fail2ban-users

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Fail2ban-users mailing list
Fail2ban-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fail2ban-users
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Fail2ban-users mailing list
Fail2ban-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fail2ban-users

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Fail2ban-users mailing list
Fail2ban-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fail2ban-users

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Fail2ban-users mailing list
Fail2ban-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fail2ban-users

Reply via email to