Bug#847728: fail2ban: Fail2ban running shorewall instructions before shorewall is started

2017-08-16 Thread Graham Bosworth 
Hello again,

I now have version 0.9.7 installed but not actually working very well.  On
the previous installation, I had needed to butcher some of the regular
expressions so that they would be triggered.  With the arrival of 0.9.7,
the stock expressions are back, and they are not catching as much.

Also, the changes have not prevented the failure to stop nicely:-

---%<---
[Gentoo] graham@kevin $ sudo -v && time ( sudo /etc/init.d/shorewall
restart && sudo /etc/init.d/fail2ban restart )
Password for graham@kevin:
 * Stopping shorewall ... [
ok ]
 * Starting shorewall ... [
ok ]
 * Stopping fail2ban ...
 * start-stop-daemon: 1 process refused to stop
 * Failed to stop fail2ban[
!! ]
 * ERROR: fail2ban failed to stop

real1m2.495s
user0m35.794s
sys 0m10.741s   load74.46%
Wed Aug 16 01:52:40
--->%---

You may like to have a look at a bug report for Gentoo on what I think is
the same topic:-
https://bugs.gentoo.org/show_bug.cgi?id=618138
I incorporated the change suggested there into the service script on my
computer, and restarted the service.  While a single instance is not a
representative sample, I see that the command takes longer but runs
cleanly:-

---%<---
[Gentoo] graham@kevin $ sudo -v && time sudo /etc/init.d/fail2ban restart
Password for graham@kevin:
 * Stopping fail2ban ...  [
ok ]
 * Starting fail2ban ...
2017-08-16 15:47:54,088 fail2ban.server [23071]: INFOStarting
Fail2ban v0.9.7
2017-08-16 15:47:54,098 fail2ban.server [23071]: INFOStarting
in daemon mode  [ ok ]

real1m12.196s
user0m37.613s
sys 0m2.371sload55.38%
Wed Aug 16 15:48:20 /
--->%---


Thanks,
Graham



On 10 August 2017 at 03:55, Brian Flaherty  wrote:

> Thanks for bumping this. I am not sure what package should fix this, but I
> don't think it is an error in fail2ban, I think the problem is that systemd
> doesn't start shorewall before fail2ban, so the firewall structure is not
> running yet. I've "fixed" the problem by adding
>
> shorewall.service
>
> to the After line in the fail2ban.service file in /lib/systemd/system.
>
> > /lib/systemd/system# cat fail2ban.service
> [Unit]
> Description=Fail2Ban Service
> Documentation=man:fail2ban(1)
> After=network.target iptables.service firewalld.service shorewall.service
> PartOf=iptables.service firewalld.service
>
> [Service]
> Type=forking
> ExecStart=/usr/bin/fail2ban-client -x start
> ExecStop=/usr/bin/fail2ban-client stop
> ExecReload=/usr/bin/fail2ban-client reload
> PIDFile=/var/run/fail2ban/fail2ban.pid
> Restart=always
>
> [Install]
> WantedBy=multi-user.target
>
>
> I have to redo it every time fail2ban is upgraded.
>
> Brian
>
>
>

Bug#847728: fail2ban: Fail2ban running shorewall instructions before shorewall is started

2017-08-09 Thread Yaroslav Halchenko 

On Tue, 08 Aug 2017, Graham Bosworth wrote:


>### Note that this is from Gentoo, rather than Debian

yeah

>On a Pentium at 200MHz, 

wow!!! do you have a physical beast like that from 90s?

> it seems that it can indeed terminateprematurely.

any chance you could also try 0.9.7 version?  or even better (eventually
we will switch there) 0.10 branch version from github (there was lots of
changed behaviors, hopefully for the best)

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


signature.asc
Description: PGP signature

Bug#847728: fail2ban: Fail2ban running shorewall instructions before shorewall is started

2017-08-08 Thread Graham Bosworth 
Hello,

I see that this bug report has gone a bit cold.  It came first in responses
to a Web search for information about a problem that I have recently
noticed.  The most recent visible update asks 'I wonder though if "service
fail2ban stop" exits prematurely'.  I think it might.

#
##
### Note that this is from Gentoo, rather than Debian
##
#

On a Pentium at 200MHz, it seems that it can indeed terminate prematurely.
Trying to restart Fail2Ban can result in an error, but it is not
guaranteed.  Here is output from a failure:-

---%<---
[Gentoo] graham@kevin $ sudo /etc/init.d/fail2ban restart
 * Caching service dependencies ...   [
ok ]
 * Stopping fail2ban ...
 * start-stop-daemon: 1 process refused to stop
 * Failed to stop fail2ban[
!! ]
 * ERROR: fail2ban failed to stop
Tue Aug 08 14:06:04 /usr/src/linux-4.9.34-gentoo
[Gentoo] graham@kevin $ sudo /etc/init.d/fail2ban stop
 * Stopping fail2ban ...  [
ok ]
Tue Aug 08 14:06:15 /usr/src/linux-4.9.34-gentoo
[Gentoo] graham@kevin $ ps aux | grep -e "f2b" -e "fail2ban"
graham   17610  0.0  1.0   4616  1880 pts/0S+   14:06   0:00 grep
--colour=auto -e f2b -e fail2ban
Tue Aug 08 14:06:21 /usr/src/linux-4.9.34-gentoo
[Gentoo] graham@kevin $ sudo /etc/init.d/fail2ban start
 * Starting fail2ban ...
2017-08-08 14:06:57,813 fail2ban.server [17655]: INFOStarting
Fail2ban v0.9.6
2017-08-08 14:06:57,823 fail2ban.server [17655]: INFOStarting
in daemon mode
  [ ok ]
Tue Aug 08 14:07:39 /usr/src/linux-4.9.34-gentoo
[Gentoo] graham@kevin $ ps aux | grep -e "f2b" -e "fail2ban"
root 17697 35.5  5.2  65868  9632 ?Sl   14:06   0:16
/usr/bin/python3.4 /usr/bin/fail2ban-server -s /run/fail2ban/fail2ban.sock
-p /run/fail2ban/fail2ban.pid -b
graham   17764  0.0  1.0   4616  1880 pts/0S+   14:07   0:00 grep
--colour=auto -e f2b -e fail2ban
Tue Aug 08 14:07:46 /usr/src/linux-4.9.34-gentoo
--->%---


Later, on trying to repeat the exercise, there was no problem detected:-

---%<---
[Gentoo] graham@kevin $ sudo -v && time sudo /etc/init.d/fail2ban restart
 * Stopping fail2ban ...  [
ok ]
 * Starting fail2ban ...
2017-08-08 15:41:48,570 fail2ban.server [25644]: INFOStarting
Fail2ban v0.9.6
2017-08-08 15:41:48,583 fail2ban.server [25644]: INFOStarting
in daemon mode
  [ ok ]

real1m15.999s
user0m41.864s
sys 0m3.598sload59.81%
Tue Aug 08 15:42:32 /usr/src/linux-4.9.34-gentoo
--->%---


There is another observation: there are occasions when fail2ban cannot ban
or unban an address because iptables does not contain any chains beginning
"f2b" - the rules disappear.  Here's a log fragment that tells part of the
story:-

---%<---
2017-08-06 11:28:43,466 fail2ban.action [31847]: ERROR   iptables
-w -n
-L INPUT | grep -q 'f2b-ssh-iptables[ \t]' -- returned 1
2017-08-06 11:28:43,472 fail2ban.CommandAction  [31847]: ERROR   Invariant
check
 failed. Trying to restore a sane environment
2017-08-06 11:28:43,798 fail2ban.action [31847]: ERROR   iptables
-w -D
INPUT -p tcp -m multiport --dports 0:65535 -j f2b-ssh-iptables
iptables -w -F f2b-ssh-iptables
iptables -w -X f2b-ssh-iptables -- stdout: b''
2017-08-06 11:28:43,806 fail2ban.action [31847]: ERROR   iptables
-w -D
INPUT -p tcp -m multiport --dports 0:65535 -j f2b-ssh-iptables
iptables -w -F f2b-ssh-iptables
iptables -w -X f2b-ssh-iptables -- stderr: b"iptables v1.4.21: Couldn't
load tar
get `f2b-ssh-iptables':No such file or directory\n\nTry `iptables -h' or
'iptabl
es --help' for more information.\niptables: No chain/target/match by that
name.\
niptables: No chain/target/match by that name.\n"
2017-08-06 11:28:43,813 fail2ban.action [31847]: ERROR   iptables
-w -D
INPUT -p tcp -m multiport --dports 0:65535 -j f2b-ssh-iptables
iptables -w -F f2b-ssh-iptables
iptables -w -X f2b-ssh-iptables -- returned 1
2017-08-06 11:28:43,820 fail2ban.actions[31847]: ERROR   Failed to
execu
te unban jail 'ssh-iptables' action 'iptables-multiport' info '{'time':
15020141
22.2688327, 'matches': 'Aug  6 11:08:28 kevin sshd[18419]: Invalid user 0
from 9
1.197.232.11 port 52798Aug  6 11:08:30 kevin sshd[18424]: Invalid user 
from
 91.197.232.11 port 43927Aug  6 11:08:33 kevin sshd[18426]: Invalid user
010101
from 91.197.232.11 port 40298Aug  6 11:08:36 kevin sshd[18428]: Invalid
user 111
1 from 91.197.232.11 port 36500Aug  6 11:08:40 kevin sshd[18447]:
Connection clo
sed by 91.197.232.11 port 60791 [preauth]', 'ip': '91.197.232.11',
'failures': 5
}': Error stopping action
--->%---

I hope this helps,
-- 
Graham Bosworth

Bug#847728: fail2ban: Fail2ban running shorewall instructions before shorewall is started

2016-12-13 Thread Yaroslav Halchenko 

On Tue, 13 Dec 2016, Brian Flaherty wrote:

> Have just confirmed this twice with clean reboots. Get a similar pattern of 
> errors in the fail2ban log. (I guess I've been getting them all along, but 
> didn't know to look for the difference. Sorry.)I know very little about 
> systemd, but I've tried to see a directive for the service file that tells 
> shorewall to wait for fail2ban to shutdown, but I haven't seen anything that 
> looks like it accomplishes that. I'll post something to Debian User and see 
> if someone can help me figure out how to solve this. Then I'll forward 
> here.In case you are interested, here's a recent section of the fail2ban.log. 
> Thanks for maintaining fail2ban!
> 2016-12-13 11:01:40,912 fail2ban.server [1348]: INFO    Stopping all 
> jails
> 2016-12-13 11:01:41,507 fail2ban.actions    [1348]: NOTICE  [sshd] Unban 
> 37.120.168.213
> 2016-12-13 11:01:41,612 fail2ban.action [1348]: ERROR   shorewall 
> allow 37.120.168.213 -- stdout: b''
> 2016-12-13 11:01:41,612 fail2ban.action [1348]: ERROR   shorewall 
> allow 37.120.168.213 -- stderr: b'   ERROR: Shorewall is not started\n'
> 2016-12-13 11:01:41,612 fail2ban.action [1348]: ERROR   shorewall 
> allow 37.120.168.213 -- returned 2
> 2016-12-13 11:01:41,612 fail2ban.actions    [1348]: ERROR   Failed to 
> execute unban jail 'sshd' action 'shorewall' info '{'ip': '37.120.168.213', 
> 'matches': 'Dec 13 09:56:37 stendahl sshd[17638]: pam_unix(sshd:auth): 
> authentication failure; logname= uid=0 euid=0 tty=ssh ruser= 
> rhost=37.120.168.213  user=rootDec 13 09:56:39 stendahl sshd[17638]: Failed 
> password for root from 37.120.168.213 port 44679 ssh2Dec 13 09:56:41 stendahl 
> sshd[17638]: Failed password for root from 37.120.168.213 port 44679 ssh2Dec 
> 13 09:56:44 stendahl sshd[17638]: Failed password for root from 
> 37.120.168.213 port 44679 ssh2', 'time': 1481655184.2452223, 'failures': 4}': 
> Error unbanning 37.120.168.213

So you added shorewall.service to After field of  fail2ban.service and
that made fail2ban start after shorewall but what seems not to
stop before shorewall... correct?

I wonder though if "service fail2ban stop" exits prematurely (i.e. not
waiting for all actions to complete -- and there is over half a second
which passes  from "Stopping all jails" to Unban) thus giving it to
shorewall to exit before fail2ban actually exits.  Since iirc 0.9.6
included some reworkings of startup -- you could try it as well.  but
meanwhile could verify on what is happening when you do "service
fail2ban stop" -- does it exit too quickly?

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

Bug#847728: fail2ban: Fail2ban running shorewall instructions before shorewall is started

2016-12-10 Thread Yaroslav Halchenko 

On Sat, 10 Dec 2016, Brian Flaherty wrote:
> However, if I stop and start fail2ban after shorewall is running from the 
> prompt. I don't get any errors.

> In the fail2ban.service file in /usr/lib/systemd/system, iptables and 
> firewalld are nammed in "After", but shorewall isn't. Can that be included?


> $ cat /lib/systemd/system/fail2ban.service
> [Unit]
> Description=Fail2Ban Service
> Documentation=man:fail2ban(1)
> After=network.target iptables.service firewalld.service
> PartOf=iptables.service firewalld.service

> [Service]
> Type=forking
> ExecStart=/usr/bin/fail2ban-client -x start
> ExecStop=/usr/bin/fail2ban-client stop
> ExecReload=/usr/bin/fail2ban-client reload
> PIDFile=/var/run/fail2ban/fail2ban.pid
> Restart=always

> [Install]
> WantedBy=multi-user.target


> If I add it myself, I assume it will be overwritten when an updated version 
> is installed.

heh -- just uploaded fresh release... could you give it a shot and see
if it resolves it.  Would be appreciated!

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik