On 08/03/2021 00:28, Phillip Carroll wrote:
I have been using fail2ban on CentOS 7 to block hosts (using an IPSET)
based on filters that watch the exim reject log.
I also run the csf/lfd firewall on the same server, which also manages
several block lists using IPSET.
These two packages have happily coexisted for many months. Until March
5. From examining various logs I have pieced together the following
sequence:
At 16:13 (MST), csf began an automatic update of itself. Among the
first things that occurred was some part of the system (perhaps IPSET?)
issued a shutdown to fail2ban. The shutdown started OK but then
apparently encountered a conflict on a system file and fail2ban then
apparently shut down abnormally. I have since been unable to get it to
start up in a normal manner. The filters are not working at all.
(It is worth mentioning here that csf/lfd has successfully updated
itself at least a dozen times in the past with no apparent ill effects
on fail2ban. I will need to go back and look at the logs on those
instances to see if any weirdness occurred that somehow managed to
resolve itself invisibly.)
Following is the fail2ban log content (with lots of debugging info that
I don't fully comprehend) beginning at the time the shutdown began:
2021-03-05 16:13:05,256 fail2ban.server [806]: INFO
Shutdown in progress...
2021-03-05 16:13:05,257 fail2ban.asyncserver [806]: DEBUG Stop
communication, shutdown
2021-03-05 16:13:05,257 fail2ban.observer [806]: INFO
Observer stop ... try to end queue 5 seconds
2021-03-05 16:13:05,320 fail2ban.observer [806]: INFO
Observer stopped, 0 events remaining.
2021-03-05 16:13:05,358 fail2ban.server [806]: INFO
Stopping all jails
2021-03-05 16:13:05,358 fail2ban.jail [806]: DEBUG
Stopping jail 'exim-reject'
2021-03-05 16:13:05,358 fail2ban.filter [806]: INFO Removed
logfile: '/var/log/exim/reject.log'
2021-03-05 16:13:05,358 fail2ban.filterpyinotif [806]: DEBUG Watch
WD=8 (None) removed
2021-03-05 16:13:05,358 fail2ban.filterpyinotif [806]: DEBUG Removed
file watcher for /var/log/exim/reject.log
2021-03-05 16:13:05,359 fail2ban.filterpyinotif [806]: DEBUG Watch
WD=1 (None) removed
2021-03-05 16:13:05,359 fail2ban.filterpyinotif [806]: DEBUG Removed
monitor for the parent directory /var/log/exim
2021-03-05 16:13:05,912 fail2ban.actions [806]: DEBUG Flush
ban list
2021-03-05 16:13:05,913 fail2ban.actions [806]: NOTICE
[exim-reject] Flush ticket(s) with iptables-ipset-proto6
2021-03-05 16:13:05,918 fail2ban.utils [806]: DEBUG
7f480348de70 -- returned successfully 0
2021-03-05 16:13:05,918 fail2ban.actions [806]: NOTICE
[exim-reject] Unban 196.242.244.79
2021-03-05 16:13:05,918 fail2ban.actions [806]: NOTICE
[exim-reject] Unban 45.85.90.208
2021-03-05 16:13:05,918 fail2ban.actions [806]: NOTICE
[exim-reject] Unban 192.241.225.115
2021-03-05 16:13:05,918 fail2ban.actions [806]: NOTICE
[exim-reject] Unban 193.56.29.116
2021-03-05 16:13:05,918 fail2ban.actions [806]: NOTICE
[exim-reject] Unban 196.242.244.42
2021-03-05 16:13:05,918 fail2ban.actions [806]: NOTICE
[exim-reject] Unban 196.242.244.59
2021-03-05 16:13:05,919 fail2ban.actions [806]: DEBUG
Unbanned 6, 0 ticket(s) in 'exim-reject'
2021-03-05 16:13:05,930 fail2ban.utils [806]: ERROR
7f4802b867b0 -- exec: iptables -w -D INPUT -p tcp -m multiport
--dports smtp -m set --match-set f2b-exim-reject src -j REJECT
--reject-with icmp-port-unreachable
ipset flush f2b-exim-reject
ipset destroy f2b-exim-reject
2021-03-05 16:13:05,930 fail2ban.utils [806]: ERROR
7f4802b867b0 -- stderr: 'ipset v7.1: Set cannot be destroyed: it is in
use by a kernel component'
2021-03-05 16:13:05,930 fail2ban.utils [806]: ERROR
7f4802b867b0 -- returned 1
2021-03-05 16:13:05,930 fail2ban.actions [806]: ERROR Failed
to stop jail 'exim-reject' action 'iptables-ipset-proto6': Error
stopping action Jail('exim-reject')/iptables-ipset-proto6: 'Script error'
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/fail2ban/server/actions.py",
line 296, in stopActions
action.stop()
File "/usr/lib/python2.7/site-packages/fail2ban/server/action.py",
line 632, in stop
return self._stop()
File "/usr/lib/python2.7/site-packages/fail2ban/server/action.py",
line 655, in _stop
return self._executeOperation('<actionstop>', 'stopping',
family=family, afterExec=_stopped)
File "/usr/lib/python2.7/site-packages/fail2ban/server/action.py",
line 463, in _executeOperation
raise RuntimeError("Error %s action %s/%s: %r" % (operation,
self._jail, self._name, err))
RuntimeError: Error stopping action
Jail('exim-reject')/iptables-ipset-proto6: 'Script error'
2021-03-05 16:13:05,932 fail2ban.actions [806]: DEBUG
exim-reject: action iptables-ipset-proto6 terminated
2021-03-05 16:13:05,964 fail2ban.filterpyinotif [806]: DEBUG
[exim-reject] filter exited (pyinotifier)
2021-03-05 16:13:06,560 fail2ban.filterpyinotif [806]: DEBUG
[exim-reject] filter terminated (pyinotifier)
2021-03-05 16:13:06,560 fail2ban.jail [806]: INFO Jail
'exim-reject' stopped
2021-03-05 16:13:06,561 fail2ban.database [806]: DEBUG Close
connection to database ...
2021-03-05 16:13:06,561 fail2ban.database [806]: INFO
Connection to database closed.
2021-03-05 16:13:06,562 fail2ban.asyncserver [806]: DEBUG Removed
socket file /var/run/fail2ban/fail2ban.sock
2021-03-05 16:13:06,562 fail2ban.asyncserver [806]: DEBUG Socket
shutdown
2021-03-05 16:13:06,562 fail2ban.server [806]: INFO Exiting
Fail2ban
2021-03-05 16:13:06,562 fail2ban.server [806]: DEBUG Remove
PID file /var/run/fail2ban/fail2ban.pid
2021-03-05 16:13:06,562 fail2ban [806]: DEBUG Exit
with code 0
Did f2b get an update? If so I have found in the past that some custom
jails have failed and you may have to revisit them.
All your script is showing is that, when shutting down, iptables is
failing to remove a rule for some reason (try running the command
manually). This leaves the rule in place which then means ipset can't
delete the set.
There is nothing here to show why f2b is not starting.
Nick
_______________________________________________
Fail2ban-users mailing list
Fail2ban-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fail2ban-users
