Bill, Sorry again, I actually miss read your first reply. I read actionban instead of actionunban.
I am indeed saving and restoring the ipset. At least, that's what I used to do until I found fail2ban taking hours to shutdown. Last time I hit the boot button after about an hour with the result that the ipset was left intact. ipset has built in and well documented method for backup and restore. While I understand your proposed method and see how it would work, I make the following observations: 1 - Your method has a certain pragmatic elegance, but is devious and will certainly confuse the uninitiated! 2 - I can see how your method will work if implemented from square one, but what about the 17000 odds IP which have been previously band with a ban time of forever? I've been running the particular jail with bantime = -1 for well over a year now. 3 - Why store anything at all in an external database. Ipsets are just that, a highly efficient linked to iptables database. The botnet problem is increasing rapidly. Today I'm seeing 8/hour originally it was 2 or 3. In the meantime > 17000 IP have been permanently banned. That says there are botnets out there with orders more than 10000 infected machines! We know not when this will, in effect, escalate to Denial of Service! Several hours to shutdown is a kind of DNS! Back on a pragmatic front, storing and manipulating vast amounts of duplicate data is simply not good practice. If you look out there you will find much discussion on the subject of how to unban the inadvertently banned. I might be wrong, but I suspect because sqlite permanent banning was implemented without due consideration of the consequences on existing installations. I think what I really need to understand now is; how does fail2ban 'think' an IP is banned or not. Where is the database? When is it written/read? In what version of fail2ban did sqlite get implemented. At present my /var/lib/fail2ban/fail2ban.sqlite3 has 7.9MB of entries. I ask again how do I turn sqlite activity off? Just point me at the documentation. Charles Bradshaw On Thu, 2016-02-11 at 22:31 -0500, Bill Shirley wrote: > When you said: > This leaves the ipset intact. > I made the the assumption, maybe incorrectly, that you were saving > your ipset with some utility on shutdown and restoring after a > re-boot. > > If that IS the case then change your jail to: > bantime = 60 > > and make actionunban empty in your .local action: > #actionunban = ipset -exist del fail2ban-<name> <ip> > actionunban = > > fail2ban will ban the IP address and in one minute it will unban it. > However, with actionunban being empty, the IP address will not be > removed from the ipset. So now fail2ban thinks very few, if any, > addresses are banned. With very few addresses to 'remove', shutdown > should be quick. > > Bill > > > On 2/11/2016 7:03 PM, Charles Bradshaw wrote: > > > Thanks Bill, > > > > Sorry I'm being a bit dim. Do you mean to temporarily modify the > > actionban in /etc/fail2ban/action.d/myaction.conf before the shutdown? > > How does that affect the shutdown? I can see how it affects the restart > > but eh.. no action actionban no bans at all after restart! > > > > Surely deleting the actionstop clause altogether, thus preventing > > deletion of the ipset and a modified actionstart to do nothing if the > > ipset already exists. Then neither start nor stop take time. > > > > I see the new sqlite behavior, but then where is the reference to dbfile > > forcing all the bans into /var/lib/fail2ban/fail2ban.sqlite3 it is not > > in my fail2ban.conf! If its use is default behaviour how do I disable > > it? > > > > On Thu, 2016-02-11 at 12:19 -0500, Bill Shirley wrote: > > > Try using an empty actionunban in your action and set the bantime = 60 in > > > your jail. This way fail2ban thinks it's unbanning > > > after a minute. fail2ban shutdown should be quick. > > > > > > Bill > > > > > > On 2/11/2016 5:15 AM, Charles Bradshaw wrote: > > > > Hello list, > > > > > > > > I am running fail2ban.noarch 0.9.3-1.el6.1 as installed from the CentOS > > > > repository. > > > > > > > > I have one ipset jail which over time has accumulated more than 17000 > > > > permanent bans. This is causing a severe problem during restarts. > > > > (obviously!) > > > > > > > > First it would take many hours to shut down fail2ban gracefully the > > > > solution is to force a power down. This leaves the ipset intact. > > > > > > > > Next when the fail2ban server restarts it takes a similar many hours for > > > > the server to redundantly restore the bans from the database to the > > > > already intact ipset. > > > > > > > > This a ridiculous process! The whole purpose of ipsets is to efficiently > > > > hold vast numbers of blocked IPs. > > > > > > > > The most importantly problem here is fail2ban is preventing fast clean > > > > shutdowns. Understand 17000 bans is nothing! an ipset can efficiently > > > > hold > 65K, under which circumstances the shutdown and restart delays > > > > would extend to weeks!! The startup delay is not a severe problem except > > > > that 17000 emails and all the disk activity is a total pain in the ass. > > > > > > > > So the question is: how to turn off fail2ban gracefully without these > > > > ridiculous delays. > > > > > > > > Also note when fail2ban shuts down the ipset entries in iptables do not > > > > get deleted, but that's another story. > > > > > > > > Thanks in advance, Charles Bradshaw > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------------ > > > > Site24x7 APM Insight: Get Deep Visibility into Application Performance > > > > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month > > > > Monitor end-to-end web transactions and take corrective actions now > > > > Troubleshoot faster and improve end-user experience. Signup Now! > > > > http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 > > > > _______________________________________________ > > > > Fail2ban-users mailing list > > > > [email protected] > > > > https://lists.sourceforge.net/lists/listinfo/fail2ban-users > > > > > > ------------------------------------------------------------------------------ > > > Site24x7 APM Insight: Get Deep Visibility into Application Performance > > > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month > > > Monitor end-to-end web transactions and take corrective actions now > > > Troubleshoot faster and improve end-user experience. Signup Now! > > > http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 > > > _______________________________________________ > > > Fail2ban-users mailing list > > > [email protected] > > > https://lists.sourceforge.net/lists/listinfo/fail2ban-users > > > > > > ------------------------------------------------------------------------------ > > Site24x7 APM Insight: Get Deep Visibility into Application Performance > > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month > > Monitor end-to-end web transactions and take corrective actions now > > Troubleshoot faster and improve end-user experience. Signup Now! > > http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 > > _______________________________________________ > > Fail2ban-users mailing list > > [email protected] > > https://lists.sourceforge.net/lists/listinfo/fail2ban-users > > ------------------------------------------------------------------------------ > Site24x7 APM Insight: Get Deep Visibility into Application Performance > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month > Monitor end-to-end web transactions and take corrective actions now > Troubleshoot faster and improve end-user experience. Signup Now! > http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 > _______________________________________________ Fail2ban-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/fail2ban-users ------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 _______________________________________________ Fail2ban-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/fail2ban-users
