|
Charles, For #2 you can easily manually add the 17,000 IP's to ipset using the command "ipset add {set-name} IP_address". It would be trivial to create a script to do it, or put your list of IP's into Excel, create a text field with "ipset add {set-name} " then join the fields together and copy and paste the results into PuTTy or a single executable file. Can I give you another idea? Have a look at the file on Emerging Threats, https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt. I've scripted that into a couple of ipset sets (one for IP's, one for subnets). I also do some other stuff, but this is a very good start. Blocking subnets gives less rules than individual IP's. Regards, Nick On 12/02/2016 13:49, Charles Bradshaw
wrote:
Bill, Sorry again, I actually miss read your first reply. I read actionban instead of actionunban.I am indeed saving and restoring the ipset. At least, that's what I used to do until I found fail2ban taking hours to shutdown. Last time I hit the boot button after about an hour with the result that the ipset was left intact. ipset has built in and well documented method for backup and restore. While I understand your proposed method and see how it would work, I make the following observations: 1 - Your method has a certain pragmatic elegance, but is devious and will certainly confuse the uninitiated! 2 - I can see how your method will work if implemented from square one, but what about the 17000 odds IP which have been previously band with a ban time of forever? I've been running the particular jail with bantime = -1 for well over a year now. 3 - Why store anything at all in an external database. Ipsets are just that, a highly efficient linked to iptables database. The botnet problem is increasing rapidly. Today I'm seeing 8/hour originally it was 2 or 3. In the meantime > 17000 IP have been permanently banned. That says there are botnets out there with orders more than 10000 infected machines! We know not when this will, in effect, escalate to Denial of Service! Several hours to shutdown is a kind of DNS! Back on a pragmatic front, storing and manipulating vast amounts of duplicate data is simply not good practice. If you look out there you will find much discussion on the subject of how to unban the inadvertently banned. I might be wrong, but I suspect because sqlite permanent banning was implemented without due consideration of the consequences on existing installations. I think what I really need to understand now is; how does fail2ban 'think' an IP is banned or not. Where is the database? When is it written/read? In what version of fail2ban did sqlite get implemented. At present my /var/lib/fail2ban/fail2ban.sqlite3 has 7.9MB of entries. I ask again how do I turn sqlite activity off? Just point me at the documentation. Charles Bradshaw On Thu, 2016-02-11 at 22:31 -0500, Bill Shirley wrote:When you said: This leaves the ipset intact. I made the the assumption, maybe incorrectly, that you were saving your ipset with some utility on shutdown and restoring after a re-boot. If that IS the case then change your jail to: bantime = 60 and make actionunban empty in your .local action: #actionunban = ipset -exist del fail2ban-<name> <ip> actionunban = fail2ban will ban the IP address and in one minute it will unban it. However, with actionunban being empty, the IP address will not be removed from the ipset. So now fail2ban thinks very few, if any, addresses are banned. With very few addresses to 'remove', shutdown should be quick. Bill On 2/11/2016 7:03 PM, Charles Bradshaw wrote:Thanks Bill, Sorry I'm being a bit dim. Do you mean to temporarily modify the actionban in /etc/fail2ban/action.d/myaction.conf before the shutdown? How does that affect the shutdown? I can see how it affects the restart but eh.. no action actionban no bans at all after restart! Surely deleting the actionstop clause altogether, thus preventing deletion of the ipset and a modified actionstart to do nothing if the ipset already exists. Then neither start nor stop take time. I see the new sqlite behavior, but then where is the reference to dbfile forcing all the bans into /var/lib/fail2ban/fail2ban.sqlite3 it is not in my fail2ban.conf! If its use is default behaviour how do I disable it? On Thu, 2016-02-11 at 12:19 -0500, Bill Shirley wrote:Try using an empty actionunban in your action and set the bantime = 60 in your jail. This way fail2ban thinks it's unbanning after a minute. fail2ban shutdown should be quick. Bill On 2/11/2016 5:15 AM, Charles Bradshaw wrote:Hello list, I am running fail2ban.noarch 0.9.3-1.el6.1 as installed from the CentOS repository. I have one ipset jail which over time has accumulated more than 17000 permanent bans. This is causing a severe problem during restarts. (obviously!) First it would take many hours to shut down fail2ban gracefully the solution is to force a power down. This leaves the ipset intact. Next when the fail2ban server restarts it takes a similar many hours for the server to redundantly restore the bans from the database to the already intact ipset. This a ridiculous process! The whole purpose of ipsets is to efficiently hold vast numbers of blocked IPs. The most importantly problem here is fail2ban is preventing fast clean shutdowns. Understand 17000 bans is nothing! an ipset can efficiently hold > 65K, under which circumstances the shutdown and restart delays would extend to weeks!! The startup delay is not a severe problem except that 17000 emails and all the disk activity is a total pain in the ass. So the question is: how to turn off fail2ban gracefully without these ridiculous delays. Also note when fail2ban shuts down the ipset entries in iptables do not get deleted, but that's another story. Thanks in advance, Charles Bradshaw ------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 _______________________________________________ Fail2ban-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/fail2ban-users------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 _______________________________________________ Fail2ban-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/fail2ban-users------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 _______________________________________________ Fail2ban-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/fail2ban-users------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 _______________________________________________ Fail2ban-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/fail2ban-users------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 _______________________________________________ Fail2ban-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/fail2ban-users |
------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________ Fail2ban-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/fail2ban-users
