On 02/24/2014 06:03 PM, Eric Shubert wrote: > On 02/24/2014 11:28 AM, Cecil Yother, Jr. wrote: >> >> On 02/24/2014 09:46 AM, Angus McIntyre wrote: >>> On Feb 22, 2014, at 12:18 PM, Eric Shubert <[email protected]> wrote: >>>> It's not a terrible idea though. I wonder if fail2ban could be >>>> configured to count DENIED_RDNS messages for each IP address, and >>>> if there were more than a certain number of failed attempts in a >>>> given time period, then block the IP address. >>>> >>>> I'd like to hear from anyone with more familiarity with F2B than >>>> myself about this possibility. This might be an additional F2B >>>> configuration we could include. >>> >>> For Fail2Ban to be effective, you'd want (a) repeat attempts from >>> one IP to be large enough that you could usefully reduce load by >>> banning offenders, and (b) the total number of offenders to be small >>> enough that you don't end up adding thousands of entries to >>> iptables. (I'm assuming that if you add enough entries to iptables >>> you will eventually start seeing some kind of a performance hit, but >>> I don't know if that is in fact the case, or where the cutoff point >>> comes). >>> >>> I did a quick informal study with: >>> >>> grep DENIED_RDNS /var/log/qmail/smtp/current | awk '{print $9}' >>> | sort | uniq -c | grep -v " 1 " | grep -v " 2 " >>> >>> to find out how many hosts made 3 or more attempts. >>> >>> In a sample based on about 90 minutes worth of mail, there were just >>> over 100 hosts, only 8 of which made 3 or more attempts to deliver. >>> The 'worst offender' made 15 attempts. >>> >>> In a larger sample, representing about 10 days worth of mail, 8206 >>> hosts were denied, with the 'worst offender' making 1325 attempts, >>> and 528 hosts making 3 or more attempts. >>> >>> In my (unscientific) sample, just under 7000 of the hosts sent a >>> single message each. There were 27 hosts that tried to send more >>> than 100 messages, and that accounted for just over 10,000 messages, >>> about 40% of the total. Of course, if you set 100 as the cut-off >>> point, you can't ban them until they deliver their 100th message, so >>> - based on my sample - you'd reduce load by about 25%. >>> >>> That doesn't sound bad, though. >>> >>> If you set the ban point at 20 messages, you'd ban about 100 hosts, >>> and reduce load by 47%. >>> >>> Banning at 10 messages leads to banning 150 hosts, and reduces load >>> by about 52%. >>> >>> But I've been looking at 10 days of mail, which is probably >>> unreasonable. Fail2Ban allows you to tune the 'findtime' for a jail >>> (i.e. the period for which Fail2Ban will track failed attempts). The >>> default seems to be 600 seconds (10 minutes). Would a 1-day window >>> be appropriate for this application? >>> >>> cat `find /var/log/qmail/smtp/ -mtime -1` | grep DENIED_RDNS | >>> awk '{print $9}' | more | sort | uniq -c >>> >>> gets me results for just 1 day. >>> >>> A 1-day sample of my messages turned up 941 unique hosts, and 2672 >>> messages. Banning after 10 failures would have banned 43 hosts and >>> reduced load by about 37% (i.e. 37% of the attempted delivery >>> connections would have been rejected by iptables rather than spamdyke). >>> >>> Incidentally, things get interesting if you look at networks rather >>> than individual IPs. There are definite 'clusters', and it could be >>> that banning a few selected class C's would have substantial payoff. >>> (The old joke about eliminating 95% of all spam by null-routing >>> China, Florida and BurstNet comes to mind). But you might not want >>> to give a robot power to ban class C's without supervision ("Things >>> have gotten awfully quiet around here lately ...") >>> >>> I think you could write a Fail2Ban rule, and depending on where you >>> set the cutoff point, it would probably reduce your server load by >>> 25-50% of the difference between banning-by-spamdyke and >>> banning-by-firewall. Assume that if you keep the number of entries >>> in iptables reasonably low, there's no cost to an iptables ban. >>> Assume also that having fail2ban scan the quickly-changing >>> smtp/current log and tracking up to n hosts (where 'n' is the number >>> of hosts failing for RDNS_DENIED per day) is also 'free'. Then you >>> can reduce load by 25-50% of however much load spamdyke is putting >>> on your box. And I have no idea how to calculate what that might be. >>> >>> TL;DR: It looks as if this approach would offer savings, but you >>> need to look at usage patterns on your own servers to figure out how >>> big those savings might be. >>> >>> Angus >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: >>> [email protected] >>> >> Nice analysis Angus. >> >> On my server I allow 3 attempts then you're gone for 24 hours. Since I >> have e mail confirmation of bans I know it's reducing the load >> considerably during peak SPAM barrages. I know in my application it has >> certainly cut down on unwanted attempts to SPAM or gain access through >> password guessing. >> >> I've seen delays in administrating iptables (hand editing) when the >> files are large, but no noticable difference in access speed. This is >> only anecdotal and I have no hard number to back up my claim. That was >> on another server where I did block the China IP's and several Russian >> ones. I no longer do that since I have customers in Russia and contacts >> in China. >> >> CJ >> > > I'd say Angus's analysis is beyond "nice". Top notch (A+) IMHO. Nice > work. > > Tony, I appreciate your work, and no doubt you learned a few things > along the way, which is good. None the less, we should be trying to > leverage other FOSS software as much as practical. In this case, I > think F2B is an appropriate tool to use for this. Let's not reinvent > any wheels if we can possibly avoid doing so. > > Which still begs the question: Is this worth doing at all? > I'd like to know what SamC thinks about this. He's intimately familiar > with spamdyke's inner workings (to say the least), and I expect would > have some meaningful insight regarding the savings involved. Would > someone care to run this by the spamdyke list? I'm on that list (too), > and see all activity there (fwiw). > > My "feeling" at this point is still that it's not worth the effort > from an efficiency point of view. spamdyke already is terribly > efficient, so any savings there would be relatively small. There would > also be additional load in other areas (iptables, log scanning) which > would offset any savings. > > There might be unintended consequences of such an approach though, > such as the spam host being smart enough to quit after being banned w/ > no connection (a different type of failure). This would be an > interesting aspect to attempt to measure. If true, it could be an > effective tool. I'm not inclined to think that spamming software is > quite that 'smart' though. > > Last (and possibly least - ;) ). CJ, I thought you were running > postfix now. If that's the case, you couldn't possibly be running > spamdyke (yet, but perhaps a future release). You could of course be > running RBLs with postfix, and I'm guessing that you're also doing > some sort of rDNS checking and verification. What exactly are you > basing your "3 strikes" rule on? You're correct. Since Postfix doesn't have a Spamdyke utility I have setup Fail2ban to basically do the same thing. While it's not the same it seems to do the trick. I ban no host name, no rDNS and failed login attempts after 3 tries for 24 hours. It's made a measurable difference on the load on the server. > > Thanks everyone. Very interesting discussion. >
-- --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
