On 02/24/2014 11:28 AM, Cecil Yother, Jr. wrote:
On 02/24/2014 09:46 AM, Angus McIntyre wrote:
On Feb 22, 2014, at 12:18 PM, Eric Shubert <[email protected]> wrote:
It's not a terrible idea though. I wonder if fail2ban could be configured to
count DENIED_RDNS messages for each IP address, and if there were more than a
certain number of failed attempts in a given time period, then block the IP
address.
I'd like to hear from anyone with more familiarity with F2B than myself about
this possibility. This might be an additional F2B configuration we could
include.
For Fail2Ban to be effective, you'd want (a) repeat attempts from one IP to be
large enough that you could usefully reduce load by banning offenders, and (b)
the total number of offenders to be small enough that you don't end up adding
thousands of entries to iptables. (I'm assuming that if you add enough entries
to iptables you will eventually start seeing some kind of a performance hit,
but I don't know if that is in fact the case, or where the cutoff point comes).
I did a quick informal study with:
grep DENIED_RDNS /var/log/qmail/smtp/current | awk '{print $9}' | sort | uniq -c | grep -v
" 1 " | grep -v " 2 "
to find out how many hosts made 3 or more attempts.
In a sample based on about 90 minutes worth of mail, there were just over 100
hosts, only 8 of which made 3 or more attempts to deliver. The 'worst offender'
made 15 attempts.
In a larger sample, representing about 10 days worth of mail, 8206 hosts were
denied, with the 'worst offender' making 1325 attempts, and 528 hosts making 3
or more attempts.
In my (unscientific) sample, just under 7000 of the hosts sent a single message
each. There were 27 hosts that tried to send more than 100 messages, and that
accounted for just over 10,000 messages, about 40% of the total. Of course, if
you set 100 as the cut-off point, you can't ban them until they deliver their
100th message, so - based on my sample - you'd reduce load by about 25%.
That doesn't sound bad, though.
If you set the ban point at 20 messages, you'd ban about 100 hosts, and reduce
load by 47%.
Banning at 10 messages leads to banning 150 hosts, and reduces load by about
52%.
But I've been looking at 10 days of mail, which is probably unreasonable.
Fail2Ban allows you to tune the 'findtime' for a jail (i.e. the period for
which Fail2Ban will track failed attempts). The default seems to be 600 seconds
(10 minutes). Would a 1-day window be appropriate for this application?
cat `find /var/log/qmail/smtp/ -mtime -1` | grep DENIED_RDNS | awk
'{print $9}' | more | sort | uniq -c
gets me results for just 1 day.
A 1-day sample of my messages turned up 941 unique hosts, and 2672 messages.
Banning after 10 failures would have banned 43 hosts and reduced load by about
37% (i.e. 37% of the attempted delivery connections would have been rejected by
iptables rather than spamdyke).
Incidentally, things get interesting if you look at networks rather than individual IPs.
There are definite 'clusters', and it could be that banning a few selected class C's
would have substantial payoff. (The old joke about eliminating 95% of all spam by
null-routing China, Florida and BurstNet comes to mind). But you might not want to give a
robot power to ban class C's without supervision ("Things have gotten awfully quiet
around here lately ...")
I think you could write a Fail2Ban rule, and depending on where you set the
cutoff point, it would probably reduce your server load by 25-50% of the
difference between banning-by-spamdyke and banning-by-firewall. Assume that if
you keep the number of entries in iptables reasonably low, there's no cost to
an iptables ban. Assume also that having fail2ban scan the quickly-changing
smtp/current log and tracking up to n hosts (where 'n' is the number of hosts
failing for RDNS_DENIED per day) is also 'free'. Then you can reduce load by
25-50% of however much load spamdyke is putting on your box. And I have no idea
how to calculate what that might be.
TL;DR: It looks as if this approach would offer savings, but you need to look
at usage patterns on your own servers to figure out how big those savings might
be.
Angus
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Nice analysis Angus.
On my server I allow 3 attempts then you're gone for 24 hours. Since I
have e mail confirmation of bans I know it's reducing the load
considerably during peak SPAM barrages. I know in my application it has
certainly cut down on unwanted attempts to SPAM or gain access through
password guessing.
I've seen delays in administrating iptables (hand editing) when the
files are large, but no noticable difference in access speed. This is
only anecdotal and I have no hard number to back up my claim. That was
on another server where I did block the China IP's and several Russian
ones. I no longer do that since I have customers in Russia and contacts
in China.
CJ
I'd say Angus's analysis is beyond "nice". Top notch (A+) IMHO. Nice work.
Tony, I appreciate your work, and no doubt you learned a few things
along the way, which is good. None the less, we should be trying to
leverage other FOSS software as much as practical. In this case, I think
F2B is an appropriate tool to use for this. Let's not reinvent any
wheels if we can possibly avoid doing so.
Which still begs the question: Is this worth doing at all?
I'd like to know what SamC thinks about this. He's intimately familiar
with spamdyke's inner workings (to say the least), and I expect would
have some meaningful insight regarding the savings involved. Would
someone care to run this by the spamdyke list? I'm on that list (too),
and see all activity there (fwiw).
My "feeling" at this point is still that it's not worth the effort from
an efficiency point of view. spamdyke already is terribly efficient, so
any savings there would be relatively small. There would also be
additional load in other areas (iptables, log scanning) which would
offset any savings.
There might be unintended consequences of such an approach though, such
as the spam host being smart enough to quit after being banned w/ no
connection (a different type of failure). This would be an interesting
aspect to attempt to measure. If true, it could be an effective tool.
I'm not inclined to think that spamming software is quite that 'smart'
though.
Last (and possibly least - ;) ). CJ, I thought you were running postfix
now. If that's the case, you couldn't possibly be running spamdyke (yet,
but perhaps a future release). You could of course be running RBLs with
postfix, and I'm guessing that you're also doing some sort of rDNS
checking and verification. What exactly are you basing your "3 strikes"
rule on?
Thanks everyone. Very interesting discussion.
--
-Eric 'shubes'
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
