1 SMTP Exceeded Hard Error Limit after CONNECT
1 RBL spamdomains.blackholes.easynet.nl
3 SMTP Exceeded Hard Error Limit after END-OF-MESSAGE
3 RBL opm.blitzed.org
4 DNS no A/MX for @recipient.domain
9 ACL helo_hostnames
16 SMTP invalid [EMAIL PROTECTED]
17 SMTP Exceeded Hard Error Limit after ETRN
24 SMTP helo hostname invalid
28 ACL header checks
30 ACL mta_clients_bw
32 ACL unauthorized relay
39 ETRN Mail theft attempt
49 ACL bogon network header
88 ACL HTML obfuscation
89 SMTP unauthorized pipelining
95 SMTP invalid [EMAIL PROTECTED]
109 ACL PTR hostname does not match hostname (forged HELO)
113 RBL list.dsbl.org
186 RBL proxies.relays.monkeys.com
190 RBL relays.ordb.org
256 SMTP Exceeded Hard Error Limit after MAIL
353 DNS nxdomain for MTA PTR hostname (forged @sender.domain)
414 RBL korea.services.net
425 DNS timeout for MTA PTR hostname (forged @sender.domain)
632 RBL dnsbl.njabl.org
634 ACL forged @sender.domain not from sender PTR domain
676 DNS no A/MX for @sender.domain
919 ACL from_senders_bw
1990 SMTP helo hostname is an IP
2323 SMTP helo hostname not fully qualified
2697 ACL from_senders_imgfx
3126 RBL sbl.spamhaus.org
4978 RBL blackholes.easynet.nl
13887 ACL from_senders_slet
18091 ACL mta_clients_dict
28076 SMTP Exceeded Hard Error Limit after DATA
69412 ACL subscriber network <<<<<<<<<<
73298 SMTP Exceeded Hard Error Limit after RCPT
96529 RBL dynablock.easynet.nl <<<<<<<<<<
111142 ACL to_relay_recipients unknown recipient
======================
430984 TOTAL rejectsSo the dynablock rejected 96.5 K msgs (16k are oranges, 80k are apples), and then the subscriber filter rejected an additional 69K msgs (apples), passed over by dynablock.
The subscriber rejects contain 6087 unique IPs.
The dynablock rejects contain 11556 unique IPs, 1811 of which have no PTR (oranges), meaning they would not be blocked by my subscriber filter which filters by PTR domain name. Note that dynablock has the disadvantage, when it works, and a vulnerability when it doesn't, of having made, at least, 11556 DNS queries to the dynablock remote server. My subscriber filter is a local ACL file, much faster (scaleable) and more reliable (no killing your mail server with timeouts for unreachable/slow/DDoSed RBL servers).
I scanned through the PTR hostnames of the dynablock rejects, and there is of course a huge amount of overlap between what dynablock rejects and what my subscriber filter rejects. Whatever, the outcome is that dynablock let pass 70K pieces of subscriber junk. (But I'm pretty sure I know what's so inferior about the dynablock concept approach to my subscriber filter approach.).
I'd bet good money that if I had run the experiment backwards, with my subscriber filter first, and dynablock second (and disqualifying the dynablock "oranges" of 16k rejects of 1811 PTR-less IPS since they are illegit by definition and not targeted by the subscriber "apples" filter), of the remaining total of
69K + (97K - 16K PTR-less) = 150 K PTR-full rejects
.... would have been very probably 150K rejected by subscriber and "round off error" rejected by dynablock. :)
The basic question is, for the subscriber filter, how many of the total 6097 blocked IPs are legit mail servers?
1% would be 61 IPs, 0.1% (1 in 1000) would be 6 IPs.
I found 4 IPs that were legit (all whitelisted now), with intelligible [EMAIL PROTECTED] and the helo domain fully qualified agreeing, and a credible website, but in all these blocked_legit cases, the PTR hostname was the network operator's domain (mismtaching the server's HELO hostname), exactly the same problem that Scott has with charter for mail.declude.com's IP.
I am willing to extrapolate from the 4 IPs I found to 20, allowing that I could have missed 10 to 15 legit IPs that sent only 1 or 2 msgs each. There was certainly no way, looking at the crap, that 100s of IPs were legit, or even the 60 IPs for a 1% rate. There is ABSOLUTELY no way the 70K subscriber msgs that were accepted as ok by dynablock were all legit.
So I estimate that 20/6087 = 0.3% would be a rough, max rate for legit IPs blocked by the subscriber filter, which would be reduced to well below that by running subscriber filter in warn_if_reject mode for a week to catch all the habitual legit senders screwed up on subscriber PTR hostnames.
For your amusement, some observations on the crap:
The top end of the rejects per IP:
8301 209.219.48.148
3840 209.219.48.136
2660 64.105.160.171
2225 64.105.160.174
1620 209.219.48.144
1252 199.222.161.177
1145 209.219.48.147
798 64.105.160.172
459 216.239.225.50
294 65.30.207.98
200 24.164.14.148
174 66.235.1.59
174 24.74.164.110
166 210.63.226.136
162 66.41.74.152
154 24.71.64.93
152 66.190.98.202
148 142.179.159.114
145 24.95.229.31
145 24.168.217.243
143 24.81.201.121
143 24.24.202.31
138 24.160.72.229
138 24.128.216.220
136 217.58.148.138
135 66.255.179.178
124 66.7.18.133
122 24.174.216.72
121 24.85.86.250
119 67.119.169.178
117 24.194.171.40
113 194.144.54.55
112 69.132.32.170
111 24.87.230.37
111 24.195.60.215
etc .... for 6087 IPs13674 subscriber rejects also had a not-fully qualified helo hostname, a single criteria that also merits blocking.
For the combined total of dyna + subs rejects:
151322 rejected msgs said HELO and only 14642 said EHLO. What good-faith mail servers send HELO? postfix defaults to EHLO, and I suppose IMail, Exchange, exim, sendmail, qmail all send EHLO unless programmed not to. EHLO is preferred because it gets the ExtendedSMTP service announcements like SIZE and MIME support and other useful features that legit servers like to work with. otoh, spammers prefer HELO since the don't want the delay of ESMTP announcements coming back. Subscriber IPs sending over 10:1 HELO:EHLO is another mark against their legitimacy.
In flagrant delicto, an entire ClassC (only one IP shown below) at transedge exploiting the Veriscam DNS validation hole:
136.48.219.209.transedge.com[209.219.48.136]: from=<[EMAIL PROTECTED]> proto=SMTP helo=<bgcemx.com>
148.48.219.209.transedge.com[209.219.48.148]: from=<[EMAIL PROTECTED]> proto=SMTP helo=<bgdcul.com>
148.48.219.209.transedge.com[209.219.48.148]: from=<[EMAIL PROTECTED]> proto=SMTP helo=<bgdcul.com>
148.48.219.209.transedge.com[209.219.48.148]: from=<[EMAIL PROTECTED]> proto=SMTP helo=<bgdplq.com>
148.48.219.209.transedge.com[209.219.48.148]: from=<[EMAIL PROTECTED]> proto=SMTP helo=<bgfhzz.com>
148.48.219.209.transedge.com[209.219.48.148]: from=<[EMAIL PROTECTED]> proto=SMTP helo=<bglxnt.com>
148.48.219.209.transedge.com[209.219.48.148]: from=<[EMAIL PROTECTED]> proto=SMTP helo=<bglxnt.com>
148.48.219.209.transedge.com[209.219.48.148]: from=<[EMAIL PROTECTED]> proto=SMTP helo=<bglxnt.com>
148.48.219.209.transedge.com[209.219.48.148]: from=<[EMAIL PROTECTED]> proto=SMTP helo=<bglxnt.com>
Note on the above: sender@ is fixed length garbage added to forged hotmail.com, and the helo domain is fully qualified junk hostname under Veriscam .com. You can block MTAs in 209.219.48/24 or with PTR transedge.com in complete tranquility. :)
@sender.domain forgeries of the usual frequently forged domains:
@aol 4702 @hotmail 21677 @msn 1173 @earthlink 1214 @yahoo 7324 @bigfoot 298 @concentric 123 @compuserve 168 @cs. 260 @email 101 @erols 110 @free 208 @gmx 891 @juno 180 @lycos 95 @nycmail 1676 @one of our domains 1411 @prodigy 141
For the subscriber rejects, I have a file containing the PTR[ip], [EMAIL PROTECTED], SMTP PROTOCOL, and HELO hostname fields if anybody wants to see it, about 1 MB zipped.
So for this experiment, the dynablock filter let pass about 70K msgs that were blocked with my subscriber filter, where 99+% of those 70K were true crap. The subscriber filter had a credible legit_IP/abuse_IP ratio max of 20/6087. Let's say those 20 legit IPs sent an avg of 5 legit msgs/day (about only about 2000 of the 6100 IPs sent had 5 or more rejects). 100 legit msg in 70K spam is roughly .2%. Good enough.
If you guys want to nitpick and hairsplit to 4 decimal places, have at it. I don't find that kind of accuracy helpful in running a mail operation.
These results totally refute Scott's unsupported, self-serving FUD of "(subscriber filter) can have very, very high false positive rate". I put up, you shut up.
This analysis also refutes Scott's claim for the dynablock filter being "better" than my subscriber filter. If it's so "better", why did it let pass 70k pieces of subscriber junk, a failure rate of approaching 50%?
The above analysis of a sufficiently large sample supports my claim that msgs originating from subscriber networks are 99+% crap, merit blanket blocking, single-criteria rejection, and are finally just illegitmate networks at this time. When the network operators stop the effluent, we'll stop the blanket blocking. (Is anybody holding their breath for the stench to clear?)
Len
To Unsubscribe: http://www.ipswitch.com/support/mailing-lists.html List Archive: http://www.mail-archive.com/imail_forum%40list.ipswitch.com/ Knowledge Base/FAQ: http://www.ipswitch.com/support/IMail/
