How much more sampling must I/we do to find a counter-example that would negate all the sampling so far?
It isn't a matter of the *amount* of sampling, it's a matter of the *quality* of the samples.
Think of it this way: Suppose a city hires you to determine if the water in their 100 lakes is safe to drink. You do a sample of the 20 lakes closest to you, and all turn out to be safe. I point out that you don't have a random sample, because you are choosing the ones closest to you (it may be that the other side of town has a factory that pollutes many of the lakes on that side of town, for example). You then say "But I've taken samples from those 20 lakes 5 times already!"
Statistics just doesn't work that way. You can take samples those non-random 20 lakes until you are blue in the face -- and while you'll get very accurate results for those 20 lakes, you won't get accurate results for the city.
For example, we have a domain that has only 2 valid accounts on it, but that gets a ton of spam. That domain will probably show 98% of the E-mail from France as being spam. But that of course does not mean that 98% of the E-mail from France is spam.
That's a really bad, silly example, and advances the discussion nil, and proves nothing.
It does show a good point: If that server you tested with gets 99% spam, the information you provided would be interpreted very differently than if it came from a server which gets 1% spam.
[2] You don't take into account that spammers won't remove your E-mail address, but legitimate people will.
What does that have to do with all the alphabet soup senders@, and the forged HELO hostname (not an accident when an rr.com subscriber says HELO as microsoft.com, and 1000's of them do it), and forged @sender.domains, and mismatch between @sender.domain and HELO domain?
Come on, Len.
You took an action, and aren't taking it into account with your numbers. Your results today indicate that almost all E-mail from Charter is spam. But, you are blocking all mail from Charter, so anyone who tried sending you E-mail from a Charter IP over the past few months is either no longer communicating with you, or re-routing their E-mail. You don't include those people in your sample. So you are just saying "Charter spammers continue to try to send spam after I block them, but legitimate Charter customers do not."
You're essentially just saying "I don't get legitimate E-mail anymore from people who I have blocked"
Quit attributing to me what I didn't say.
But you did, with your numbers. You don't understand statistics, but make it seem like you do. You used tainted data, that made the information you presented nearly identical to the quote above.
I said "blocking subscriber networks is reliable and effective way to stop a horrendous source of mail abuse"
You didn't (that's what you *meant* to say). And I agree with that statement 100%, just as I agree 100% with "blocking all E-mail is a reliable and effective way to stop a horrendous source of mail abuse". But, both are flawed.
Why not just use something like EASYNET-DNSBL, which is much more accurate, as legitimate mailers can get removed?
Sending mail to my MX is a right that I grant, not a right you are born with and can ram down my throat.
And you certainly can block whatever mail you want. If you don't mind blocking some legitimate E-mail, that's your choice. But please don't push your ideas on others without explaining that they will lose legitimate E-mail.
[3] You are assuming that you can tell by an E-mail address that an E-mail is spam.
The report in my msg referred to charter PTR hostname, MAIL FROM, and HELO hostname. Anybody on this list, without any agenda to push, can see
1) charter PTRs were sending 2) crap MAIL FROM and 3) crap HELO hostnames
We don't need to accept the DATA command, and read it to see whether it's legit or not. We all discard junk letters by looking at envelope only. Email is the same.
Well, Len, you listed "from=<[EMAIL PROTECTED]> helo=<margaret.mollerus.org>" as spam. Could you please explain to me how you know it is spam (or in your words, "crap", "abuse", and "illegitimate)?
In fact, at least one of the E-mail addresses you posted is from a domain of an IMail customer I recognize (no, not declude.com!). Some appear to be from a mailing list.
So? If they come from subscriber PTRs, they're illegit.
How so? Are you saying that a small business that has a business class connection (where they pay extra for business class service, and have a TOS that allows them to run a server) is illegitimate? How so?
Come on, Len. Learn statistics before you mis-use words like "impossible".
If a mail admin decides his policy is to block all of .kr, edu.tw, ac.tr, dial-up networks, or cable/DSL networks, then it is impossible, by definition, to have false positives.
No, Len. In the field of spam control, "false positive" means "a non-spam that was caught by the anti-spam software".
If you are going to use terms outside of their normal definitions, you must define them each time you use them. Using your definition, no spam test has any false positives!
* Hobbyists that run their own mailserver
Hobby mailers? GMAFB. We are running a business for our mostly business customers.
That's exactly why nobody should listen to you! That's proof that your sample of E-mail is not a good sample.
If you want people to take you seriously, state ALL the relevant facts. State clearly that "false positive" doesn't mean what everyone expects it to mean. State clearly what "illegitimate mail" means to you.
A large percentage of IMail users are ISPs, web hosts, colleges, and small businesses -- almost all of whom need to be able to get mail from those hobbiests.
How about using something like EASYNET-DNSBL, which lists IPs on subscriber networks, but removes ones that send legitimate mail? All of a sudden, you'll improve your false positive rate tremendously.
Blocking all subscriber networks with my local ACLs:
1) is more efficient and scaleable (no DNS query delays)
Please, Len.
Your ACL test requires a reverse DNS lookup (often requiring several queries). EASYNET-DNSBL requires a forward DNS lookup (requiring just 1 query).
However, using EASYNET-DNSBL, you can probably get permission to do a zone transfer to significantly speed up lookups and reduce network traffic.
Advantage: EASYNET-DNSBL.
2) my mailer is less vulnerable (no near-fatal delays on my mail server because osirus or easynet RBL servers are DDoSed or otherwise unreachable)
It's extremely rare for DNS-based (no, not RBL -- that's a trademark of MAPS -- how long have you been dealing with spam control?) spam databases to cause delivery problems like the Osirusoft one (note that Joe Jared did that intentionally; it was not a side effect of the DDoS attack). With a DDoS attack, you may have some extra spam make it past that test (assuming you are not using zone transfers), but other tests can catch the spam.
3) and more accurate (no RBL server is can possibly keep up with the huge volumes of DSL/cable networks that are now and will be deployed.)
Wrong, Len.
EASYNET-DNSBL probably lists more subscriber networks than your ACL test, so it has the advantage there of catching more spam. Plus, it can easily whitelist IPs, so a single "subscriber" can contact one source to get all his mail through, as opposed to having to contact lots of different people. That's more accurate.
And how is it that when a new DSL/cable network is deployed it is easier for you to find out about it and add their reverse DNS information, than it is for the DNS-based spam test to list its IPs?
The base problem is that almost no network operators police their subscriber networks for mail abuse.
Agreed.
-Scott
---
Declude JunkMail: The advanced anti-spam solution for IMail mailservers.
Declude Virus: Catches known viruses and is the leader in mailserver vulnerability detection.
Find out what you've been missing: Ask about our free 30-day evaluation.
--- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]
To Unsubscribe: http://www.ipswitch.com/support/mailing-lists.html List Archive: http://www.mail-archive.com/imail_forum%40list.ipswitch.com/ Knowledge Base/FAQ: http://www.ipswitch.com/support/IMail/
