Re: [Mimedefang] Sender Address Verification

2016-11-23 Thread Richard Laager
On 11/22/2016 12:55 PM, Bill Cole wrote:
> the SAV rule was never decisive in a correct SA 'spam' determination

Thanks for sharing. This is good information.

I've made a note to re-evaluate my SAV rules after the holiday. I have
some test harnesses to determine whether an individual rule "made a
difference" in the outcome of the spam decision. If it turns out it's
useless for me, I can rip out my SAV code, which is a win regardless of
one's viewpoint on whether it is abusive.

-- 
Richard
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang


Re: [Mimedefang] Sender Address Verification

2016-11-23 Thread Bill Cole

On 19 Nov 2016, at 7:01, Simon Standley wrote:


Guys,

In conversations on this list in days gone by, I seem to recall sender 
address verification via the likes of smf-sav milter, or through 
mimedefang, was considered evil heresy, likely to get you blacklisted.


Yes. For good reason. If you ever have random addresses in a domain you 
handle forged on a big spam run aimed at one of the few sites still 
doing SAV, you'll understand why. Some years ago miscreants could (and 
did) use Verizon's SAV-ing mail system as a tool of reflection DDoS 
attacks.


Recently though, I've seen an increasing number of recipients do this 
kind of thing when we send mail out, and personally I've always liked 
the idea.


And yet indiscriminate SAV remains a fundamentally abusive practice, 
unchanged by how much of it you see or what you think of it...


SAV is a mechanism for offloading your anti-spam work to random innocent 
victims of forgery. You can mitigate that abuse to a great extent by 
only doing SAV when a SPF check of the sender domain returns an 
affirmative result, but when you restrict your use of SAV to that degree 
you sharply reduce the ratios of repudiation to verification AND of 
correct repudiation to incorrect repudiation. Or in simpler terms: it 
becomes less useful and less accurate. Doing SAV on an address that 
fails or softfails a SPF check is lazy and abusive.


For this reason, I've been trying out various schemes on a test 
domain, and find tagging mail which cannot be replied to (for reason 
of non-existent user, rather than broken DNS) for later 'mark-up' by 
SpamAssassin, works quite well.


Can you quantify that "quite well?" How often is the SAV tag decisive in 
catching spam?


I was wondering what current thoughts were re- this kind of approach, 
and if anyone else had good/bad results to share?


I help run a mail system where the ultimate policy authority had been an 
unmovable fan of SAV for many years, despite my insistence that it was 
JUST WRONG and not really very useful as an adjunct to SA because 
forging undeliverable senders is an obsolete tactic of shoddy spammers 
whose crap is mostly going to score in double digits anyway, with much 
of the rest still triggering SA autolearning as spam. After 2 years of 
pleading, I got him to accept tag+filter SAV instead of outright 
rejection. This revealed that not only was I correct in my prediction 
(the SAV rule was never decisive in a correct SA 'spam' determination in 
the course of 6 weeks) but that the SAV implementation was flawed, 
interpreting some 5xx replies to RCPT as "no such user" incorrectly and 
causing incorrect classification as spam. This got worse in week 7, when 
the IP address used for SAV did in fact land on some blacklists because 
one or more of the addresses it tried to test were spamtraps.


___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang


Re: [Mimedefang] Sender Address Verification

2016-11-23 Thread Richard Laager
We have been doing sender address verification for years.

Looking through the code...

We use custom MIMEDefang code around DNS::Resolver and
md_check_against_smtp_server(). We wrap the md check in a block and use
alarm() to timeout, as MD's timeout doesn't always work (e.g. if the
server is intentionally or unintentionally tarpitting where TCP responds
quickly but SMTP is slow). We do our callback from the NULL sender
address. We do not perform the check if the sender to us is the NULL sender.

If we get a TEMPFAIL result, but the message matches /gr[ae]ylist/i, we
treat the address as valid.

We exempt *.bounces.google.com because they fail verification and we
don't want the score increase from it. We special-case *@charter.net, as
they (at least at one time) reject the NULL sender completely! We accept
MXes that point to CNAMEs, even though that is an RFC violation.

We cache invalid results for 1 hour and valid results for 7 days. We add
1.5 points for a tempfail and 3 points for a reject. None of these
values were exhaustively optimized.

-- 
Richard
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang