Re: SMTP over IPv6 : gmail classifying lots of IPv6 mail as spam since 20140818
Le Friday 22 August 2014 à 07:16 -0700, Lorenzo Colitti a écrit : On Fri, Aug 22, 2014 at 12:56 AM, Laurent GUERBY laur...@guerby.net wrote: We've been running SMTP over IPv6 with postfix successfully for over a year and since 20140818 gmail.com IPv6 MX started to classify most IPv6 sourced emails sent from our machine to @gmail.com as spam. The exact same message sent using IPv4 within one minute of the IPv6 bounce is accepted. As there's no way to reach google mailops we had to remove IPv6 from our mail machines and go back to IPv4 only for mail, which is sad. Are you following the Additional guidelines for IPv6 section of https://support.google.com/mail/answer/81126 ? Hi, We have reverse and SPF in place so I believe we're following them: root@lists:~# dig +short TXT lists.tetaneutral.net v=spf1 mx a ptr:lists.tetaneutral.net ip4:91.224.149.207 ip6:2a01:6600:8081:cf00::1 -all root@lists:~# dig +short -x 2a01:6600:8081:cf00::1 lists.tetaneutral.net. (The for lists.tetaneutral.net is currently removed until the current issue with IPv6 is sorted out.) I did some statistics in the last few days since 20140818: IPv6: root@lists:~# grep gmail-smtp-in /var/log/mail.log|grep ::|grep status=sent|wc -l 236 root@lists:~# grep gmail-smtp-in /var/log/mail.log|grep ::|grep status=bounced|grep message is likely unsolicited mail|wc -l 29 IPv4: root@lists:~# grep gmail-smtp-in /var/log/mail.log|grep -v ::|grep status=sent|wc -l 564 root@lists:~# grep gmail-smtp-in /var/log/mail.log|grep -v ::|grep status=bounced|grep message is likely unsolicited mail|wc -l 0 We have another MX out host with similar statistics: about 10-15% of rejected mails as spam in IPv6 since 20140818 and exactly none in IPv4. For reference the same statistics on logs before 20140818: root@lists:~# grep gmail-smtp-in /var/log/mail.log-20140817|grep ::|grep status=sent|wc -l 1135 root@lists:~# grep gmail-smtp-in /var/log/mail.log-20140817|grep ::|grep status=bounced|grep message is likely unsolicited mail|wc -l 0 root@lists:~# grep gmail-smtp-in /var/log/mail.log-20140817|grep -v ::|grep status=sent|wc -l 778 root@lists:~# grep gmail-smtp-in /var/log/mail.log-20140817|grep -v ::|grep status=bounced|grep message is likely unsolicited mail|wc -l 0 Absolutely zero issue with IPv6 and it has been the case for more than a year until 20140818. Note: I changed the subject since it's not nearly all IPv6 mail sorry I did not take time to make some statistics in the first mail. Sincerely, Laurent
Re: SMTP over IPv6 : gmail classifying nearly all IPv6 mail as spam since 20140818
Le 23 août 2014 à 07:51, Michael Chang thenewm...@gmail.com a écrit : I was under the impression that it wasn't so much about there being more IPv6 spam as much as tracking IPv6 reputation based on addresses was computationally infeasible. If a spammer gets a hold of a /64, then the spammer can send 18 billion billion (~2^64) different email addresses, each coming from a different IP address. Never-mind that a spammer can go to a half-dozen tunnel brokers and get /48s for free. Indeed, if your repudiation algorithm is naïve. Blacklisting by /128 is not viable. But you can definitely filter by /64. For smaller prefixes (/48, /56), you can try to put a reputation on prefixes (depending on the number of /64 you already blacklisted) in order to blacklist the entire prefix. Best regards. Emmanuel Thierry On Fri, Aug 22, 2014 at 8:18 PM, Brian E Carpenter brian.e.carpen...@gmail.com wrote: On 23/08/2014 11:16, Dan Wing wrote: On Aug 22, 2014, at 7:42 AM, Matthew Huff mh...@ox.com wrote: Currently it is not feasible to do ipv6 reputation filtering. IPv4 reputation filtering is a big part of most anti-spam engines, so without it, SPF / DKIM of domain reputation is the best alternative. BTW, we have had to remove all IPv6 from our mail gateways due to the large number of Exchange SBS with broken isatap/6to4 tunnels causing mail to blackhole. MTU issue? I can't speak for Teredo, but for 6to4 there is a whole list of possible issues ( http://tools.ietf.org/html/rfc6343 ). PMTUD failure and/or MSS negotiation failure are on the list, and so is reverse DNS failure. Brian -d These have been at small web based retailers which don't have hosted email. After the third incident, we yanked our IPv6 from our MX/gateways. Matthew Huff | 1 Manhattanville Rd Director of Operations | Purchase, NY 10577 OTA Management LLC | Phone: 914-460-4039 -Original Message- From: ipv6-ops-bounces+mhuff=ox@lists.cluenet.de [mailto:ipv6-ops-bounces+mhuff=ox@lists.cluenet.de] On Behalf Of Nick Hilliard Sent: Friday, August 22, 2014 10:25 AM To: Lorenzo Colitti; Laurent GUERBY Cc: IPv6 Ops list Subject: Re: SMTP over IPv6 : gmail classifying nearly all IPv6 mail as spam since 20140818 On 22/08/2014 15:16, Lorenzo Colitti wrote: Are you following the Additional guidelines for IPv6 section of https://support.google.com/mail/answer/81126 ? Lorenzo, it looks like Google is trying to enforce SPF / DKIM on ipv6 connections where there is no similar requirement for ipv4. Is there a particular reason for this? It's causing a lot of breakage. Nick
Re: SMTP over IPv6 : gmail classifying nearly all IPv6 mail as spam since 20140818
Hi, On Fri, Aug 22, 2014 at 10:51:26PM -0700, Michael Chang wrote: If a spammer gets a hold of a /64, then the spammer can send 18 billion billion (~2^64) different email addresses, each coming from a different IP address. Never-mind that a spammer can go to a half-dozen tunnel brokers and get /48s for free. And if the reputation system is worth a cup of salt, it will notice that it already has down-graded sufficient addresses inside the /64 (/56, /48, /32) to down-grade the rest of it. Just because it needs a bit more thinking than for IPv4 doesn't mean it cannot be done. Gert Doering -- NetMaster -- have you enabled IPv6 on something today...? SpaceNet AGVorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (0)89/32356-444 USt-IdNr.: DE813185279
Re: SMTP over IPv6 : gmail classifying nearly all IPv6 mail as spam since 20140818
On 22 Aug 2014, at 17:56, Lorenzo Colitti lore...@google.com wrote: I'm not on the gmail team and don't have those numbers. Nick asked me for an answer, and I gave him what information I have. My assumption was that since they do receive a lot of email, they have statistics on this, but of course you may not agree with that assumption and assume that they're just doing this for whatever other arbitrary reason. No doubt they're good at what they do, but if mta operators felt that Google's FP detection rate was acceptable for v6 originated mail, we wouldn't be having this discussion - particularly as the other major mail operators don't appear to have this issue. Nick
Re: SMTP over IPv6 : gmail classifying nearly all IPv6 mail as spam since 20140818
FWIW, I agree with Matthew 100%, especially about the fact that the SMTP world is changing. It's also worth noting that it's been in constant (although not always rapid) flux since I first got involved in Internet stuff 20+ years ago. Back then it was common for any connected system to be able to send mail, nowadays that's unthinkable of course. Also FWIW, since I know Nick a bit based on his postings on other lists it's probably worth pointing out that in all likelihood his concern here is based on not slowing down IPv6 adoption. That's a worthy goal, which I'm fully in support of. However the change has been blowing in the wind towards SPF/DKIM/DMARC now for years, and as Matthew pointed out there are only going to be more folks requiring it, not less. Fortunately SPF is dead simple, and DKIM isn't that much harder. In fact for one domain it's also dead simple (ProTip: Use OpenDKIM). I couldn't find a good, concise, up to date guide on using OpenDKIM for multiple domains, so it took me 2 tries to get it right, but I'm happy to write up my results if there's a need. DMARC is also pretty painless, although right now I'm set up for report only, at least until the mailing list software folks get that problem fixed. I do find it interesting to see how often mail is being Joe-jobbed from some of my mostly-unused domains though. So yes, rDNS/SPF/DKIM at minimum to get in the game, regardless of your IP protocol. DMARC is highly recommended. I realize that change is always painful, more so to folks who've been in the game just long enough to be really comfortable with their IPv4 address-based reputation stuff. But the times, they are a-changin'. Doug On 8/23/14 8:52 AM, Matthew Huff wrote: Nick, I would expect the response will be silence. Since the current RBL methods are not currently operational with IPv6 due to design issues and that IPv4 reputation is a large part of anti-spam, there is a fundamental difference currently between the two protocols. As IPv6 smtp ramps up, I would expect more to move to Googles direction than vice versa. The idea that you will be able to send email from an IPv6 address without rDNS, SPF and DKIM and have it end up in anything other than the spam folder is a pipe dream. Hell, I helped a friend that was running a hosted domain with only IPv4 and he had difficulty getting email delivered to corporate emails systems without SPF/DKIM. The SMTP world is changing, I doubt it is going to go back. -Original Message- From: ipv6-ops-bounces+mhuff=ox@lists.cluenet.de [mailto:ipv6-ops-bounces+mhuff=ox@lists.cluenet.de] On Behalf Of Nick Hilliard Sent: Saturday, August 23, 2014 11:37 AM To: Lorenzo Colitti Cc: IPv6 Ops list; Marco d'Itri; Jared Mauch Subject: Re: SMTP over IPv6 : gmail classifying nearly all IPv6 mail as spam since 20140818 On 22 Aug 2014, at 20:26, Lorenzo Colitti lore...@google.com wrote: What specifically would you like me to pass on? Dear gmail team, can you please publicly present data on IPv4 spam vs IPv6 spam in order to justify your documented policy? ? How about: Dear gmail team, v6 mta operators have noticed that there is a substantial difference between how spam detection is handled for ipv4 and ipv6 connections and this appears to be causing problems with high rates of false positives on v6 sessions. These problems appear to be specific to gmail and are not seen with connections to other major mail operators. Where SPF/dkim are not feasible/possible, this causes people to either implement gmail specific hacks or else disable ipv6. Both these workarounds act against the interests of both Google and the internet at large. Can you please reach out to the ipv6 operator community about this? ? Nick
Re: SMTP over IPv6 : gmail classifying nearly all IPv6 mail as spam since 20140818
On Aug 23, Brian E Carpenter brian.e.carpen...@gmail.com wrote: Actually I think you should quibble. The issue isn't bad software used by intermediaries, it's that by design DMARC p=reject breaks a very common model used by intermediaries. Whether that is a bug or a feature in DMARC is out of scope for this thread, however. I was not referring to the mailing lists issue, which is not relevant unless you also use DMARC with p=reject, but to broken MTAs which mangle forwarded messages (look for DKIM validation failures and you will easily find many). There is a reason if DMARC was designed to use both SPF and DKIM. -- ciao, Marco signature.asc Description: Digital signature
Re: SMTP over IPv6 : gmail classifying nearly all IPv6 mail as spam since 20140818
On 24/08/2014 09:20, Marco d'Itri wrote: On Aug 23, Brian E Carpenter brian.e.carpen...@gmail.com wrote: Actually I think you should quibble. The issue isn't bad software used by intermediaries, it's that by design DMARC p=reject breaks a very common model used by intermediaries. Whether that is a bug or a feature in DMARC is out of scope for this thread, however. I was not referring to the mailing lists issue, which is not relevant unless you also use DMARC with p=reject, but to broken MTAs which mangle forwarded messages (look for DKIM validation failures and you will easily find many). Oh, OK, that wasn't obvious from the context. Brian There is a reason if DMARC was designed to use both SPF and DKIM.
