Re: new plugin: helo
On Jun 15, 2012, at 12:32 PM, Charlie Brady wrote: > On Fri, 15 Jun 2012, Matt Simerson wrote: > >>> In my experience, reverse DNS is frequently managed (or, frequently, not >>> managed) by an ISP, not by the admin of the mail server. >> >> In my experience, every ISP that provides clients with dedicated IPs >> also provides the client with the ability to manage the reverse DNS for >> them. > > My ISP (a smallish not-for-profit affair) claims to be unable to do that. > I've heard of and seen other ISPs which fail also. I agree that most ISPs > offer this service. A single exception is sufficient to say this can't be > guaranteed. Why does this matter? Having a custom FQDN that you can manage is not required by RFC 2821 or my helo plugin. The RFC requires that you use the FQDN (and presumably matching rDNS) that's assigned to your IP as the HELO hostname. If your smallish ISP isn't capable of managing DNS properly, and you don't have a meaningful FQDN, then you use an address literal. The RFC tests in the new helo plugin are VERY low bars to clear. In the default 'lenient' mode, no DNS validation is performed at all. In RFC mode, if you present a HELO hostname, then that hostname must have SOME kind of working DNS. It doesn't even have to be a match unless 'policy strict' is chosen. Of the 40,494 messages in my logs, 22,418 of them have invalid HELO hostnames and 158 messages are using address literals. Almost half (67) of the literals are from valid users of my mail server. Which is why I don't block address literals. But I can safely do so now without causing my users problems because of the deferred rejection and auth clearing features of the naughty plugin. Of the 22,418 messages with invalid HELO hostnames, Windows based senders accounted for 18,971. Linux accounted for 3,172 more, no doubt consisting of compromised and illegally obtained VPSes. After I further eliminate the senders who had relay_client set, what's left is a small handful of valid servers using invalid HELO hostnames, like mail.es.freebsd.org, nkNNpNNmm-asmtpout001.mac.com (where the NN numbers mismatch), and lists.openbsd.org. In all of those cases, the servers DO have valid forward and reverse DNS, as defined in the helo plugin tests. It's just that the DNS does not match. Of course, there's also plenty of matching HELO hostnames with DNS that doesn't match, due to server clusters (Yahoo, Gmail, .mac, etc) and the like, which is why the no_matching_dns test only validates that the domain is the same. The false positive rate with 'policy rfc' is presently 0%. With 'policy strict', the FP rate is 0.025%. If only 2 servers had matching DNS (mail.es.freebsd.org and lists.openbsd.org), the FP rate for 'policy strict' as currently implemented would be 0%. Matt
Re: new plugin: helo
On Fri, 15 Jun 2012, Matt Simerson wrote: > > In my experience, reverse DNS is frequently managed (or, frequently, not > > managed) by an ISP, not by the admin of the mail server. > > In my experience, every ISP that provides clients with dedicated IPs > also provides the client with the ability to manage the reverse DNS for > them. My ISP (a smallish not-for-profit affair) claims to be unable to do that. I've heard of and seen other ISPs which fail also. I agree that most ISPs offer this service. A single exception is sufficient to say this can't be guaranteed. > Either via a support request or automated means. I did when I > owned my ISP in the early 90s, and every ISP and hosting provider I've > ever worked for or with since does as well. The good providers even > provide rwhois on the delegated IP space and automatically populate it > with the clients billing data. (see RFC 1941). Yes, good providers do that.
Re: new plugin: helo
On Jun 15, 2012, at 6:44 AM, Charlie Brady wrote: > On Wed, 13 Jun 2012, Matt Simerson wrote: > >> https://github.com/smtpd/qpsmtpd/pull/29 > > I consider this statement to be troublesome also: > > Per RFC 2821, the HELO hostname must be the FQDN of the sending server or > address literal. -Per RFC 2821, the HELO hostname must be the FQDN of the sending server or an +Per RFC 2821, the HELO hostname is the FQDN of the sending server or an s/must be/is/ > RFC 2821 doesn't say "must". It says neither should nor must, but simply > > The argument field contains the fully-qualified domain name of the SMTP > client if one is available. In that very same paragraph, and the preceding one, the RFC spells out the purpose of the HELO hostname, which is to identify the sending system. I believe that's why the RFC writers chose to use the definite article when defining the FQDN, rather than saying "a FQDN," or "any FQDN," or "anything that suits your fancy." The purpose is explicit, to identify the sending system. > Mail servers can certainly be expected to have a FQDN. But expecting them > to have *only one* FQDN may be too tight a constraint. Where did I say I expect the smtp client to have *only one* FQDN? You are arguing with the RFC, not me. :-) In principle, I mostly agree with you. Limiting a SMTP client to a single FQDN would be too strict. Which is why I don't do it. Not even when 'policy strict' is set. Read the POD or the code, and you'll see that my optional, off by default, DNS tests do not limit the HELO hostname to 'the' FQDN, but to any A or record that matches the hostname, and any PTR that matches the IP. The match test is only enabled with 'policy strict.' Although I believe the RFC allows it, the 'policy rfc' setting does not impose matching DNS. It simply requires that the HELO hostname and IP address both resolves to something. Anything. > Is there a definition of canonical FQDN? RFC 1035. Legal characters. Length limits. The whole enchilada. > Is the canonical FQDN anything > other than just the name defined by whoever manages reverse DNS? That's not a FQDN, that's a matching PTR record, aka reverse DNS. However, a FQDN normally does, but is not required to have a matching PTR record. Think of cases like apache virtual hosting, where a thousand FQDNs share a single IP and do not, and should not, have matching PTR records. > In my experience, reverse DNS is frequently managed (or, frequently, not > managed) by an ISP, not by the admin of the mail server. In my experience, every ISP that provides clients with dedicated IPs also provides the client with the ability to manage the reverse DNS for them. Either via a support request or automated means. I did when I owned my ISP in the early 90s, and every ISP and hosting provider I've ever worked for or with since does as well. The good providers even provide rwhois on the delegated IP space and automatically populate it with the clients billing data. (see RFC 1941). Matt
Re: new plugin: helo
On Wed, 13 Jun 2012, Matt Simerson wrote: > https://github.com/smtpd/qpsmtpd/pull/29 I consider this statement to be troublesome also: Per RFC 2821, the HELO hostname must be the FQDN of the sending server or address literal. RFC 2821 doesn't say "must". It says neither should nor must, but simply The argument field contains the fully-qualified domain name of the SMTP client if one is available. Mail servers can certainly be expected to have a FQDN. But expecting them to have *only one* FQDN may be too tight a constraint. Is there a definition of canonical FQDN? Is the canonical FQDN anything other than just the name defined by whoever manages reverse DNS? In my experience, reverse DNS is frequently managed (or, frequently, not managed) by an ISP, not by the admin of the mail server.
Re: new plugin: helo
On Wed, 13 Jun 2012, Matt Simerson wrote: > https://github.com/smtpd/qpsmtpd/pull/29 Make sure the HELO hostname has an A or record that matches the senders IP address, and make sure that the senders IP has a PTR that resolves to the HELO hostname. This might sound pedantic, but since time immemorial, having matching DNS is a minimum standard expected, and frequently required, of mail servers. I think that "time immemorial" is hyperbole, "a minimum standard" is vague (expected by whom?) and "frequently required" is inaccurate - I think this is infrequently required by mail servers - it certainly isn't mandated by any RFC, or enforced by qpsmtpd.org (or you wouldn't be reading this message).
