Re: reject_unknown_sender_domain and DNS SERVFAIL result
On 10/3/2012 1:15 PM, Bill Cole wrote: I recently updated a Postfix system from 2.4 to 2.9 and I have found what I believe is a change in behavior for reject_unknown_sender_domain which is confusing. In the past, an effective means of dealing with some classes of persistent spammers was to tell the local DNS resolver (BIND 9) to blackhole the authoritative nameservers of spammers who cycled rapidly through changes in nearly every other easily detected aspect of their spam. In conjunction with reject_unknown_sender_domain, this rejected a lot of spam cheaply for a while but in recent years I've not paid much attention to it because there are fewer spammers using their own fixed IP space for DNS. Last week I started getting spam again that fit this tactic well, so for the first time in years I added to my DNS blackhole list. And the subsequent spam was not rejected. Upon investigation I have determined that if a domain definitively has no A or MX records (i.e. DNS answers with NXDOMAIN or NOERR with zero answers) then Postfix rejects the mail at RCPT. However, if DNS queries garner SERVFAIL responses, as happens when authorities are blackholed, Postfix is permitting delivery. This is not normal postfix behavior, and suggests either your DNS is giving some sort of unintended answer, or the client is getting an OK/permit somewhere prior to the unknown domain check. This is definitely not what I want. This may be related to the addition in version 2.6 of unknown_address_tempfail_action, but it seems to me based on the postconf manpage that since this defaults via reject_tempfail_action to defer_if_permit (and I have confirmed that this is so on this system) that Postfix should *at best* be sending a 4xx reply to RCPT rather than accepting mail sent from these intentionally unresolvable domains. Postfix has always deferred upon DNS failure, and that behavior has not changed. The *_tempfail_action defaults duplicate previous behavior. At any rate, the tool to reject spammer-haven DNS is a check_sender_ns_access map. http://www.postfix.org/postconf.5.html#check_sender_ns_access -- Noel Jones
Re: reject_unknown_sender_domain and DNS SERVFAIL result
Bill Cole: I recently updated a Postfix system from 2.4 to 2.9 and I have found what I believe is a change in behavior for reject_unknown_sender_domain which is confusing. In the past, an effective means of dealing with some Sort answer: Postfix does not pass SERVFAIL, it just rejects them later. Long answer: Postfix 2.6 and later reject the unknown sender - With $unknown_address_tempfail_action after temporary lookup error (including timeout and SERVFAIL). Earlier Postfix versions hard-coded the result. - With $unknown_address_reject_code after all other lookup errors. Wietse unknown_address_tempfail_action (default: $reject_tempfail_action) The Postfix SMTP server's action when reject_unknown_sender_domain or reject_unknown_recipient_domain fail due to a temporary error condi- tion. Specify defer to defer the remote SMTP client request immedi- ately. With the default defer_if_permit action, the Postfix SMTP server continues to look for opportunities to reject mail, and defers the client request only if it would otherwise be accepted. This feature is available in Postfix 2.6 and later.
Re: reject_unknown_sender_domain and DNS SERVFAIL result
On 3 Oct 2012, at 14:48, Wietse Venema wrote: Bill Cole: I recently updated a Postfix system from 2.4 to 2.9 and I have found what I believe is a change in behavior for reject_unknown_sender_domain which is confusing. In the past, an effective means of dealing with some Sort answer: Postfix does not pass SERVFAIL, it just rejects them later. Long answer: Postfix 2.6 and later reject the unknown sender - With $unknown_address_tempfail_action after temporary lookup error (including timeout and SERVFAIL). Earlier Postfix versions hard-coded the result. - With $unknown_address_reject_code after all other lookup errors. This is what I would expect, based on the documentation. However, it is accepting and delivering mail whose sender domain yields a SERVFAIL and I can't figure out why. Note that as I stated in my first message, Postfix *is* rejecting sender domains with definitive lack of DNS. Here's a simple test message sent by hand from a machine with no special trust: ***BEGIN MESSAGE Return-Path: domain.spam...@dfleur.com X-Original-To: b...@scconsult.com Delivered-To: d...@toaster.scconsult.com Received: from sirius.cipherspace.net (sirius.CipherSpace.net [74.115.116.138]) by toaster.scconsult.com (Postfix) with ESMTP id 3XX4dN3MM4zkV2V for b...@scconsult.com; Wed, 3 Oct 2012 13:55:20 -0400 (EDT) Subject: How is it that this works? To: b...@scconsult.com From: domain.spam...@dfleur.com X-Spam-Score: 1.461 (*) BAYES_50,MISSING_DATE,NO_DNS_FOR_FROM,SCC_DEBUG,SCC_RCVD_FORMAT_NOT_SENDMAIL X-Spam-Status: No, score=1.461 required=4.3 tests=[BAYES_50,MISSING_DATE,NO_DNS_FOR_FROM,SCC_DEBUG,SCC_RCVD_FORMAT_NOT_SENDMAIL] The domain doesn't resolve. Yet Postfix takes it. Hmmm... END MESSAGE The log shows nothing interesting: lazarus:~# grep 3XX4dN3MM4zkV2V /var/log/mail.log Oct 3 13:55:20 lazarus postfix/smtpd[632]: 3XX4dN3MM4zkV2V: client=sirius.CipherSpace.net[74.115.116.138] Oct 3 13:56:45 lazarus postfix/cleanup[638]: 3XX4dN3MM4zkV2V: message-id= Oct 3 13:56:46 lazarus mimedefang.pl[64141]: 3XX4dN3MM4zkV2V: MDLOG,3XX4dN3MM4zkV2V,spam,1.461,74.115.116.138,domain.spam...@dfleur.com,b...@scconsult.com,How is it that this works? Oct 3 13:56:46 lazarus mimedefang.pl[64141]: 3XX4dN3MM4zkV2V: 3XX4dN3MM4zkV2V: SA: 1.461 (*) BAYES_50,MISSING_DATE,NO_DNS_FOR_FROM,SCC_DEBUG,SCC_RCVD_FORMAT_NOT_SENDMAIL Oct 3 13:56:46 lazarus postfix/qmgr[72555]: 3XX4dN3MM4zkV2V: from=domain.spam...@dfleur.com, size=410, nrcpt=1 (queue active) Oct 3 13:56:47 lazarus postfix/local[653]: 3XX4dN3MM4zkV2V: to=d...@toaster.scconsult.com, orig_to=b...@scconsult.com, relay=local, delay=87, delays=87/0.02/0/0.13, dsn=2.0.0, status=sent (delivered to command: /usr/bin/procmail -Y ) Oct 3 13:56:47 lazarus postfix/qmgr[72555]: 3XX4dN3MM4zkV2V: removed lazarus:~# DNS is definitely failing for dfleur.com, as the hit on the SA rule NO_DNS_FOR_FROM indicates and as confirmed by a manual query: lazarus:~# dig dfleur.com mx ; DiG 9.9.1-P3 dfleur.com mx ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: SERVFAIL, id: 41183 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;dfleur.com.IN MX ;; Query time: 4 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Wed Oct 3 15:07:35 2012 ;; MSG SIZE rcvd: 39 lazarus:~ root# dig dfleur.com A ; DiG 9.9.1-P3 dfleur.com A ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: SERVFAIL, id: 23577 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ;; QUESTION SECTION: ;dfleur.com.IN A ;; Query time: 3 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Wed Oct 3 15:07:44 2012 ;; MSG SIZE rcvd: 39 lazarus:~# And finally, confirming the relevant config variables: lazarus:~# postconf unknown_address_tempfail_action reject_tempfail_action unknown_address_reject_code unknown_address_tempfail_action = $reject_tempfail_action reject_tempfail_action = defer_if_permit unknown_address_reject_code = 553 lazarus:~# The only quirk I can think of in the config (included in my first message) is that the naming of the machine and the domains it accepts and locally delivers mail for is a bit of a tangle as an artifact of historical architectural evolution. I don't see anything in that which would redeem a message that otherwise deserves deferral or rejection.
Re: reject_unknown_sender_domain and DNS SERVFAIL result
DNS is definitely failing for dfleur.com, as the hit on the SA rule NO_DNS_FOR_FROM indicates and as confirmed by a manual query: ~$ dig dfleur.com mx ; DiG 9.8.1-P1 dfleur.com mx ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: NOERROR, id: 47102 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 2 ;; QUESTION SECTION: ;dfleur.com.INMX ;; ANSWER SECTION: dfleur.com.3566INMX10 mail.dfleur.com. dfleur.com.3566INMX20 ny.dfleur.com. ;; ADDITIONAL SECTION: mail.dfleur.com.3566INA184.82.205.246 ny.dfleur.com.3566INA209.144.26.231 ;; Query time: 4 msec ;; SERVER: 127.0.1.1#53(127.0.1.1) ;; WHEN: Wed Oct 3 22:21:22 2012 ;; MSG SIZE rcvd: 100 -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: reject_unknown_sender_domain and DNS SERVFAIL result
Bill Cole: ; DiG 9.9.1-P3 dfleur.com mx ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: SERVFAIL, id: 41183 How will I reproduce this quickly? Wietse
Re: reject_unknown_sender_domain and DNS SERVFAIL result
On Wed, Oct 03, 2012 at 04:00:05PM -0400, Bill Cole wrote: reject_unknown_sender_domain This is what I would expect, based on the documentation. However, it is accepting and delivering mail whose sender domain yields a SERVFAIL and I can't figure out why. Note that as I stated in my first message, Postfix *is* rejecting sender domains with definitive lack of DNS. Either Postfix is querying a different DNS service, or the reject_unknown_sender_domain is short-circuited by earlier permit statements. -- Viktor.
Re: reject_unknown_sender_domain and DNS SERVFAIL result
On 3 Oct 2012, at 16:21, Ralf Hildebrandt wrote: DNS is definitely failing for dfleur.com, as the hit on the SA rule NO_DNS_FOR_FROM indicates and as confirmed by a manual query: ~$ dig dfleur.com mx ; DiG 9.8.1-P1 dfleur.com mx ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: NOERROR, id: 47102 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 2 ;; QUESTION SECTION: ;dfleur.com.INMX ;; ANSWER SECTION: dfleur.com.3566INMX10 mail.dfleur.com. dfleur.com.3566INMX20 ny.dfleur.com. ;; ADDITIONAL SECTION: mail.dfleur.com.3566INA184.82.205.246 ny.dfleur.com.3566INA209.144.26.231 ;; Query time: 4 msec ;; SERVER: 127.0.1.1#53(127.0.1.1) ;; WHEN: Wed Oct 3 22:21:22 2012 ;; MSG SIZE rcvd: 100 Please read the first message in the thread to understand how testing against your local DNS resolver is not relevant.
Re: reject_unknown_sender_domain and DNS SERVFAIL result
On Wed, 2012-10-03 at 16:00 -0400, Bill Cole wrote: lazarus:~# dig dfleur.com mx ; DiG 9.9.1-P3 dfleur.com mx ;; Got answer: ;; -HEADER- opcode: QUERY, status: SERVFAIL, id: 41183 ... ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Wed Oct 3 15:07:35 2012 Your locally installed DNS server does not work as you expect. -stefan-
Re: reject_unknown_sender_domain and DNS SERVFAIL result
On Wed, Oct 03, 2012 at 04:26:33PM -0400, Wietse Venema wrote: Bill Cole: ; DiG 9.9.1-P3 dfleur.com mx ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: SERVFAIL, id: 41183 How will I reproduce this quickly? Comcast owns dnssec-failed.org, a zone set up with deliberately broken DNSSEC. If your nameserver is verifying signatures, you will get a SERVFAIL for any names in that zone. It's also easy to set up a zone locally to have a SERVFAIL result. Probably an invalid zone file is all it takes. Insert a record with an out-of-zone owner name. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if /dev/rob0 is in the Subject:
Re: reject_unknown_sender_domain and DNS SERVFAIL result
On 3 Oct 2012, at 16:26, Wietse Venema wrote:
Bill Cole:
; DiG 9.9.1-P3 dfleur.com mx
;; global options: +cmd
;; Got answer:
;; -HEADER- opcode: QUERY, status: SERVFAIL, id: 41183
How will I reproduce this quickly?
I am not sure. If your resolver is BIND you can make dfleur.com (and as
far as I can tell, nothing else but other spammer domains) yield
SERVFAIL by adding this to the options section of named.conf:
blackhole {
108.161.130.187;
};
Then (after running rndc reconfig) you can test by trying to send mail
claiming to be from any dfleur.com address, as I did.
I don't have a handy generic way to make a domain fail in this way. It
does require a resolver failure.
Re: reject_unknown_sender_domain and DNS SERVFAIL result
On 3 Oct 2012, at 16:38, Stefan Palme wrote: On Wed, 2012-10-03 at 16:00 -0400, Bill Cole wrote: lazarus:~# dig dfleur.com mx ; DiG 9.9.1-P3 dfleur.com mx ;; Got answer: ;; -HEADER- opcode: QUERY, status: SERVFAIL, id: 41183 ... ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Wed Oct 3 15:07:35 2012 Your locally installed DNS server does not work as you expect. No, it is working precisely as expected given its intentional configuration. As I said in the first message in this thread, it is intentionally refusing to speak to a specified list of name servers, using the BIND blackhole option. Any query which requires answers from those servers returns a SERVFAIL result.
Re: reject_unknown_sender_domain and DNS SERVFAIL result
On Wed, Oct 03, 2012 at 04:35:59PM -0500, I wrote: On Wed, Oct 03, 2012 at 04:26:33PM -0400, Wietse Venema wrote: Bill Cole: ; DiG 9.9.1-P3 dfleur.com mx ;; global options: +cmd ;; Got answer: ;; -HEADER- opcode: QUERY, status: SERVFAIL, id: 41183 How will I reproduce this quickly? Comcast owns dnssec-failed.org, a zone set up with deliberately broken DNSSEC. If your nameserver is verifying signatures, you will get a SERVFAIL for any names in that zone. 450-4.1.8 r...@dnssec-failed.org: Sender address rejected: Domain not found smtpd_recipient_restrictions = permit_mynetworks, reject_unauth_destination, reject_unknown_sender_domain ... It's also easy to set up a zone locally to have a SERVFAIL result. Probably an invalid zone file is all it takes. Insert a record with an out-of-zone owner name. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if /dev/rob0 is in the Subject:
Re: reject_unknown_sender_domain and DNS SERVFAIL result
On 3 Oct 2012, at 14:46, Noel Jones wrote: On 10/3/2012 1:15 PM, Bill Cole wrote: I recently updated a Postfix system from 2.4 to 2.9 and I have found what I believe is a change in behavior for reject_unknown_sender_domain which is confusing. In the past, an effective means of dealing with some classes of persistent spammers was to tell the local DNS resolver (BIND 9) to blackhole the authoritative nameservers of spammers who cycled rapidly through changes in nearly every other easily detected aspect of their spam. In conjunction with reject_unknown_sender_domain, this rejected a lot of spam cheaply for a while but in recent years I've not paid much attention to it because there are fewer spammers using their own fixed IP space for DNS. Last week I started getting spam again that fit this tactic well, so for the first time in years I added to my DNS blackhole list. And the subsequent spam was not rejected. Upon investigation I have determined that if a domain definitively has no A or MX records (i.e. DNS answers with NXDOMAIN or NOERR with zero answers) then Postfix rejects the mail at RCPT. However, if DNS queries garner SERVFAIL responses, as happens when authorities are blackholed, Postfix is permitting delivery. This is not normal postfix behavior, and suggests either your DNS is giving some sort of unintended answer, or the client is getting an OK/permit somewhere prior to the unknown domain check. It certainly seems abnormal, especially since it is dependent on whether DNS answers with NXDOMAIN or SERVFAIL. Since rejection works as intended when a sender in a definitively bogus domain is given, it is clear that reject_unknown_sender_domain is being applied rather than bypassed. The key seems to be in the unknown_address_tempfail_action handling cited by Dr. Venema, which came with Postfix 2.6. The docs say that Postfix by default postpones deferral on temporary DNS failures while looking for reasons to reject outright. This is definitely not what I want. This may be related to the addition in version 2.6 of unknown_address_tempfail_action, but it seems to me based on the postconf manpage that since this defaults via reject_tempfail_action to defer_if_permit (and I have confirmed that this is so on this system) that Postfix should *at best* be sending a 4xx reply to RCPT rather than accepting mail sent from these intentionally unresolvable domains. Postfix has always deferred upon DNS failure, and that behavior has not changed. The *_tempfail_action defaults duplicate previous behavior. I would like to believe that to be accurate, but the evidence I have implies otherwise. I am sure the ancestor of the current rig was not accepting the targeted spam as intended at some point using Postfix 2.4.5 and that the config changes since I last paid attention to this particular tactic have been minimal and almost entirely due to the 2.4.5-2.9.3 update. The reject_tempfail_action documentation says that it came with 2.6, so if prior behavior was to defer immediately rather than the defer_if_permit strategy, it would partially explain what I am seeing. However, nothing I can find documented should be redeeming the transaction from eventual deferral, and that is what is puzzling me. At any rate, the tool to reject spammer-haven DNS is a check_sender_ns_access map. http://www.postfix.org/postconf.5.html#check_sender_ns_access THANK YOU! That should do precisely what I need and eliminates the need to poke around in named.conf and ponder how Postfix is interacting with BIND. The only remaining mystery is how I managed to not notice its existence before, but that's not a pressing issue.
Re: reject_unknown_sender_domain and DNS SERVFAIL result
Wietse Venema:
Bill Cole:
; DiG 9.9.1-P3 dfleur.com mx
;; global options: +cmd
;; Got answer:
;; -HEADER- opcode: QUERY, status: SERVFAIL, id: 41183
Net::DNS::Nameserver to the rescue, with a trivial reply handler of:
sub reply_handler
{
my ($qname, $qclass, $qtype, $peerhost,$query,$conn) = @_;
print [$peerhost: $qname $qtype?]\n if $opt_v;
return (SERVFAIL, );
}
Verified that dig mx example.com, dig a example.com, dig
example.com fail with SERVFAIL. Output omitted for brevity.
Postfix configuration:
# postconf -f smtpd_recipient_restrictions unknown_address_tempfail_action
reject_tempfail_action
smtpd_recipient_restrictions = reject_unknown_sender_domain, permit,
reject_unauth_destination
unknown_address_tempfail_action = $reject_tempfail_action
reject_tempfail_action = defer_if_permit
Telnet session:
# telnet 127.0.0.1 smtp
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 hostname.watson.ibm.com ESMTP Postfix
mail from:u...@example.com
250 2.1.0 Ok
rcpt to:wietse@localhost
450 4.1.8 u...@example.com: Sender address rejected: Domain not found
quit
221 2.0.0 Bye
Connection closed by foreign host.
As expected, Postfix logging confirms that MX, A and lookups
for example.com fail with a try again error. Logging omitted
for brevity.
reject_unknown_sender_domain then returns defer_if_permit. This
result overrides the effect of the subsequent permit action,
causing the RCPT TO to be deferred with 450 ... Sender address
rejected
There is nothing magical about permit here; any other action that
accepts the RCPT TO will have the same effect.
Wietse
Re: reject_unknown_sender_domain and DNS SERVFAIL result
Bill Cole:
On 3 Oct 2012, at 16:26, Wietse Venema wrote:
Bill Cole:
; DiG 9.9.1-P3 dfleur.com mx
;; global options: +cmd
;; Got answer:
;; -HEADER- opcode: QUERY, status: SERVFAIL, id: 41183
How will I reproduce this quickly?
I am not sure. If your resolver is BIND you can make dfleur.com (and as
far as I can tell, nothing else but other spammer domains) yield
SERVFAIL by adding this to the options section of named.conf:
blackhole {
108.161.130.187;
};
This produces the same result as in my Net::DNS example with a
forced SERVFAIL response.
# telnet hostname smtp
Trying 9.2.193.248...
Connected to hostname.watson.ibm.com.
Escape character is '^]'.
220 hostname.watson.ibm.com ESMTP Postfix
mail from:u...@dfleur.com
250 2.1.0 Ok
rcpt to:wietse@localhost
450 4.1.8 u...@dfleur.com: Sender address rejected: Domain not found
quit
221 2.0.0 Bye
Connection closed by foreign host.
Parameters:
reject_tempfail_action = defer_if_permit
unknown_address_tempfail_action = $reject_tempfail_action
smtpd_recipient_restrictions = reject_unknown_sender_domain, permit,
reject_unauth_destination
The only way to make Postfix accept the recipient is that you have
something before reject_unknown_sender_domain that accepts the
recipient.
Wietse
SOLVED! was Re: reject_unknown_sender_domain and DNS SERVFAIL result
Predictably, the cause of this odd behavior was in fact external to Postfix. The server has 3 DNS servers in resolv.conf: itself, another one sitting across the room, and a third far away which was added in the same disaster recovery event that precipitated the upgrade from 2.4.5 to 2.9.3 a few months ago. The first 2 have my blackhole nameserver list, the third does not and will not. I had not considered the fact that libresolv (or perhaps Postfix itself?) sees a SERVFAIL reply as sufficiently dubious that it does not accept a single nameserver's response as definitive and instead tries them all. This is a rational strategy, but not something I had considered until an extended discussion of this as a tempfail condition. Removing the external nameserver solved the problem. My thanks go to all who spoke up and esp. Dr. Venema for writing an MTA that always ends up being the wrong place to be looking for a source of trouble.
