Re: Email Server and DNS
I suggest moving this to mailop, where it arguably belongs. But I'm going to follow up on a few points, anyway. First, I forgot to mention two other highly effective mail system defense methods: geoblocking and passive OS fingerprinting. Geoblocking: A mail server for a local construction business in Arizona is unlikely to require mail from Poland, Peru or Pakistan. So there's no reason to go with a default-permit model: use default-deny and only allow mail from places where legitimate mail might originate. (In this case, perhaps: the US, Mexico, and Canada.) Use the ranges from ipdeny.com. This will stop an astonishing amount of spam (and other SMTP-borne abuse) cold. And it can be done at the MTA or in the firewall: which is better depends on circumstances. Obviously this doesn't work for everyone. Obviously this (like everything else) runs the risk of false positives -- but it's easy to mitigate that. Obviously it does require understanding the patterns in your mail traffic, but any competent mail system admin has long since performed detailed statistical analysis and has a pretty good idea what the characteristics of their incoming mail stream look like. Passive OS fingerprinting: regard anything originating from an OS that fingerprints as Windows as dubious, at best. Possible actions vary: graylisting (more precisely, graylisting regardless of previous traffic) is one good option. Utilizing this in concert with geoblocking (above) works beautifully, e.g., I'm in Arizona and something in Portugal that fingerprints as Windows is trying to send me SMTP traffic: the probability approaches unity that this is spam. When combined with rDNS information, this becomes a highly efficient mechanism with a FP rate that's ridiculously low. (In other words, if that same system has rDNS that looks like 123-45-67-8.example.com then it either really is a bot or it's a mail system run by someone with no clue.) A few short points and one long one in response: On Sun, Nov 03, 2013 at 12:00:23PM -0600, Jimmy Hess wrote: The RFC contains a MUST NOT in regards to verifying the HELO name matches. So, the HELO can use any hostname --- as long as the hostname forward resolves to something; it should resolve to the IP address of one of your mail servers.Some mail servers provide service for many domains, and have many DNS names that could be placed in a HELO. All true. But none of this argues against using the canonical hostname in the HELO. It's the simplest, easiest option (and quite often the one that software will pick by default). SPF is worthless crap: don't bother. Use a real MTA, e.g., postfix I do not believe that is the consensus of the community -- or the working groups behind the SPF-related RFCs. I'm well aware it's not the consensus. It's my opinion. Clue #1 that SPF is crap should have been its grandiose self-promoting announcement (Spam as a technical problem is solved by SPF). Clue #2 should have been the observation that -- by far -- the most prolific early adopters were spammers. When your enemy latches on to your new weapon much faster than you do, that should be a tipoff that maybe it's not what you hope it is. Clue #3 is available to anyone who deploys a sufficiently large and diverse set of spamtraps for several years and analyzes the data that arrives: SPF presence/absence or contents have no statistically useful anti-spam value in a properly-designed mail defense architecture. Don't believe me? Okay. Fine. Set up a few thousand spamtraps, gather data for 3-5 years, see for yourself. So yes, it's standardized; so what? So is (sort of) DNS forgery, see http://tools.ietf.org/html/draft-livingood-dns-redirect-03 for example. That's also crap, it just happens to be well-documented crap. So: if you feel you must use something, use DKIM, which I think shows vastly more promise. Just don't expect it to be a panacea, because the current miserable state of security at *all* levels undercuts it badly. Not DKIM's fault, really, but it does impact its usefulness in the real world. Message quarantines are great; they are helpful for mitigating the false positives of overly-agressive filters. This one I'll spend more time on. Quarantines are a worst practice in mail systems engineering. Here are some assorted reasons why, briefly: - One of the fundamental principles of mail system defense is that you should make your mistakes early, consistently, and loudly. (And you WILL make mistakes. Everyone does.) The point of doing this is that it enables all concerned, including you, to have a decent chance of noticing them, figuring out that they're mistakes, and correcting them. Quarantines hide, defer, and silence your mistakes, making it much more difficult to run your system properly. They're a lazy admin's coverup, not a fix. - Quarantines present deadlock problems. Mail from user A to user B which is quarantined and causes B to eventually send a
Re: Email Server and DNS
Thanks to everyone for all the tips and info. I think I have compiled plenty of info to get this done. I will probably start with some of the basics and see how things go. THen as needed start putting in some additional features as I see how things progress. Robert On Fri, 8 Nov 2013 07:37:40 -0500 Rich Kulawiec r...@gsp.org wrote: I suggest moving this to mailop, where it arguably belongs. But I'm going to follow up on a few points, anyway. First, I forgot to mention two other highly effective mail system defense methods: geoblocking and passive OS fingerprinting. Geoblocking: A mail server for a local construction business in Arizona is unlikely to require mail from Poland, Peru or Pakistan. So there's no reason to go with a default-permit model: use default-deny and only allow mail from places where legitimate mail might originate. (In this case, perhaps: the US, Mexico, and Canada.) Use the ranges from ipdeny.com. This will stop an astonishing amount of spam (and other SMTP-borne abuse) cold. And it can be done at the MTA or in the firewall: which is better depends on circumstances. Obviously this doesn't work for everyone. Obviously this (like everything else) runs the risk of false positives -- but it's easy to mitigate that. Obviously it does require understanding the patterns in your mail traffic, but any competent mail system admin has long since performed detailed statistical analysis and has a pretty good idea what the characteristics of their incoming mail stream look like. Passive OS fingerprinting: regard anything originating from an OS that fingerprints as Windows as dubious, at best. Possible actions vary: graylisting (more precisely, graylisting regardless of previous traffic) is one good option. Utilizing this in concert with geoblocking (above) works beautifully, e.g., I'm in Arizona and something in Portugal that fingerprints as Windows is trying to send me SMTP traffic: the probability approaches unity that this is spam. When combined with rDNS information, this becomes a highly efficient mechanism with a FP rate that's ridiculously low. (In other words, if that same system has rDNS that looks like 123-45-67-8.example.com then it either really is a bot or it's a mail system run by someone with no clue.) A few short points and one long one in response: On Sun, Nov 03, 2013 at 12:00:23PM -0600, Jimmy Hess wrote: The RFC contains a MUST NOT in regards to verifying the HELO name matches. So, the HELO can use any hostname --- as long as the hostname forward resolves to something; it should resolve to the IP address of one of your mail servers.Some mail servers provide service for many domains, and have many DNS names that could be placed in a HELO. All true. But none of this argues against using the canonical hostname in the HELO. It's the simplest, easiest option (and quite often the one that software will pick by default). SPF is worthless crap: don't bother. Use a real MTA, e.g., postfix I do not believe that is the consensus of the community -- or the working groups behind the SPF-related RFCs. I'm well aware it's not the consensus. It's my opinion. Clue #1 that SPF is crap should have been its grandiose self-promoting announcement (Spam as a technical problem is solved by SPF). Clue #2 should have been the observation that -- by far -- the most prolific early adopters were spammers. When your enemy latches on to your new weapon much faster than you do, that should be a tipoff that maybe it's not what you hope it is. Clue #3 is available to anyone who deploys a sufficiently large and diverse set of spamtraps for several years and analyzes the data that arrives: SPF presence/absence or contents have no statistically useful anti-spam value in a properly-designed mail defense architecture. Don't believe me? Okay. Fine. Set up a few thousand spamtraps, gather data for 3-5 years, see for yourself. So yes, it's standardized; so what? So is (sort of) DNS forgery, see http://tools.ietf.org/html/draft-livingood-dns-redirect-03 for example. That's also crap, it just happens to be well-documented crap. So: if you feel you must use something, use DKIM, which I think shows vastly more promise. Just don't expect it to be a panacea, because the current miserable state of security at *all* levels undercuts it badly. Not DKIM's fault, really, but it does impact its usefulness in the real world. Message quarantines are great; they are helpful for mitigating the false positives of overly-agressive filters. This one I'll spend more time on. Quarantines are a worst practice in mail systems engineering. Here are some assorted reasons why, briefly: - One of the fundamental principles of mail system defense is that you should make your mistakes early, consistently, and loudly. (And you WILL make mistakes. Everyone does.) The point of doing this is that it enables all concerned, including you,
Re: Email Server and DNS
On Sun, Nov 3, 2013 at 11:39 AM, rw...@ropeguru.com wrote: I am looking for some info on current practice for an email server and SMTP delivery. It has been a while since I have had to setup an email server and I have been tasked with setting up a small one for a friend. My question centers around the server sending outgoing email and the current practices requirements for other servers to accept email Things like rDNS, SPF records, etc... Hi Robert, Current best practices are: don't run your own email server unless you're willing to spend the ongoing time and effort it takes to keep up with the current solutions to the spam, hacking and abuse problems. Corollary: when you get bored of doing so for a tiny mail server, stop running it and buy a service. Other than that, the _changes_ of note in the last decade are: 1. The blacklist aggregators and IP reputation services have changed so you have to find the new ones, 2. There are email whitelist services now, some free others for a nominal cost. Use them. 3. Phishing and spear phishing are relatively sophisticated now, so your spam solution has to deal reasonably with it. 4. Relay from and to an external address without changing the envelope sender no longer functions reliably due to things like SPF enforcement and no mail servers I've noticed have such a translator built in. Regards, Bill Herrin -- William D. Herrin her...@dirtside.com b...@herrin.us 3005 Crane Dr. .. Web: http://bill.herrin.us/ Falls Church, VA 22042-3004
Re: Email Server and DNS
On Fri, Nov 08, 2013 at 08:37:32AM -0500, William Herrin wrote: On Sun, Nov 3, 2013 at 11:39 AM, rw...@ropeguru.com wrote: I am looking for some info on current practice for an email server and SMTP delivery. It has been a while since I have had to setup an email server and I have been tasked with setting up a small one for a friend. My question centers around the server sending outgoing email and the current practices requirements for other servers to accept email Things like rDNS, SPF records, etc... Hi Robert, Current best practices are: don't run your own email server unless you're willing to spend the ongoing time and effort it takes to keep up with the current solutions to the spam, hacking and abuse problems. Corollary: when you get bored of doing so for a tiny mail server, stop running it and buy a service. and yet, at the IETF this week, in the technical plenary, a call to diffuse the target space by running your own services. much harder to have your mail scrapped from your servers than from your providers. /bill Other than that, the _changes_ of note in the last decade are: 1. The blacklist aggregators and IP reputation services have changed so you have to find the new ones, 2. There are email whitelist services now, some free others for a nominal cost. Use them. 3. Phishing and spear phishing are relatively sophisticated now, so your spam solution has to deal reasonably with it. 4. Relay from and to an external address without changing the envelope sender no longer functions reliably due to things like SPF enforcement and no mail servers I've noticed have such a translator built in. Regards, Bill Herrin -- William D. Herrin her...@dirtside.com b...@herrin.us 3005 Crane Dr. .. Web: http://bill.herrin.us/ Falls Church, VA 22042-3004
Re: Email Server and DNS
On 11/3/2013 8:11 PM, John Levine wrote: I would recommend you go a step further and use DKIM, ADSP, and DMARC. Using DKIM is a good idea. Do *not* use ADSP. It is a failed experiment which will provide no benefit and considerable pain. +1 If you believe that your domain is heavily forged (which if you are not Paypal, Facebook, or a large bank or ISP, it almost certainly is not), you can set up a DMARC record to collect some statistics about what mail other people are getting that appears to be from you. Do not try to use DMARC to tell people to quarantine or reject your mail until you are really sure you understand the statistics you're getting. +1 The 'reporting' function in DMARC appears to have wide applicability and substantial benefit. The handling (rejection, etc.) function has very narrow benefit. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net
Re: Email Server and DNS
www.maawg.org has published a sender BCP, please read it signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Email Server and DNS
On Nov 4, 2013, at 8:41 AM, Franck Martin fmar...@linkedin.com wrote: www.maawg.org has published a sender BCP, please read it You mean http://www.maawg.org/sites/maawg/files/news/MAAWG_Senders_BCP_Ver2a-updated.pdf? Regards, -drc signature.asc Description: Message signed with OpenPGP using GPGMail
Email Server and DNS
So I figured a little break from the NSA was in order. I am looking for some info on current practice for an email server and SMTP delivery. It has been a while since I have had to setup an email server and I have been tasked with setting up a small one for a friend. My question centers around the server sending outgoing email and the current practices requirements for other servers to accept email Things like rDNS, SPF records, etc... I am pretty much set on the issue of incoming spam and virus. Probably overkill but it is checked at the Sophos UTM firewall and at the email server itself. Thanks, Robert
Re: Email Server and DNS
On Sun, Nov 03, 2013 at 12:39:25PM -0400, rw...@ropeguru.com wrote: I am looking for some info on current practice for an email server and SMTP delivery. It has been a while since I have had to setup an email server and I have been tasked with setting up a small one for a friend. My question centers around the server sending outgoing email and the current practices requirements for other servers to accept email Things like rDNS, SPF records, etc... If you want to minimize your hassles: make sure you have matching non-generic DNS/rDNS. (non-generic meaning something that looks like a host that should sending and receiving email. In other words, mailgw.example.net looks real. ip-137-12-16-164.example.com looks like a random host that's probably part of a botnet.) Make sure that you HELO/EHLO as the same host -- unless there's some good reason not to. There probably isn't. SPF is worthless crap: don't bother. Use a real MTA, e.g., postfix or sendmail or exim or courier. Consider adjusting the settings to make them as conservative as you can while still leaving you with a functional setup. (e.g., if your MTA supports connection rate throttling, use it.) Read your logs. Use the Spamhaus DROP and EDROP lists, and use them bidirectionally. If your MTA supports greetpause or similar mechanisms, use it. Graylisting is still reasonably effective as well. Don't use a quarantine, it's a horrible idea. (Ask RSA how that worked out for them.) Make sure you don't backscatter. Make sure you don't use SMTP callouts, which are just as abusive as spam. Make sure you have working postmaster and abuse addresses. Make sure your MTA doesn't emit or respond to return-receipts. Read your logs (again). ---rsk
Re: Email Server and DNS
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/3/2013 8:39 AM, rw...@ropeguru.com wrote: So I figured a little break from the NSA was in order. I am looking for some info on current practice for an email server and SMTP delivery. It has been a while since I have had to setup an email server and I have been tasked with setting up a small one for a friend. My question centers around the server sending outgoing email and the current practices requirements for other servers to accept email Things like rDNS, SPF records, etc... I am pretty much set on the issue of incoming spam and virus. Probably overkill but it is checked at the Sophos UTM firewall and at the email server itself. Thanks, Robert MX, PTR, and SPF are really all you need. I would recommend you go a step further and use DKIM, ADSP, and DMARC. It will help keep asshat spammers from flaming your domain all over the internet. I use http://www.unlocktheinbox.com/ to verify my configuration. - -- - -Bret Taylor -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.20 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJSdn6aAAoJEFL3ObmpFQy/GG8H/1WnDVLF/53rE+JjxscUOTBj JLppOrSGGHnOB3HtljJt6g7T0ehA2ZNGjVUG7q22G8fJ76br6Ih3eRGLaYDycgkb FPB/lhs2C9yWBlwSjZ6zE8ufATPj1gIU9QIx2Tq+9ndcXMUtVjiLHfpUd1PNVORE jL7PSD2alSSoa29e3BXx1/reCtRPTH3FgAu7WDTwV0LL15hTx5n7gpBae7WtUcWq Yt9nwTGp2XAbZ7pJKDAuoqOQKACwBo2WdVDJwDj7Tn8W4XzY+pTWoQzquqTrR8At jhyGI9L1JIanHnYuGzZUX12JCmkOmu9f2QuqZygJ7ieZ8KvnYQPeFsXM/vVcsVQ= =ADhR -END PGP SIGNATURE-
Re: Email Server and DNS
On Sun, Nov 3, 2013 at 11:08 AM, Rich Kulawiec r...@gsp.org wrote: non-generic DNS/rDNS. (non-generic meaning something that looks like a host that should sending and receiving email. In other words, mailgw.example.net looks real. ip-137-12-16-164.example.com looks like a random host that's probably part of a botnet.) This is a good idea, but you can use a generic hostname if you want. It's not a consensus that you need a non-generic hostname. There can be negative consequences for mail delivered to some recipients, because SpamAssassin and some other reasonably popular spamfilters, will use a regular expression based on a sending mail server's rDNS name to decide that ip-137-12-16-14.example.com is not a bonafide mail server, based on the hostname alone; this can increase the probability that legitimate mail you send will be miscategorized as spam, if you do choose to use a generic hostname for a mail server. http://svn.apache.org/repos/asf/spamassassin/branches/jm_bug_3852_two_level_configs/rules/20_dynrdns.cf There are many ad-hoc rules that spam filters will subject messages and sending mail servers to, that are not part of the formal requirements for sending MTAs. Make sure that you HELO/EHLO as the same host -- unless there's some good reason not to. There probably isn't. The RFC contains a MUST NOT in regards to verifying the HELO name matches. So, the HELO can use any hostname --- as long as the hostname forward resolves to something; it should resolve to the IP address of one of your mail servers.Some mail servers provide service for many domains, and have many DNS names that could be placed in a HELO. The general rule here is: Look at what is common, what is the simplest and most likely way for mail server operators to present themselves. You don't want your mail server to look any different to the outside world, either in operation, naming, or forward and reverse DNS configuration, HELO messages, etc, from a majority of other legitimate mail servers. SPF is worthless crap: don't bother. Use a real MTA, e.g., postfix I do not believe that is the consensus of the community -- or the working groups behind the SPF-related RFCs. There are many mail servers and spam filters that will enforce SPF policies; there are also a number that will synthesize an implicit policy of Soft (or hard) Fail for any sending IP, except the MX or base A name IP, if you do not publish explicit SPF records. I have a spam filter that penalizes messages from domains with no SPF published, or not matching a high-granularity allow condition by scoring them as more likely to be spam; greylisting may be caused by a soft fail, and of course, messages will be rejected on a hard fail. If you are serious about running an outgoing mail server; I do not believe you have the luxury of completely ignoring SPF, DKIM / domainkeys / sender-id, and other established or emerging community standards. After all, they can be used as a tool to help reject some spam. e.g. spoofed @usps.com messages Don't use a quarantine, it's a horrible idea. (Ask RSA how that worked Message quarantines are great; they are helpful for mitigating the false positives of overly-agressive filters. A combination of quarantine and reject policies can be useful, for border cases. Especially, if there are some users that want spammy messages, which are actually HTML bulk-sent newsletters of some sort or another. out for them.) Make sure you don't backscatter. Make sure you don't use SMTP callouts, which are just as abusive as spam. Make sure you Implemented appropriately; an SMTP callout (or SMTP connect verification) to the connecting IP is a great way to help resolve the suspiciousness level of the sending mail server. SMTP connect verification is a good thing to be using, when the sending server is using an envelope From domain, that has not yet bothered to publish the proper SPF records to verify them. There are bad broken SMTP callout implementations that should be avoided, and there are rather useful ones. have working postmaster and abuse addresses. Make sure your MTA doesn't emit or respond to return-receipts. Read your logs (again). Of course, read receipt requests should be suppressed when dealing with internet hosts. Having a working postmaster address is a requirement. Using the name abuse@ for the abuse contact is not required, and it is likely to be targetted by spammers. Working abuse and technical contacts should be published in the IP address and domain name WHOIS databases for the mail server's primary domain, and all envelope from domains that can be originated on that mail server. ---rsk -- -JH
Re: Email Server and DNS
In addition to all the other reco's below, 1) only allow sending by your users from the submit port and only with authentication. There should be no client sending through the SMTP port. 2) Implement SSL on POP IMAP if at all possible Otherwise enforce CRAM-MD5 3) Review logs esp pop and imap login failures. 4) Turn off VRFY. On Nov 3, 2013, at 11:49 AM, Private Sender wrote: Signed PGP part On 11/3/2013 8:39 AM, rw...@ropeguru.com wrote: So I figured a little break from the NSA was in order. I am looking for some info on current practice for an email server and SMTP delivery. It has been a while since I have had to setup an email server and I have been tasked with setting up a small one for a friend. My question centers around the server sending outgoing email and the current practices requirements for other servers to accept email Things like rDNS, SPF records, etc... I am pretty much set on the issue of incoming spam and virus. Probably overkill but it is checked at the Sophos UTM firewall and at the email server itself. Thanks, Robert MX, PTR, and SPF are really all you need. I would recommend you go a step further and use DKIM, ADSP, and DMARC. It will help keep asshat spammers from flaming your domain all over the internet. I use http://www.unlocktheinbox.com/ to verify my configuration. - -- - -Bret Taylor signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Email Server and DNS
On Sun, Nov 3, 2013 at 11:49 AM, Private Sender nob...@snovc.com wrote: I would recommend you go a step further and use DKIM, ADSP, and DMARC. Don't do DMARC if you expect to have end-users forward emails, or subscribe to mailinglists. Despite the removal from the current DMARC spec, the original guidelines called for DMARC to be used for auto-generated transactional email, and expressly not for end-user generated content. -Jim P.
Re: Email Server and DNS
MX, PTR, and SPF are really all you need. So far so good, noting that a host name that doesn't look generic is better than one that does. I would recommend you go a step further and use DKIM, ADSP, and DMARC. Using DKIM is a good idea. Do *not* use ADSP. It is a failed experiment which will provide no benefit and considerable pain. (Check the author list on RFC 5617 before arguing, please.) If you believe that your domain is heavily forged (which if you are not Paypal, Facebook, or a large bank or ISP, it almost certainly is not), you can set up a DMARC record to collect some statistics about what mail other people are getting that appears to be from you. Do not try to use DMARC to tell people to quarantine or reject your mail until you are really sure you understand the statistics you're getting. R's, John
Re: Email Server and DNS
On Sun, Nov 03, 2013 at 08:49:32AM -0800, Private Sender wrote: On 11/3/2013 8:39 AM, rw...@ropeguru.com wrote: I am looking for some info on current practice for an email server and SMTP delivery. It has been a while since I have had to setup an email server and I have been tasked with setting up a small one for a friend. My question centers around the server sending outgoing email and the current practices requirements for other servers to accept email Things like rDNS, SPF records, etc... I am pretty much set on the issue of incoming spam and virus. Probably overkill but it is checked at the Sophos UTM firewall and at the email server itself. Thanks, Robert MX, PTR, and SPF are really all you need. I would recommend you go a step further and use DKIM, ADSP, and DMARC. It will help keep asshat spammers from flaming your domain all over the internet. I use http://www.unlocktheinbox.com/ to verify my configuration. - -- - -Bret Taylor small - so you likely want to avoid the problems of SMTP with thyroid problems. simple is good. practically, you don't need DKIM, ADSP, DMARC or really any quasi reputation systems in play. For that matter you don't need SPF either... PTR's are good to have and an MX can be more useful than not - BUT - none of them are required for a host to received and process SMTP. /bill
Re: Email Server and DNS
* Private Sender nob...@snovc.com: On 11/3/2013 8:39 AM, rw...@ropeguru.com wrote: I am looking for some info on current practice for an email server and SMTP delivery. It has been a while since I have had to setup an email server and I have been tasked with setting up a small one for a friend. My question centers around the server sending outgoing email and the current practices requirements for other servers to accept email Things like rDNS, SPF records, etc... [...] MX, PTR, and SPF are really all you need. I would recommend you go a step further and use DKIM, ADSP, and DMARC. It will help keep asshat spammers from flaming your domain all over the internet. And while you are at it - why not implement DNSSEC for the domain in question and publish some DANE TLSA records? Stefan signature.asc Description: Digital signature
