Re: DNS Whitelisting
I'm working on Spamhaus' new whitelist where our goal is to list only mail sources clean enough that you can skip the rest of the filtering. (So far so good, but it's still pretty small.) You're welcome to use it. The IP address version is at swl.spamhaus.org. For people who like DKIM, there's also domain version at dwl.spamhaus.org. It lists domains, with the ONLY use that we support being DKIM d= signing domains on mail with valid signatures. See RFC 5518. The terms of use are the same as the rest of the Spamhaus lists, moderate number of queries are fine, larger than that and you have to buy a feed. If you already have a Spamhaus feed, the SWL and DWL should now be included in it. The plan for the SWL and DWL is that we will eventually charge for listings, but for now it's free, in limited beta. See http://www.spamhauswhitelist.com/en/, and drop me a line if you'd like an invitation. Because I like Spamhaus and dnswl.org, I had written a policy service for postfix several weeks ago. It is stable, as far as I could test it. Maybe you also like to have a look at this project. I won't talk about it here anymore, if somebody feels bothered. Promised! http://www.roessner-network-solutions.com/?page_id=639 I really, really would appreciate a feedback. ;-) Thx Christian PGP.sig Description: Signierter Teil der Nachricht
Re: DNS Whitelisting support, uploaded
Wietse Venema: This is now implemented with minor changes. [...] I have uploaded postfix-2.8-20101105-nonprod for testing (nonprod because this is SMTP server code, and I mostly rely on postscreen's DNS whitelisting feature). Same code, now available as postfix-2.8-20101108 regular snapshot release. I updated some comments and documentation. Wietse
Re: DNS Whitelisting
Noel Jones put forth on 11/5/2010 11:04 AM: On 11/5/2010 10:03 AM, Wietse Venema wrote: This is now implemented with minor changes. Excellent! Looking forward to a test drive. Excellent indeed. Thank you for implementing this Wietse. Jerrale, it appears Wietse just solved your problem WRT dnswl.org access. :) -- Stan
Re: DNS Whitelisting
On Fri, Nov 05, 2010 at 11:03:34AM -0400, Wietse Venema wrote: The current manpage text reads: reject_rbl_client rbl_domain=d.d.d.d ... permit_dnswl_client dnswl_domain=d.d.d.d Accept the request when the reversed client network address is listed with the A record d.d.d.d under dnswl_domain. If no =d.d.d.d is specified, accept the request when the reversed client network address is listed with any A record under dnswl_domain. For safety, permit_dnswl_client is silently ignored when it would override reject_unauth_destination.The result is DEFER_IF_REJECT when whitelist lookup fails. This feature is available in Postfix 2.8 and later. ... reject_rhsbl_client rbl_domain=d.d.d.d ... permit_rhswl_client rhswl_domain=d.d.d.d Accept the request when the client hostname is listed with the A record d.d.d.d under rhswl_domain. If no =d.d.d.d is speci- fied, accept the request when the client hostname is listed with any A record under rhswl_domain. For safety, permit_rhswl_client is silently ignored when it would override reject_unauth_destination.The result is DEFER_IF_REJECT when whitelist lookup fails. This feature is available in Postfix 2.8 and later. Looks good. Should we mention that these should only be used to reduce FPs from blacklists that follow, and that are expected to not list legitimate clients. Thus any temporary DNS lookup error would likely result an an additional lookup that incurrs a low risk or rejection, but does *imply* rejection. DNS-based positive access rules are fragile with respect to the semantics of temporary lookup failures, and must not be used to permit some while denying all. There will at some point be interest in DNSWL support for verified DKIM d= domains. For now that's out of scope (milters, pre-queue filters, ...) I've recently starting using the OpenDKIM library, ... it is fairly easy to support. If there is ever interest in directly supporting DKIM in the Postfix SMTP server, I'm game to talk design. Due to the DNS lookup latency inherent in incoming DKIM checks, doing DKIM in post-queue content-filters is somewhat unattractive, as typically one wants low-latency, modest concurrency in a post-queue filter. -- Viktor.
Re: DNS Whitelisting
On 11/5/2010 10:03 AM, Wietse Venema wrote: This is now implemented with minor changes. Excellent! Looking forward to a test drive. -- Noel Jones
Re: DNS Whitelisting
Victor Duchovni: On Fri, Nov 05, 2010 at 11:03:34AM -0400, Wietse Venema wrote: The current manpage text reads: reject_rbl_client rbl_domain=d.d.d.d ... permit_dnswl_client dnswl_domain=d.d.d.d Accept the request when the reversed client network address is listed with the A record d.d.d.d under dnswl_domain. If no =d.d.d.d is specified, accept the request when the reversed client network address is listed with any A record under dnswl_domain. For safety, permit_dnswl_client is silently ignored when it would override reject_unauth_destination.The result is DEFER_IF_REJECT when whitelist lookup fails. This feature is available in Postfix 2.8 and later. ... reject_rhsbl_client rbl_domain=d.d.d.d ... permit_rhswl_client rhswl_domain=d.d.d.d Accept the request when the client hostname is listed with the A record d.d.d.d under rhswl_domain. If no =d.d.d.d is speci- fied, accept the request when the client hostname is listed with any A record under rhswl_domain. For safety, permit_rhswl_client is silently ignored when it would override reject_unauth_destination.The result is DEFER_IF_REJECT when whitelist lookup fails. This feature is available in Postfix 2.8 and later. Looks good. Should we mention that these should only be used to reduce FPs from blacklists that follow, and that are expected to not list legitimate clients. Thus any temporary DNS lookup error would likely result an an additional lookup that incurrs a low risk or rejection, but does *imply* rejection. DNS-based positive access rules are fragile with respect to the semantics of temporary lookup failures, and must not be used to permit some while denying all. I agree that whitelisting on client names is fragile (whether one uses a static whitelist map or DNS lookups), because the client name lookup will sometimes fail due to some temporary error. The DEFER_IF_REJECT result cited above handles only whitelist lookup problems; it is easy enough to hard-code the same in permit_*wl_client for temporary client name lookup problems, but I see no easy way to automatically fall back to DEFER_IF_REJECT for all name-based mechanisms. There will at some point be interest in DNSWL support for verified DKIM d= domains. For now that's out of scope (milters, pre-queue filters, ...) I've recently starting using the OpenDKIM library, ... it is fairly easy to support. If there is ever interest in directly supporting DKIM in the Postfix SMTP server, I'm game to talk design. I'm not yet in a hurry to build those 3-4 letter alphabet soup protocols into Postfix. Due to the DNS lookup latency inherent in incoming DKIM checks, doing DKIM in post-queue content-filters is somewhat unattractive, as typically one wants low-latency, modest concurrency in a post-queue filter. This is where before-queue filtersand Milters are in a better position. I expect that there will be pressure to move away from after-queue filters, despite the limitations of the before-queue approach. Wietse
Re: DNS Whitelisting
Should we mention that these should only be used to reduce FPs from blacklists that follow, and that are expected to not list legitimate clients. ... Depends on the whitelist. I'm working on Spamhaus' new whitelist where our goal is to list only mail sources clean enough that you can skip the rest of the filtering. (So far so good, but it's still pretty small.) You're welcome to use it. The IP address version is at swl.spamhaus.org. For people who like DKIM, there's also domain version at dwl.spamhaus.org. It lists domains, with the ONLY use that we support being DKIM d= signing domains on mail with valid signatures. See RFC 5518. The terms of use are the same as the rest of the Spamhaus lists, moderate number of queries are fine, larger than that and you have to buy a feed. If you already have a Spamhaus feed, the SWL and DWL should now be included in it. The plan for the SWL and DWL is that we will eventually charge for listings, but for now it's free, in limited beta. See http://www.spamhauswhitelist.com/en/, and drop me a line if you'd like an invitation. R's, John
Re: DNS Whitelisting
On Fri, Nov 05, 2010 at 12:27:06PM -0400, Wietse Venema wrote: Should we mention that these should only be used to reduce FPs from blacklists that follow, and that are expected to not list legitimate clients. Thus any temporary DNS lookup error would likely result an an additional lookup that incurrs a low risk or rejection, but does *imply* ^^^ Missed a: not rejection. DNS-based positive access rules are fragile with respect to the semantics of temporary lookup failures, and must not be used to permit some while denying all. I agree that whitelisting on client names is fragile (whether one uses a static whitelist map or DNS lookups), because the client name lookup will sometimes fail due to some temporary error. The DEFER_IF_REJECT result cited above handles only whitelist lookup problems; it is easy enough to hard-code the same in permit_*wl_client for temporary client name lookup problems, but I see no easy way to automatically fall back to DEFER_IF_REJECT for all name-based mechanisms. This strategy upgrades all temporary name lookup problems to DEFER_IF_REJECT when a permit_rhswl_client is encountered before a reject action. This has the unfortunate effect of making all invalid traffic from systems with broken DNS retry. I think this is not viable, that the better option is to have a fragile whitelist, which sometimes fails to whitelist, but that's OK, because one expects the blacklists to not list the good guys. We are changing the FP rate, not promising zero FPs. In any case, this needs a bit of care in documentation and perhaps design. -- Viktor.
Re: DNS Whitelisting
On Fri, Nov 05, 2010 at 04:51:14PM -, John Levine wrote: Should we mention that these should only be used to reduce FPs from blacklists that follow, and that are expected to not list legitimate clients. ... Depends on the whitelist. I'm working on Spamhaus' new whitelist where our goal is to list only mail sources clean enough that you can skip the rest of the filtering. (So far so good, but it's still pretty small.) Yes, and same said sources should not be blacklisted by anything that follows. If Postfix can't determine the client's reverse domain (tempfail) and therefore cannot even ask SpamHaus whether the (verified) client (PTR) domain is on the whitelist, one can either tempfail all mail from said client (bad, lots of retransmitted garbage) or fail to white-list (not so bad, worst case the blacklists will FP a small fraction of legit clients, choose your blacklists with care). You're welcome to use it. The IP address version is at swl.spamhaus.org. The IP version does not encounter the issue, as we don't expect systemic tempfail issues with swl lookups, but we expect systemic problems obtaining verified (IP - PTR - matching-IP) names for many clients. For people who like DKIM, there's also domain version at dwl.spamhaus.org. It lists domains, with the ONLY use that we support being DKIM d= signing domains on mail with valid signatures. See RFC 5518. Hence my comment about possible appetite for DKIM in smtpd/cleanup. The terms of use are the same as the rest of the Spamhaus lists, moderate number of queries are fine, larger than that and you have to buy a feed. If you already have a Spamhaus feed, the SWL and DWL should now be included in it. The plan for the SWL and DWL is that we will eventually charge for listings, but for now it's free, in limited beta. See http://www.spamhauswhitelist.com/en/, and drop me a line if you'd like an invitation. I have an invitation, my problem is that I don't yet have separate infrastructure for just non-marketing email. In a large enough organization, someone, somewhere will unilaterally engage in some marketing under the radar, so we need to think about separating the known good, rather than trying to preclude the unknown bad. -- Viktor.
Re: DNS Whitelisting support, uploaded
On 11/5/2010 6:24 PM, Wietse Venema wrote: This is now implemented with minor changes. [...] I have uploaded postfix-2.8-20101105-nonprod for testing (nonprod because this is SMTP server code, and I mostly rely on postscreen's DNS whitelisting feature). ftp://ftp.porcupine.org/mirrors/postfix-release/index.html and mirror sites. Once the code stops changing I can make back-port patches available for older Postfix versions, but I won't include whitelist support with stable releases. Wietse Seems to work as advertised. This is what I'm using this at the end of smtpd_recipient_restrictions to exempt clients from greylisting. (The warn_if_reject is instrumentation to log when there's a hit during testing.) smtpd_recipient_restrictions = ... usual stuff ... warn_if_reject reject_rbl_client hostkarma.junkemailfilter.com=127.0.0.1 warn_if_reject reject_rbl_client list.dnswl.org warn_if_reject reject_rbl_client swl.spamhaus.org permit_dnswl_client swl.spamhaus.org permit_dnswl_client hostkarma.junkemailfilter.com=127.0.0.1 permit_dnswl_client list.dnswl.org check_policy_service unix:private/greylist -- Noel Jones
Re: DNS Whitelisting
* Wietse Venema wie...@porcupine.org: Noel Jones: As I see it, there are two complementary paths we can take with DNS whitelists, each with a slightly different purpose. While these are both useful, neither depends on the other, so postfix can implement either or both. I'll read the entire proposal later. Would this notation work: dnswl1.example.com=127.0.0.2*weight1, dnswl2.example.com=127.0.0.1*weight2 dnsbl3.example.com=127.0.0.3*weight3, dnsbl4.example.com=127.0.0.1*weight4 weightn can be negative? Do we want to allow mixing DNSWLs and DNSBLs in one list? Probably, with positiv and negative weights? -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: DNS Whitelisting
On Wed, Aug 25, 2010 at 11:27 PM, Wietse Venema wie...@porcupine.org wrote: dnswl1.example.com=127.0.0.2*weight1, dnswl2.example.com=127.0.0.1*weight2 dnsbl3.example.com=127.0.0.3*weight3, dnsbl4.example.com=127.0.0.1*weight4 What about wildcarding? dnswl.org currently returns 127.0.n.[0-3], with n being numerical for the category (eg banks etc). People may want to have something like whitelist on 127.0.*.2 and 127.0.*.3. // Yes, the use and ordering of the octets was possibly not the best design choice back when. -- Matthias
Re: DNS Whitelisting
Matthias Leisi: On Wed, Aug 25, 2010 at 11:27 PM, Wietse Venema wie...@porcupine.org wrote: ?dnswl1.example.com=127.0.0.2*weight1, dnswl2.example.com=127.0.0.1*weight2 ?dnsbl3.example.com=127.0.0.3*weight3, dnsbl4.example.com=127.0.0.1*weight4 What about wildcarding? dnswl.org currently returns 127.0.n.[0-3], with n being numerical for the category (eg banks etc). People may want to have something like whitelist on 127.0.*.2 and 127.0.*.3. The * operator is already used for the weight factor. I'm not sure that it is a good idea to use the same * operator for wildcarding. Wietse
Re: DNS Whitelisting
Wietse Venema put forth on 8/25/2010 4:27 PM: Noel Jones: As I see it, there are two complementary paths we can take with DNS whitelists, each with a slightly different purpose. While these are both useful, neither depends on the other, so postfix can implement either or both. I'll read the entire proposal later. Would this notation work: dnswl1.example.com=127.0.0.2*weight1, dnswl2.example.com=127.0.0.1*weight2 dnsbl3.example.com=127.0.0.3*weight3, dnsbl4.example.com=127.0.0.1*weight4 IMO this really depends on the role you expect postscreen to play. If it is expected that all Postfix implementations are going to be using postscreen as in integral important feature, not just the big operations with hundreds of thousands or millions of connections per day, then I would suggest this user configuration interface is too complicated. There will be OPs who simply want to use a single dns whitelist without any kind of scoring. They will want an OK on any 127.0.x.x response. It would reduce the number of config parameters. Always a plus, but in this case it may sacrifice (ease of) usability for some OPs. Do we want to allow mixing DNSWLs and DNSBLs in one list? If you're implementing a pure scoring system this might work. I don't know what the math should be though for calculating an OK threshold. Currently, postscreen does not look up the client hostname, that is something that can be added later when there is time. Won't all of these dns lookups slow postscreen down? I thought it was supposed to be a really high throughput low latency check/allow/dump filter mainly to shield smtpd processes from the bot load. What's the average dnsxl response time? 50ms? 100ms? Won't this terribly decrease the throughput of postscreen? If postscreen is doing dnsxl lookups merely to decide if a connection gets past postscreen, and then we do the same dnsbl lookup in smtpd to decide accept/reject because postscreen doesn't pass the result to smtpd, it makes me wonder why we're doing two such lookups for each connect that is OK'd by postscreen. Again, I'm not so sure that doing dnswl lookups with postscreen is productive. Most hosts listed on dnswls are going to pass postscreen's normal checks anyway. By doing this it seems we're merely adding processing latency to a latency sensitive code path, and we're not really gaining anything by doing so. These dnswl lookups will have to be duplicated in smtpd anyway to get the desired whitelisting result OPs expect from a dnswl positive return. -- Stan
Re: DNS Whitelisting
On 8/26/2010 4:14 PM, Wietse Venema wrote: The more precise solution is to implement wildcards with ranges: example.com=127.0.[0-128].3*1 example.com=127.0.[0-5,6-9].3*1 Noel Jones: I like the range idea. You want proto docs reflecting that syntax? Yes, that would help everyone to understand how it will work. Wietse
Re: DNS Whitelisting
Updated Proposal for weighted dnsXl support in postscreen.
(Change parameter names to all start with postscreen_dns* for
easy reading in postconf. Get rid of negative site weight
values [the client dnsxl score total may still be negative].
Add filter octet range docs.)
(The weight ranges documented are arbitrary.)
- postscreen_dnsbl_sites (default empty); A comma separated
list of dnsbl IP blacklist sites with optional result filter
and optional weight. When the reversed client network address
is listed with an A record matching the result filter, add
{weight} points (default 1) to the client dnsxl score. If no
result filter is specified, add {weight} points (default 1) to
the client dnsxl score if any A record is found. If multiple
A records are found, {weight} will only be added once.
Specify one or more dnsbl sites as:
dnsbl_site[=d.d.d.d][*N]
where dnsbl_site is the site name, d.d.d.d is the optional
result filter, and N is the optional weight value. A range in
the result filter may be specified within brackets in place of
an octet. Multiple ranges or single values may be separated
by commas within the brackets. The weight may be specified as
the character * followed by a value in the range [0~99]
inclusive.
Examples:
postscreen_dnsbl_sites =
dnsbl_site1,
dnsbl_site2=127.0.[0-5,22,128-255].2*5,
dnsbl_site3=*6
- postscreen_dnswl_sites (default empty); A comma separated
list of dnswl IP whitelist sites with optional result filter
and optional weight. When the reversed client network address
is listed with an A record matching the result filter,
subtract {weight} points (default 1) from the client dnsxl
score. If no result filter is specified, subtract {weight}
points (default 1) from the client dnsxl score if any A record
is listed. If multiple A records are found, {weight} will
only be subtracted once.
Specify one or more dnswl sites as:
dnswl_site[=d.d.d.d][*N]
where dnswl_site is the site name, d.d.d.d is the optional
result filter, and N is the optional weight value. A range in
the result filter may be specified within brackets in place of
an octet. Multiple ranges or single values may be separated
by commas within the brackets. The weight may be specified as
the character * followed by a value in the range [0~99]
inclusive.
Examples:
postscreen_dnswl_sites =
dnswl_site1,
dnswl_site2=127.0.[0-5,22,128-255].2*5,
dnswl_site3=*6
(these next parameters behavior is unchanged, but the docs
have been updated)
(Require a + or - sign for the score thresholds to prevent
ambiguity. The alternatives are to assume - for the
whitelist and + for the blacklist, or always assume +. I
think it's least confusing to just require the sign.)
(The score threshold range is arbitrary.)
- postscreen_dnsxl_whitelist_score (default -1); a pass
threshold for the total of the client's dnsxl points. Specify
a value in the range [-999~+999] inclusive. The sign must be
specified. Clients scoring at or BELOW this value trigger the
postscreen_dnsxl_whitelist_action. Clients scoring greater
than postscreen_dnsxl_whitelist_score, but less than
postscreen_dnsxl_blacklist_score continue with postscreen
analysis for disposition.
Example:
postscreen_dnsxl_whitelist_score = -5
- postscreen_dnsxl_blacklist_score (default=1) a drop
threshold for the total of the client's dnsxl points. Specify
a value in the range [-999~+999] inclusive. The sign must be
specified. Clients scoring at or ABOVE this value trigger the
postscreen_dnsxl_blacklist_action. Clients scoring greater
than postscreen_dnsxl_whitelist_score, but less than
postscreen_dnsxl_blacklist_score continue with postscreen
analysis for disposition.
Example:
postscreen_dnsxl_blacklist_score = +5
- postscreen_dnsxl_whitelist_action (default continue); the
action postscreen takes when a client matches the
postscreen_dnsxl_whitelist_score.
Specify one of:
continue; perform additional postscreen tests to determine
disposition.
pass; exempt the client from further postscreen tests and pass
it to a real SMTP server process
- postscreen_dnsxl_blacklist_action (default continue); the
action postscreen takes when a client exceeds the
postscreen_dnsxl_blacklist_score.
Specify one of:
continue; perform additional postscreen tests to determine
disposition.
drop; drop the connection with a 521 SMTP reply
(next two items are for future expansion if hostnames are
available)
- postscreen_dnsbl_hostname_sites (default empty); A comma
separated list of rhsbl hostname blacklist sites using the
unverified client hostname with optional result filter and
optional weight. When the unverified reverse client hostname
is listed with an A record matching the result filter, add
{weight} points (default 1) to the client dnsxl score. If no
result filter is specified, add {weight} points (default 1) to
the client dnsxl score if any A record is listed. If multiple
A records are
Re: DNS Whitelisting
Noel Jones put forth on 8/24/2010 2:18 PM: - This is specific for dnswl.org. Postfix needs a general mechanism. Other whitelists are not required to follow dnswl.org's 127.0.x.y mechanism. Yeah, I used this example as dnswl is, afaik, the most established of the dns whitelists. I haven't yet looked at the return codes the others use. - what do you mean by accept the message; OK? suppress further rbl lookups? This is something that needs discussion due to various expectations of what a dns whitelist entry actually means and how it can best be implemented. I think messages with a positive dnswl return code should obviously bypass certain restrictions, mainly dnsbl hits, but not all restrictions--we shouldn't bypass virus checks and content filters. And if an admin wants to locally ban an IP listed in a dnswl for any reason, we shouldn't be bypassing user defined access tables. I think bypassing reject_r*bl_* and most of the inbuilt/standard client, sender, helo, etc sanity checks would be a good idea. Obviously we don't want to bypass something like reject_unlisted_recipient. I think what we really need here is consensus on the use of local whitelists. Should dns whitelists carry the same weight? If so, we may want to bypass all restrictions but virus/content filters and recipient address verification. - If accept means OK, how will you protect postfix from open relay if the dns whitelist accidentally or intentionally lists the whole internet? We currently run similar risk, but in the opposite direction, using dnsbls. I don't believe there is a technical solution to this potential issue. Every dnsbl comes with a use at your own risk... no warranty... disclaimer for this reason and others. Though I think the dnsbl case is worse as it can potentially stop all email flow. In the dnswl case, you'd simply see much more spam. Although, at the inbox level, the denial of service result may be equivalent, much like yesteryear when folks had to sift through 100 spams to find 3 legit emails they received on a given day. But at least they still received those 3. Probably difficult to implement, but maybe some kind of analysis on mail flow would help. Keep statistics counters and establish an average volume baseline. If volume increases 10x over baseline, throttle all connections. The programming effort may not be worth the payoff here given the odds of list world occurrences. - what would the user interface look like? Is it possible to document it clearly? I think something like I mentioned before would be good. Simple single line configuration parameter populating one global program variable. How the variable would be used is the tricky part, and up to others. I'm not much of a programmer these days. :( I think it would be wise to allow flexible use of accept_dnswl_client much like reject_r*bl_foo which can today be used within access tables in addition to directly within smtpd_foo_restrictions. If we make its functionality very permissive by default, I probably wouldn't make it globally effective by default. WRT documentation, that will depend on the values we allow for the variable. If we do something like the 0-3 thing similar to dnswl.org, it's not as clear due to complexity. If we make it boolean, it's very clear. In this latter case, lots of bold starred italic documentation needs to inform the OP of the risks of turning it on. -- Stan
Re: DNS Whitelisting
Wietse Venema put forth on 8/24/2010 2:37 PM: With reject_rbl_client etc. Postfix can use different DNSXLs names in different access lists, and filter the result. For example, to select responses from some.example.com with value 127.0.0.4: smtpd_mumble_restrictions = ... reject_rbl_client some.example.com=127.0.0.4 ... I suppose that similar selection would be help with whitelists. Yes, I believe the dnswl.org implementation is based on RFC5782. Regarding safety against the list world and other scenarios, see section 5, and paragraph 2 of section 7. The tests are the same for dnsbls and dnswls. You simply check for responses to make sure a dnsXl is running on the host you're querying before sending real queries. I'm guessing you already have code for this in the reject_rbl_client code. I assume postscreen processes or passes this data to smtpd in a way that smtpd will automatically bypass all checks normally performed during the CONNECT phase. Postscreen passes no session information to the SMTP server. The file handle is all the SMTP server gets. We're talking about a really tight development budget here. Darn. With all candor and humility Wietse, I don't think postscreen is the right place to implement dnswl whitelisting. Or, I should say, it's not a complete dns whitelisting solution, but only a small first step. If I understand correctly, all this will do is shoot such a whitelisted client past all the postscreen checks. Virtually all MTAs at IPs listed by dnswl servers are going to pass the postscreen checks anyway since postscreen is looking for bots and other ill behaved SMTP clients. Thus there is little gain in dns whitelisting here here but maybe some extra performance on loaded MX'en. IMHO, for dnswl whitelisting to be implemented properly, or the way most people will expect it to behave, it should be implemented almost exactly like reject_rbl_client abc.dnsbl.tld n.n.n.n but with the opposite action--immediately queuing the message for mailbox delivery--but still executing recipient address verification, virus checks, and possibly content filter checks etc. For a full up dnswl implementation, if we can assume some but not all dnswl servers have multiple responses, then I'm thinking we should do something just like your reject_rbl_client example, so an OP can configure the level of trust/accept behavior based on his/her knowledge of a particular configured dnswl server's responses: accept_dnswl_client abc.dnsbl.tld [n.n.n.n] As with reject_rbl_client the n.n.n.n would be optional and the default behavior would be to accept or OK a message with any positive response code from the dnswl server, in the same manner as we would process a local access table whitelist OK action. We obviously still want recipient verification, virus scanning, maybe content filtering. So if I understand Postfix design correctly, we'd implement this in smtpd and skip all smtpd checks on a positive dnswl hit, assuming the OP inserts this parameter appropriately close to the top of the restrictions list. As with reject_rbl_client an OP would be free to insert this dnswl check in the most appropriate place in the OP's processing order. -- Stan
Re: DNS Whitelisting
On 24 Aug 2010, at 21:37, Wietse Venema wrote: Stan Hoeppner: Wietse Venema put forth on 8/23/2010 10:11 AM: Noel Jones: (Might be time to revisit DNS whitelists in postfix.) Maybe someone can draft a strawman user interface: - what is the configuration syntax - what does that syntax mean - how to make it safe ( we don't want open relay problems) I'm currently doing this for postscreen, and won't have time for other Postfix features. accept_dnswl_client (default: 0) 0 - accept all messages 1 - accept messages with trust level 1-3 2 - accept messages with trust level 2-3 3 - accept messages with trust level 3 This looks somewhat like RFC 5782, with reputation scores and confidence values encoded in the lower octets as numbers in the range 0-255. With reject_rbl_client etc. Postfix can use different DNSXLs names in different access lists, and filter the result. For example, to select responses from some.example.com with value 127.0.0.4: smtpd_mumble_restrictions = ... reject_rbl_client some.example.com=127.0.0.4 ... I suppose that similar selection would be help with whitelists. Just to add to the mix if Postfix is working on whitelist implementation... Spamhaus has assigned 127.0.2.0/24 for whitelist return codes. The new Spamhaus Whitelist (SWL) due out very shortly will return 127.0.2.2 and 127.0.2.3 and Spamhaus' new Domain Whitelist (DWL) will return 127.0.2.12 and 127.0.2.13. We will explain what these codes mean closer to the release date (sometime in September). In the case of the Spamhaus Whitelist (SWL), the implementation we want people to use is that a message from an IP on the SWL should be allowed immediately past all spam filtering and all content filtering and passed straight to the virus filter if any. Steve Linford The Spamhaus Project http://www.spamhaus.org
Re: DNS Whitelisting
Steve Linford put forth on 8/25/2010 8:27 AM: Just to add to the mix if Postfix is working on whitelist implementation... Spamhaus has assigned 127.0.2.0/24 for whitelist return codes. The new Spamhaus Whitelist (SWL) due out very shortly will return 127.0.2.2 and 127.0.2.3 and Spamhaus' new Domain Whitelist (DWL) will return 127.0.2.12 and 127.0.2.13. We will explain what these codes mean closer to the release date (sometime in September). In the case of the Spamhaus Whitelist (SWL), the implementation we want people to use is that a message from an IP on the SWL should be allowed immediately past all spam filtering and all content filtering and passed straight to the virus filter if any. Thanks for the heads up Steve. :) Great timing. -- Stan
Re: DNS Whitelisting
As I see it, there are two complementary paths we can take
with DNS whitelists, each with a slightly different purpose.
While these are both useful, neither depends on the other, so
postfix can implement either or both.
My proposals:
A) scoring in postscreen
A dns whitelist/blacklist scoring system in postscreen,
whereby a dnswl hit can overrule dnsbl hits and other
postscreen tests.
Since this is in postscreen, the only data available would be
the client IP and hopefully the hostname. This would be
automatically safe from open-relay abuse since the client gets
no more rights than other screened clients. The transaction
could still be rejected by other tests done in the real SMTP
server. The user interface would consist of
- postscreen_dnsl_client_sites (default empty); a list of
dnsbl and dnswl domains for client IP lookup.
Specify a list of dns blacklist and whitelist sites with an
optional A result: dnsl_site=d.d.d.d, dnsl_site=d.d.d.d, ...
Count one hit when the A record =d.d.d.d is listed. If no
=d.d.d.d is specified, count one hit if one or more A records
is listed.
- postscreen_dnsl_hostname_sites (default empty); a list of
dnsbl and dnswl sites for hostname lookup using the verified
client hostname. This is recommended for domain-based whitelists.
Specify a list of dns blacklist and whitelist sites with an
optional A result, dnsl_site=d.d.d.d, dnsl_site=d.d.d.d, ...
Count one hit when the A record =d.d.d.d is listed. If no
=d.d.d.d is specified, count one hit if one or more A records
is listed.
- postscreen_dnsl_reverse_hostname_sites (default empty); a
list of dnsbl and dnswl sites for hostname lookup using the
unverified client hostname. This is recommended for
domain-based blacklists.
Specify a list of dns blacklist and whitelist sites with an
optional A result, dnsl_site=d.d.d.d, dnsl_site=d.d.d.d, ...
Count one hit when the A record =d.d.d.d is listed. If no
=d.d.d.d is specified, count one hit if one or more A records
is listed.
- postscreen_dnsl_site_score_default (default 1); default
score for any dns list result. I suppose this could be a
non-user-visible default, since it doesn't seem to make much
sense to adjust this; any adjustments would be made in the
score map.
- postscreen_dnsl_site_score_maps (no default); an optional
table that overrides postscreen_dnsl_site_score_default,
indexed by the list name. The key is the exact dnsl site
name as specified in the dnsl sites maps, including any
optional =d.d.d.d result filter. The result is a positive
number for blacklists, negative number for whitelist.
This table is required when using dns whitelists.
- postscreen_dnsl_whitelist_score (default=-1); a pass
threshold score. clients scoring at or BELOW this value
trigger the postscreen_dnsl_whitelist_action.
- postscreen_dnsl_blacklist_score (default=1) a drop
threshold score. Clients scoring at or ABOVE this value
trigger the postscreen_dnsl_blacklist_action.
- postscreen_dnsl_whitelist_action (default continue); the
action postscreen takes when a client matches the
postscreen_dnsl_whitelist_score. Specify one of:
continue; perform additional postscreen tests to determine
disposition.
pass; exempt the client from further postscreen tests and pass
it to a real SMTP server process
- postscreen_dnsl_blacklist_action (default continue); the
action postscreen takes when a client exceeds the
postscreen_dnsl_blacklist_score. Specify one of:
continue; perform additional postscreen tests to determine
disposition.
drop; drop the connection with a 521 SMTP reply
There are a lot of parameters, but sane defaults would allow a
simple drop everything using this one blacklist config,
while still allowing much more complex multi-list scoring. A
minimal config would be simply:
postscreen_dnsl_client_sites = zen.spamhaus.org
postscreen_dnsl_blacklist_action = drop
The downside of this method is it will require a lot of new code.
B) a permit based system, a mirror of reject_rbl_client.
This would have a user interface similar to the existing
reject_rbl_client with expected usage similar to access(5)
based whitelists.
Seems to me that checks using sender-supplied info such as
{helo, sender domain, recipient domain} are unsafe -- give
whitelist control to unverified information -- and probably
shouldn't be implemented.
To prevent open-relay accidents, this would need to return
permit_auth_destination rather than a blanket permit. Maybe
the result action will need to be configurable? Nah.
The user interface would be familiar to anyone using rbl
checks. Sample documentation under the appropriate
smtpd_mumble_restrictions section:
- permit_dnswl_client dnswl_domain=d.d.d.d
Accept the request when the reversed client IP network
address is listed with an A record of d.d.d.d under
dnswl_domain. If no =d.d.d.d is given, accept the request
with any A record under dnswl_domain. For safety, only
Re: DNS Whitelisting
Noel Jones: As I see it, there are two complementary paths we can take with DNS whitelists, each with a slightly different purpose. While these are both useful, neither depends on the other, so postfix can implement either or both. I'll read the entire proposal later. Would this notation work: dnswl1.example.com=127.0.0.2*weight1, dnswl2.example.com=127.0.0.1*weight2 dnsbl3.example.com=127.0.0.3*weight3, dnsbl4.example.com=127.0.0.1*weight4 It would reduce the number of config parameters. Do we want to allow mixing DNSWLs and DNSBLs in one list? Currently, postscreen does not look up the client hostname, that is something that can be added later when there is time. Wietse
Re: DNS Whitelisting
On 8/25/2010 6:17 PM, Wietse Venema wrote: Noel Jones: On 8/25/2010 4:27 PM, Wietse Venema wrote: Noel Jones: As I see it, there are two complementary paths we can take with DNS whitelists, each with a slightly different purpose. While these are both useful, neither depends on the other, so postfix can implement either or both. I'll read the entire proposal later. Would this notation work: dnswl1.example.com=127.0.0.2*weight1, dnswl2.example.com=127.0.0.1*weight2 dnsbl3.example.com=127.0.0.3*weight3, dnsbl4.example.com=127.0.0.1*weight4 It would reduce the number of config parameters. Yes. Both the A filter and the weight would need to be optional. ie, if you want any result to have a custom weight, it must accept dnsbl1.example.com=*weight I assume the syntax would be like this: domain[=addr][*weight] where content inside [] is optional. Excellent. Currently, postscreen does not look up the client hostname, that is something that can be added later when there is time. I know. I was just hoping the hostname would be available in the new-and-improved postscreen discussed elsewhere. The primary goal is to keep spambots away from Postfix, so I'll focus on features that do for the short term. The only luxury will be the dummy SMTP engine that logs the rejected client, helo, sender, recipient. Right now I don't miss that at all. I've been reluctant to turn on postscreen dns blacklists on the main MX because it's so much harder for me to find the once-in-a-blue-moon FPs. But it runs gangbusters on the secondary. I won't miss the dnsbl hostname lookups in postscreen (it also eliminates a few more parameters), but it would still be nice to have simple hostname rhswl support in smtpd as described later in my outline. I see these as complementary features. -- Noel Jones
Re: DNS Whitelisting
Stan Hoeppner: Wietse Venema put forth on 8/23/2010 10:11 AM: Noel Jones: (Might be time to revisit DNS whitelists in postfix.) Maybe someone can draft a strawman user interface: - what is the configuration syntax - what does that syntax mean - how to make it safe ( we don't want open relay problems) I'm currently doing this for postscreen, and won't have time for other Postfix features. accept_dnswl_client (default: 0) 0 - accept all messages 1 - accept messages with trust level 1-3 2 - accept messages with trust level 2-3 3 - accept messages with trust level 3 This looks somewhat like RFC 5782, with reputation scores and confidence values encoded in the lower octets as numbers in the range 0-255. With reject_rbl_client etc. Postfix can use different DNSXLs names in different access lists, and filter the result. For example, to select responses from some.example.com with value 127.0.0.4: smtpd_mumble_restrictions = ... reject_rbl_client some.example.com=127.0.0.4 ... I suppose that similar selection would be help with whitelists. I assume postscreen processes or passes this data to smtpd in a way that smtpd will automatically bypass all checks normally performed during the CONNECT phase. Postscreen passes no session information to the SMTP server. The file handle is all the SMTP server gets. We're talking about a really tight development budget here. Wietse
