Re: DNS Whitelisting support, uploaded
Wietse Venema: > > This is now implemented with minor changes. [...] > > I have uploaded postfix-2.8-20101105-nonprod for testing (nonprod > because this is SMTP server code, and I mostly rely on postscreen's > DNS whitelisting feature). Same code, now available as postfix-2.8-20101108 regular snapshot release. I updated some comments and documentation. Wietse
Re: DNS Whitelisting
> > I'm working on Spamhaus' new whitelist where our goal is to list only > mail sources clean enough that you can skip the rest of the filtering. > (So far so good, but it's still pretty small.) > > You're welcome to use it. The IP address version is at swl.spamhaus.org. > > For people who like DKIM, there's also domain version at > dwl.spamhaus.org. It lists domains, with the ONLY use that we support > being DKIM d= signing domains on mail with valid signatures. See RFC > 5518. > > The terms of use are the same as the rest of the Spamhaus lists, moderate > number of queries are fine, larger than that and you have to buy a feed. > If you already have a Spamhaus feed, the SWL and DWL should now be > included in it. > > The plan for the SWL and DWL is that we will eventually charge for > listings, but for now it's free, in limited beta. See > http://www.spamhauswhitelist.com/en/, and drop me a line if you'd like > an invitation. Because I like Spamhaus and dnswl.org, I had written a policy service for postfix several weeks ago. It is stable, as far as I could test it. Maybe you also like to have a look at this project. I won't talk about it here anymore, if somebody feels bothered. Promised! http://www.roessner-network-solutions.com/?page_id=639 I really, really would appreciate a feedback. ;-) Thx Christian PGP.sig Description: Signierter Teil der Nachricht
Re: DNS Whitelisting
Noel Jones put forth on 11/5/2010 11:04 AM: > On 11/5/2010 10:03 AM, Wietse Venema wrote: >> This is now implemented with minor changes. > > Excellent! Looking forward to a test drive. Excellent indeed. Thank you for implementing this Wietse. Jerrale, it appears Wietse just solved your problem WRT dnswl.org access. :) -- Stan
Re: DNS Whitelisting support, uploaded
On 11/5/2010 6:24 PM, Wietse Venema wrote: This is now implemented with minor changes. [...] I have uploaded postfix-2.8-20101105-nonprod for testing (nonprod because this is SMTP server code, and I mostly rely on postscreen's DNS whitelisting feature). ftp://ftp.porcupine.org/mirrors/postfix-release/index.html and mirror sites. Once the code stops changing I can make back-port patches available for older Postfix versions, but I won't include whitelist support with stable releases. Wietse Seems to work as advertised. This is what I'm using this at the end of smtpd_recipient_restrictions to exempt clients from greylisting. (The warn_if_reject is instrumentation to log when there's a hit during testing.) smtpd_recipient_restrictions = ... usual stuff ... warn_if_reject reject_rbl_client hostkarma.junkemailfilter.com=127.0.0.1 warn_if_reject reject_rbl_client list.dnswl.org warn_if_reject reject_rbl_client swl.spamhaus.org permit_dnswl_client swl.spamhaus.org permit_dnswl_client hostkarma.junkemailfilter.com=127.0.0.1 permit_dnswl_client list.dnswl.org check_policy_service unix:private/greylist -- Noel Jones
Re: DNS Whitelisting
On Fri, Nov 05, 2010 at 04:51:14PM -, John Levine wrote: > >Should we mention that these should only be used to reduce FPs from > >blacklists that follow, and that are expected to not list legitimate > >clients. ... > > Depends on the whitelist. > > I'm working on Spamhaus' new whitelist where our goal is to list only > mail sources clean enough that you can skip the rest of the filtering. > (So far so good, but it's still pretty small.) Yes, and same said sources should not be blacklisted by anything that follows. If Postfix can't determine the client's reverse domain (tempfail) and therefore cannot even ask SpamHaus whether the (verified) client (PTR) domain is on the whitelist, one can either tempfail all mail from said client (bad, lots of retransmitted garbage) or fail to white-list (not so bad, worst case the blacklists will FP a small fraction of legit clients, choose your blacklists with care). > You're welcome to use it. The IP address version is at swl.spamhaus.org. The IP version does not encounter the issue, as we don't expect systemic tempfail issues with swl lookups, but we expect systemic problems obtaining verified (IP -> PTR -> matching-IP) names for many clients. > For people who like DKIM, there's also domain version at > dwl.spamhaus.org. It lists domains, with the ONLY use that we support > being DKIM d= signing domains on mail with valid signatures. See RFC > 5518. Hence my comment about possible appetite for DKIM in smtpd/cleanup. > The terms of use are the same as the rest of the Spamhaus lists, moderate > number of queries are fine, larger than that and you have to buy a feed. > If you already have a Spamhaus feed, the SWL and DWL should now be > included in it. > > The plan for the SWL and DWL is that we will eventually charge for > listings, but for now it's free, in limited beta. See > http://www.spamhauswhitelist.com/en/, and drop me a line if you'd like > an invitation. I have an invitation, my problem is that I don't yet have separate infrastructure for just non-marketing email. In a large enough organization, someone, somewhere will unilaterally engage in some marketing under the radar, so we need to think about separating the known good, rather than trying to preclude the unknown bad. -- Viktor.
Re: DNS Whitelisting
On Fri, Nov 05, 2010 at 12:27:06PM -0400, Wietse Venema wrote: > > Should we mention that these should only be used to reduce FPs from > > blacklists that follow, and that are expected to not list legitimate > > clients. Thus any temporary DNS lookup error would likely result an an > > additional lookup that incurrs a low risk or rejection, but does *imply* ^^^ Missed a: not > > rejection. DNS-based positive access rules are fragile with respect > > to the semantics of temporary lookup failures, and must not be used to > > permit some while denying all. > > I agree that whitelisting on client names is fragile (whether one > uses a static whitelist map or DNS lookups), because the client > name lookup will sometimes fail due to some temporary error. The > DEFER_IF_REJECT result cited above handles only whitelist lookup > problems; it is easy enough to hard-code the same in permit_*wl_client > for temporary client name lookup problems, but I see no easy way > to automatically fall back to DEFER_IF_REJECT for all name-based > mechanisms. This strategy upgrades all temporary name lookup problems to DEFER_IF_REJECT when a "permit_rhswl_client" is encountered before a reject action. This has the unfortunate effect of making all invalid traffic from systems with broken DNS retry. I think this is not viable, that the better option is to have a fragile whitelist, which sometimes fails to whitelist, but that's OK, because one expects the blacklists to not list the good guys. We are changing the FP rate, not promising zero FPs. In any case, this needs a bit of care in documentation and perhaps design. -- Viktor.
Re: DNS Whitelisting
>Should we mention that these should only be used to reduce FPs from >blacklists that follow, and that are expected to not list legitimate >clients. ... Depends on the whitelist. I'm working on Spamhaus' new whitelist where our goal is to list only mail sources clean enough that you can skip the rest of the filtering. (So far so good, but it's still pretty small.) You're welcome to use it. The IP address version is at swl.spamhaus.org. For people who like DKIM, there's also domain version at dwl.spamhaus.org. It lists domains, with the ONLY use that we support being DKIM d= signing domains on mail with valid signatures. See RFC 5518. The terms of use are the same as the rest of the Spamhaus lists, moderate number of queries are fine, larger than that and you have to buy a feed. If you already have a Spamhaus feed, the SWL and DWL should now be included in it. The plan for the SWL and DWL is that we will eventually charge for listings, but for now it's free, in limited beta. See http://www.spamhauswhitelist.com/en/, and drop me a line if you'd like an invitation. R's, John
Re: DNS Whitelisting
Victor Duchovni: > On Fri, Nov 05, 2010 at 11:03:34AM -0400, Wietse Venema wrote: > > > The current manpage text reads: > > > >reject_rbl_client rbl_domain=d.d.d.d > > ... > >permit_dnswl_client dnswl_domain=d.d.d.d > > Accept the request when the reversed client network address > > is > > listed with the A record "d.d.d.d" under dnswl_domain. If > > no > > "=d.d.d.d" is specified, accept the request when the > > reversed > > client network address is listed with any A record > > under > > dnswl_domain. > > For safety, permit_dnswl_client is silently ignored when > > it > > would override reject_unauth_destination.The result > > is > > DEFER_IF_REJECT when whitelist lookup fails. This feature > > is > > available in Postfix 2.8 and later. > > ... > >reject_rhsbl_client rbl_domain=d.d.d.d > > ... > >permit_rhswl_client rhswl_domain=d.d.d.d > > Accept the request when the client hostname is listed with > > the A > > record "d.d.d.d" under rhswl_domain. If no "=d.d.d.d" is > > speci- > > fied, accept the request when the client hostname is listed > > with > > any A record under rhswl_domain. > > For safety, permit_rhswl_client is silently ignored when > > it > > would override reject_unauth_destination.The result > > is > > DEFER_IF_REJECT when whitelist lookup fails. This feature > > is > > available in Postfix 2.8 and later. > > Looks good. > > Should we mention that these should only be used to reduce FPs from > blacklists that follow, and that are expected to not list legitimate > clients. Thus any temporary DNS lookup error would likely result an an > additional lookup that incurrs a low risk or rejection, but does *imply* > rejection. DNS-based positive access rules are fragile with respect > to the semantics of temporary lookup failures, and must not be used to > permit some while denying all. I agree that whitelisting on client names is fragile (whether one uses a static whitelist map or DNS lookups), because the client name lookup will sometimes fail due to some temporary error. The DEFER_IF_REJECT result cited above handles only whitelist lookup problems; it is easy enough to hard-code the same in permit_*wl_client for temporary client name lookup problems, but I see no easy way to automatically fall back to DEFER_IF_REJECT for all name-based mechanisms. > There will at some point be interest in DNSWL support for verified DKIM > "d=" domains. For now that's out of scope (milters, pre-queue filters, ...) > I've recently starting using the OpenDKIM library, ... it is fairly easy > to support. If there is ever interest in directly supporting DKIM in the > Postfix SMTP server, I'm game to talk design. I'm not yet in a hurry to build those 3-4 letter alphabet soup protocols into Postfix. > Due to the DNS lookup latency inherent in incoming DKIM checks, doing > DKIM in post-queue content-filters is somewhat unattractive, as typically > one wants low-latency, modest concurrency in a post-queue filter. This is where before-queue filtersand Milters are in a better position. I expect that there will be pressure to move away from after-queue filters, despite the limitations of the before-queue approach. Wietse
Re: DNS Whitelisting
On 11/5/2010 10:03 AM, Wietse Venema wrote: This is now implemented with minor changes. Excellent! Looking forward to a test drive. -- Noel Jones
Re: DNS Whitelisting
On Fri, Nov 05, 2010 at 11:03:34AM -0400, Wietse Venema wrote: > The current manpage text reads: > >reject_rbl_client rbl_domain=d.d.d.d > ... >permit_dnswl_client dnswl_domain=d.d.d.d > Accept the request when the reversed client network address is > listed with the A record "d.d.d.d" under dnswl_domain. If no > "=d.d.d.d" is specified, accept the request when the reversed > client network address is listed with any A record under > dnswl_domain. > For safety, permit_dnswl_client is silently ignored when it > would override reject_unauth_destination.The result is > DEFER_IF_REJECT when whitelist lookup fails. This feature is > available in Postfix 2.8 and later. > ... >reject_rhsbl_client rbl_domain=d.d.d.d > ... >permit_rhswl_client rhswl_domain=d.d.d.d > Accept the request when the client hostname is listed with the A > record "d.d.d.d" under rhswl_domain. If no "=d.d.d.d" is speci- > fied, accept the request when the client hostname is listed with > any A record under rhswl_domain. > For safety, permit_rhswl_client is silently ignored when it > would override reject_unauth_destination.The result is > DEFER_IF_REJECT when whitelist lookup fails. This feature is > available in Postfix 2.8 and later. Looks good. Should we mention that these should only be used to reduce FPs from blacklists that follow, and that are expected to not list legitimate clients. Thus any temporary DNS lookup error would likely result an an additional lookup that incurrs a low risk or rejection, but does *imply* rejection. DNS-based positive access rules are fragile with respect to the semantics of temporary lookup failures, and must not be used to permit some while denying all. There will at some point be interest in DNSWL support for verified DKIM "d=" domains. For now that's out of scope (milters, pre-queue filters, ...) I've recently starting using the OpenDKIM library, ... it is fairly easy to support. If there is ever interest in directly supporting DKIM in the Postfix SMTP server, I'm game to talk design. Due to the DNS lookup latency inherent in incoming DKIM checks, doing DKIM in post-queue content-filters is somewhat unattractive, as typically one wants low-latency, modest concurrency in a post-queue filter. -- Viktor.
Re: DNS Whitelisting
On 8/25/2010 6:20 PM, Rob Foehl wrote: On Wed, 25 Aug 2010, Noel Jones wrote: The user interface would be familiar to anyone using rbl checks. Sample documentation under the appropriate smtpd_mumble_restrictions section: - permit_dnswl_client dnswl_domain=d.d.d.d Accept the request when the reversed client IP network address is listed with an A record of d.d.d.d under dnswl_domain. If no =d.d.d.d is given, accept the request with any A record under dnswl_domain. For safety, only authorized destinations are accepted, see permit_auth_destination. - permit_rhswl_client rhswl_domain=d.d.d.d Accept the request when the client hostname is listed with an A record of d.d.d.d under rhswl_domain. If no =d.d.d.d is given, accept the request with any A record under rhswl_domain. For safety, only authorized destinations are accepted, see permit_auth_destination. Seems like this one would be very easy to use, and fairly easy to implement. This sounds like a reasonable proposal, and I would argue that maintaining parity with existing smtpd features is important, whether or not postscreen ever grows an analogous mechanism. Unconditionally returning permit_auth_destination should make this suitably safe, despite the simple interface. Although most discussion has been about postscreen, I'm still very interested in dns whitelisting in smtpd. Once we (collectively) get the postscreen dnsxl scoring user interface sorted out, it should be possible to adapt the framework for smtpd without reinventing the wheel. Then there would be a unified interface for dnsxl scoring. As it happens, I have a partial implementation of such a feature that I was playing with a few years ago, and could probably be coerced into updating it for current releases and posting a patch, if there is further consensus that this is the desired interface and/or mechanism. -Rob The simple interface proposed above should be much easier to implement (I'll bet a lot of existing rbl code could be reused), but let's shoot for Mars right now. If we have to settle for the Moon later, that's not so bad. -- Noel Jones
Re: DNS Whitelisting
Updated Proposal for weighted dnsXl support in postscreen.
(Change parameter names to all start with postscreen_dns* for
easy reading in postconf. Get rid of negative site weight
values [the client dnsxl score total may still be negative].
Add filter octet range docs.)
(The weight ranges documented are arbitrary.)
- postscreen_dnsbl_sites (default empty); A comma separated
list of dnsbl IP blacklist sites with optional result filter
and optional weight. When the reversed client network address
is listed with an A record matching the result filter, add
{weight} points (default 1) to the client dnsxl score. If no
result filter is specified, add {weight} points (default 1) to
the client dnsxl score if any A record is found. If multiple
A records are found, {weight} will only be added once.
Specify one or more dnsbl sites as:
dnsbl_site[=d.d.d.d][*N]
where dnsbl_site is the site name, d.d.d.d is the optional
result filter, and N is the optional weight value. A range in
the result filter may be specified within brackets in place of
an octet. Multiple ranges or single values may be separated
by commas within the brackets. The weight may be specified as
the character "*" followed by a value in the range [0~99]
inclusive.
Examples:
postscreen_dnsbl_sites =
dnsbl_site1,
dnsbl_site2=127.0.[0-5,22,128-255].2*5,
dnsbl_site3=*6
- postscreen_dnswl_sites (default empty); A comma separated
list of dnswl IP whitelist sites with optional result filter
and optional weight. When the reversed client network address
is listed with an A record matching the result filter,
subtract {weight} points (default 1) from the client dnsxl
score. If no result filter is specified, subtract {weight}
points (default 1) from the client dnsxl score if any A record
is listed. If multiple A records are found, {weight} will
only be subtracted once.
Specify one or more dnswl sites as:
dnswl_site[=d.d.d.d][*N]
where dnswl_site is the site name, d.d.d.d is the optional
result filter, and N is the optional weight value. A range in
the result filter may be specified within brackets in place of
an octet. Multiple ranges or single values may be separated
by commas within the brackets. The weight may be specified as
the character "*" followed by a value in the range [0~99]
inclusive.
Examples:
postscreen_dnswl_sites =
dnswl_site1,
dnswl_site2=127.0.[0-5,22,128-255].2*5,
dnswl_site3=*6
(these next parameters behavior is unchanged, but the docs
have been updated)
(Require a "+" or "-" sign for the score thresholds to prevent
ambiguity. The alternatives are to assume "-" for the
whitelist and "+" for the blacklist, or always assume "+". I
think it's least confusing to just require the sign.)
(The score threshold range is arbitrary.)
- postscreen_dnsxl_whitelist_score (default -1); a "pass"
threshold for the total of the client's dnsxl points. Specify
a value in the range [-999~+999] inclusive. The sign must be
specified. Clients scoring at or BELOW this value trigger the
postscreen_dnsxl_whitelist_action. Clients scoring greater
than postscreen_dnsxl_whitelist_score, but less than
postscreen_dnsxl_blacklist_score continue with postscreen
analysis for disposition.
Example:
postscreen_dnsxl_whitelist_score = -5
- postscreen_dnsxl_blacklist_score (default=1) a "drop"
threshold for the total of the client's dnsxl points. Specify
a value in the range [-999~+999] inclusive. The sign must be
specified. Clients scoring at or ABOVE this value trigger the
postscreen_dnsxl_blacklist_action. Clients scoring greater
than postscreen_dnsxl_whitelist_score, but less than
postscreen_dnsxl_blacklist_score continue with postscreen
analysis for disposition.
Example:
postscreen_dnsxl_blacklist_score = +5
- postscreen_dnsxl_whitelist_action (default continue); the
action postscreen takes when a client matches the
postscreen_dnsxl_whitelist_score.
Specify one of:
continue; perform additional postscreen tests to determine
disposition.
pass; exempt the client from further postscreen tests and pass
it to a real SMTP server process
- postscreen_dnsxl_blacklist_action (default continue); the
action postscreen takes when a client exceeds the
postscreen_dnsxl_blacklist_score.
Specify one of:
continue; perform additional postscreen tests to determine
disposition.
drop; drop the connection with a 521 SMTP reply
(next two items are for future expansion if hostnames are
available)
- postscreen_dnsbl_hostname_sites (default empty); A comma
separated list of rhsbl hostname blacklist sites using the
unverified client hostname with optional result filter and
optional weight. When the unverified reverse client hostname
is listed with an A record matching the result filter, add
{weight} points (default 1) to the client dnsxl score. If no
result filter is specified, add {weight} points (default 1) to
the client dnsxl score if any A record is listed. If multip
Re: DNS Whitelisting
On 8/26/2010 4:14 PM, Wietse Venema wrote: > The more precise solution is to implement wildcards with ranges: > > example.com=127.0.[0-128].3*1 > example.com=127.0.[0-5,6-9].3*1 Noel Jones: > I like the range idea. You want proto docs reflecting that > syntax? Yes, that would help everyone to understand how it will work. Wietse
Re: DNS Whitelisting
On 8/26/2010 4:14 PM, Wietse Venema wrote: On 8/26/2010 2:28 PM, Wietse Venema wrote: You can't use an alphanumerical operator such as "w", because the "=127.0.*.3" portion is optional. ... The more precise solution is to implement wildcards with ranges: example.com=127.0.[0-128].3*1 example.com=127.0.[0-5,6-9].3*1 Wietse I like the range idea. You want proto docs reflecting that syntax? -- Noel Jones
Re: DNS Whitelisting
Noel Jones: > This looks like a useful concept. If we use "*" as an octet > wildcard, we'll need to use something else as the weight modifier. > dnsbl_site=127.0.*.3w1 seems reasonable. On 8/26/2010 2:28 PM, Wietse Venema wrote: > You can't use an alphanumerical operator such as "w", because the > "=127.0.*.3" portion is optional. Noel Jones: > Rats. =127.0.*.3^1 maybe? Your suggestion? * is such a > natural wildcard I would hate to use something else for the octet. Use "*" for the multiplier and "?" for the wildcard? Another idea is to implement wildcards with net/mask patterns: example.com=127.0.0.3/255.255.0.255*1 The more precise solution is to implement wildcards with ranges: example.com=127.0.[0-128].3*1 example.com=127.0.[0-5,6-9].3*1 Wietse
Re: DNS Whitelisting
On 8/26/2010 2:28 PM, Wietse Venema wrote: Noel Jones: This looks like a useful concept. If we use "*" as an octet wildcard, we'll need to use something else as the weight modifier. dnsbl_site=127.0.*.3w1 seems reasonable. You can't use an alphanumerical operator such as "w", because the "=127.0.*.3" portion is optional. Wietse Rats. =127.0.*.3^1 maybe? Your suggestion? * is such a natural wildcard I would hate to use something else for the octet. -- Noel Jones
Re: DNS Whitelisting
Noel Jones: > This looks like a useful concept. If we use "*" as an octet > wildcard, we'll need to use something else as the weight > modifier. dnsbl_site=127.0.*.3w1 seems reasonable. You can't use an alphanumerical operator such as "w", because the "=127.0.*.3" portion is optional. Wietse
Re: DNS Whitelisting
On 8/25/2010 4:54 PM, Noel Jones wrote: On 8/25/2010 4:27 PM, Wietse Venema wrote: Noel Jones: Do we want to allow mixing DNSWLs and DNSBLs in one list? I see them as being the same thing; just different weights. Default to blacklist weight of 1; the user must specify a negative weight for a whitelist. I've changed my mind on this. While might make sense to combine dnsbl & dnswl in one list, the problem comes up if we ever implement rhsXl because the hostname requirements are different. With an rhsbl, you want the unverified reverse name; with an rhswl, you need the FCrDNS standard postfix hostname. Separating the lists also allows us to use whitelist and blacklist default weight values, reducing configuration burden. Matthias Leisi wrote: What about wildcarding? dnswl.org currently returns 127.0.n.[0-3], with "n" being numerical for the category (eg banks etc). People may want to have something like "whitelist on 127.0.*.2 and 127.0.*.3". This looks like a useful concept. If we use "*" as an octet wildcard, we'll need to use something else as the weight modifier. dnsbl_site=127.0.*.3w1 seems reasonable. So, the modified proposal for postscreen would be: - postscreen_dnsbl_sites (default empty); A comma separated list of dnsbl IP blacklist sites with optional result filter and optional weight. Specify one or more dnsbl sites as: dnsbl_site[=d.d.d.d][wN] where dnsbl_site is the site name, d.d.d.d is the optional result filter, and N is the optional weight value. Wildcard octets in the result filter can be indicated with "*". If no result filter is given, any result is considered a match. The weight is given as the letter "w" followed by a value in the range [-99~+99] inclusive. If "+" or "-" is not specified, "+" is assumed. If no weight is given, the default weight value is +1. Examples: postscreen_dnsbl_sites = dnsbl_site1 dnsbl_site2=127.0.*.2w5 dnsbl_site3=w+6 - postscreen_dnswl_sites (default empty); A comma separated list of dnswl IP whitelist sites with optional result filter and optional weight. Specify one or more dnswl sites as: dnswl_site[=d.d.d.d][wN] where dnswl_site is the site name, d.d.d.d is the optional result filter, and N is the optional weight value. Wildcard octets in result filter can be indicated with "*". If no result filter is given, any result is considered a match. The weight is given as the letter "w" followed by a value in the range [-99~+99] inclusive. If "+" or "-" is not specified, "-" is assumed. If no weight is given, the default weight value is -1. Examples: postscreen_dnswl_sites = dnswl_site1 dnswl_site2=127.0.*.2w5 dnswl_site3=w-6 (next two items are for future expansion if hostnames are available) (name tweaks? postscreen_dnsbl_hostname_sites and postscreen_dnswl_hostname_sites so one can grep all related parameters with postscreen_dns*?) - postscreen_rhsbl_sites (default empty); A comma separated list of rhsbl hostname blacklist sites using the unverified client hostname with optional result filter and optional weight. Specify one or more rhsbl sites as: rhsbl_site[=d.d.d.d][wN] where rhsbl_site is the site name, d.d.d.d is the optional result filter, and N is the optional weight value. Wildcard octets in the result filter can be indicated with "*". If no result filter is given, any result is considered a match. The weight is given as the letter "w" followed by a value in the range [-99~+99] inclusive. If "+" or "-" is not specified, "+" is assumed. If no weight is given, the default weight value is +1. Examples: postscreen_rhsbl_sites = rhsbl_site1 rhsbl_site2=127.0.*.2w5 rhsbl_site3=w+6 - postscreen_rhswl_sites (default empty); A comma separated list of rhswl hostname whitelist sites using the client hostname with optional result filter and optional weight. Specify one or more rhswl sites as: rhswl_site[=d.d.d.d][wN] where rhswl_site is the site name, d.d.d.d is the optional result filter, and N is the optional weight value. Wildcard octets in the result filter can be indicated with "*". If no result filter is given, any result is considered a match. The weight is specified as the letter "w" followed by a value in the range [-99~+99] inclusive. If "+" or "-" is not specified, "-" is assumed. If no weight is given, the default weight value is -1. Examples: postscreen_rhsbl_sites = rhswl_site1 rhswl_site2=127.0.*.2w5 rhswl_site3=w-6 (below is unchanged except for the name change to *_dnsxl_*) - postscreen_dnsxl_whitelist_score (default=-1); a "pass" threshold score. clients scoring at or BELOW this value trigger the postscreen_dnsxl_whitelist_action. - postscreen_dnsxl_blacklist_score (default=1) a "drop" threshold score. Clients scoring at or ABOVE this value trigger the postscreen_dnsxl_blacklist_action. - postscreen_dnsxl_whitelist_action (default continue); the action postscreen takes when a client matches t
Re: DNS Whitelisting
Stan Hoeppner: > Wietse Venema put forth on 8/25/2010 4:27 PM: > > Noel Jones: > >> As I see it, there are two complementary paths we can take > >> with DNS whitelists, each with a slightly different purpose. > >> While these are both useful, neither depends on the other, so > >> postfix can implement either or both. > > > > I'll read the entire proposal later. > > > > Would this notation work: > > > > dnswl1.example.com=127.0.0.2*weight1, dnswl2.example.com=127.0.0.1*weight2 > > dnsbl3.example.com=127.0.0.3*weight3, dnsbl4.example.com=127.0.0.1*weight4 > > IMO this really depends on the "role" you expect postscreen to play. If The 127.x.x.x filters are OPTIONAL, as are the weight factors. Sites that don't need them don't specify them. This is a basic Postfix principle. If you don't need some feature then you don't need to know how to configure it. > > Currently, postscreen does not look up the client hostname, that > > is something that can be added later when there is time. > > Won't all of these dns lookups slow postscreen down? Hostname DNS lookup is no worse than DNSBL or DNSWL lookup, or talking with an SMTP client for pregreet or greylist tests. Postscreen does these things in parallel: one postscreen process handles many clients, instead of wasting one smtpd process per spambot. It's a scalability feature. > If postscreen is doing dnsxl lookups merely to decide if a connection > gets past postscreen, and then we do the same dnsbl lookup in smtpd to In that case Postscreen's DNSWL lookup would work as a DNS cache prefetch, so it would still improve Postfix over-all performance. Wietse
Re: DNS Whitelisting
Wietse Venema put forth on 8/25/2010 4:27 PM: > Noel Jones: >> As I see it, there are two complementary paths we can take >> with DNS whitelists, each with a slightly different purpose. >> While these are both useful, neither depends on the other, so >> postfix can implement either or both. > > I'll read the entire proposal later. > > Would this notation work: > > dnswl1.example.com=127.0.0.2*weight1, dnswl2.example.com=127.0.0.1*weight2 > dnsbl3.example.com=127.0.0.3*weight3, dnsbl4.example.com=127.0.0.1*weight4 IMO this really depends on the "role" you expect postscreen to play. If it is expected that all Postfix implementations are going to be using postscreen as in integral important feature, not just the big operations with hundreds of thousands or millions of connections per day, then I would suggest this user configuration interface is too complicated. There will be OPs who simply want to use a single dns whitelist without any kind of scoring. They will want an "OK" on any 127.0.x.x response. > It would reduce the number of config parameters. Always a plus, but in this case it may sacrifice (ease of) usability for some OPs. > Do we want to allow mixing DNSWLs and DNSBLs in one list? If you're implementing a pure scoring system this might work. I don't know what the math should be though for calculating an "OK" threshold. > Currently, postscreen does not look up the client hostname, that > is something that can be added later when there is time. Won't all of these dns lookups slow postscreen down? I thought it was supposed to be a really high throughput low latency check/allow/dump filter mainly to shield smtpd processes from the bot load. What's the average dnsxl response time? 50ms? 100ms? Won't this terribly decrease the throughput of postscreen? If postscreen is doing dnsxl lookups merely to decide if a connection gets past postscreen, and then we do the same dnsbl lookup in smtpd to decide accept/reject because postscreen doesn't pass the result to smtpd, it makes me wonder why we're doing two such lookups for each connect that is OK'd by postscreen. Again, I'm not so sure that doing dnswl lookups with postscreen is productive. Most hosts listed on dnswls are going to pass postscreen's normal checks anyway. By doing this it seems we're merely adding processing latency to a latency sensitive code path, and we're not really gaining anything by doing so. These dnswl lookups will have to be duplicated in smtpd anyway to get the desired whitelisting result OPs expect from a dnswl positive return. -- Stan
Re: DNS Whitelisting
Matthias Leisi: > On Wed, Aug 25, 2010 at 11:27 PM, Wietse Venema wrote: > > > ?dnswl1.example.com=127.0.0.2*weight1, dnswl2.example.com=127.0.0.1*weight2 > > ?dnsbl3.example.com=127.0.0.3*weight3, dnsbl4.example.com=127.0.0.1*weight4 > > What about wildcarding? dnswl.org currently returns 127.0.n.[0-3], > with "n" being numerical for the category (eg banks etc). People may > want to have something like "whitelist on 127.0.*.2 and 127.0.*.3". The "*" operator is already used for the weight factor. I'm not sure that it is a good idea to use the same "*" operator for wildcarding. Wietse
Re: DNS Whitelisting
On Wed, Aug 25, 2010 at 11:27 PM, Wietse Venema wrote: > dnswl1.example.com=127.0.0.2*weight1, dnswl2.example.com=127.0.0.1*weight2 > dnsbl3.example.com=127.0.0.3*weight3, dnsbl4.example.com=127.0.0.1*weight4 What about wildcarding? dnswl.org currently returns 127.0.n.[0-3], with "n" being numerical for the category (eg banks etc). People may want to have something like "whitelist on 127.0.*.2 and 127.0.*.3". // Yes, the use and ordering of the octets was possibly not the best design choice back when. -- Matthias
Re: DNS Whitelisting
* Wietse Venema : > Noel Jones: > > As I see it, there are two complementary paths we can take > > with DNS whitelists, each with a slightly different purpose. > > While these are both useful, neither depends on the other, so > > postfix can implement either or both. > > I'll read the entire proposal later. > > Would this notation work: > > dnswl1.example.com=127.0.0.2*weight1, dnswl2.example.com=127.0.0.1*weight2 > dnsbl3.example.com=127.0.0.3*weight3, dnsbl4.example.com=127.0.0.1*weight4 weightn can be negative? > Do we want to allow mixing DNSWLs and DNSBLs in one list? Probably, with positiv and negative weights? -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: DNS Whitelisting
On 8/25/2010 6:17 PM, Wietse Venema wrote: Noel Jones: On 8/25/2010 4:27 PM, Wietse Venema wrote: Noel Jones: As I see it, there are two complementary paths we can take with DNS whitelists, each with a slightly different purpose. While these are both useful, neither depends on the other, so postfix can implement either or both. I'll read the entire proposal later. Would this notation work: dnswl1.example.com=127.0.0.2*weight1, dnswl2.example.com=127.0.0.1*weight2 dnsbl3.example.com=127.0.0.3*weight3, dnsbl4.example.com=127.0.0.1*weight4 It would reduce the number of config parameters. Yes. Both the A filter and the weight would need to be optional. ie, if you want any result to have a custom weight, it must accept dnsbl1.example.com=*weight I assume the syntax would be like this: domain[=addr][*weight] where content inside [] is optional. Excellent. Currently, postscreen does not look up the client hostname, that is something that can be added later when there is time. I know. I was just hoping the hostname would be available in the new-and-improved postscreen discussed elsewhere. The primary goal is to keep spambots away from Postfix, so I'll focus on features that do for the short term. The only luxury will be the dummy SMTP engine that logs the rejected client, helo, sender, recipient. Right now I don't miss that at all. I've been reluctant to turn on postscreen dns blacklists on the main MX because it's so much harder for me to find the once-in-a-blue-moon FPs. But it runs gangbusters on the secondary. I won't miss the dnsbl hostname lookups in postscreen (it also eliminates a few more parameters), but it would still be nice to have simple hostname rhswl support in smtpd as described later in my outline. I see these as complementary features. -- Noel Jones
Re: DNS Whitelisting
On Wed, 25 Aug 2010, Noel Jones wrote: The user interface would be familiar to anyone using rbl checks. Sample documentation under the appropriate smtpd_mumble_restrictions section: - permit_dnswl_client dnswl_domain=d.d.d.d Accept the request when the reversed client IP network address is listed with an A record of d.d.d.d under dnswl_domain. If no =d.d.d.d is given, accept the request with any A record under dnswl_domain. For safety, only authorized destinations are accepted, see permit_auth_destination. - permit_rhswl_client rhswl_domain=d.d.d.d Accept the request when the client hostname is listed with an A record of d.d.d.d under rhswl_domain. If no =d.d.d.d is given, accept the request with any A record under rhswl_domain. For safety, only authorized destinations are accepted, see permit_auth_destination. Seems like this one would be very easy to use, and fairly easy to implement. This sounds like a reasonable proposal, and I would argue that maintaining parity with existing smtpd features is important, whether or not postscreen ever grows an analogous mechanism. Unconditionally returning permit_auth_destination should make this suitably safe, despite the simple interface. As it happens, I have a partial implementation of such a feature that I was playing with a few years ago, and could probably be coerced into updating it for current releases and posting a patch, if there is further consensus that this is the desired interface and/or mechanism. -Rob
Re: DNS Whitelisting
Noel Jones: > On 8/25/2010 4:27 PM, Wietse Venema wrote: > > Noel Jones: > >> As I see it, there are two complementary paths we can take > >> with DNS whitelists, each with a slightly different purpose. > >> While these are both useful, neither depends on the other, so > >> postfix can implement either or both. > > > > I'll read the entire proposal later. > > > > Would this notation work: > > > >dnswl1.example.com=127.0.0.2*weight1, > > dnswl2.example.com=127.0.0.1*weight2 > >dnsbl3.example.com=127.0.0.3*weight3, > > dnsbl4.example.com=127.0.0.1*weight4 > > > > It would reduce the number of config parameters. > > Yes. Both the A filter and the weight would need to be > optional. ie, if you want any result to have a custom weight, > it must accept > > dnsbl1.example.com=*weight I assume the syntax would be like this: domain[=addr][*weight] where content inside [] is optional. > > Currently, postscreen does not look up the client hostname, that > > is something that can be added later when there is time. > > I know. I was just hoping the hostname would be available in > the new-and-improved postscreen discussed elsewhere. The primary goal is to keep spambots away from Postfix, so I'll focus on features that do for the short term. The only luxury will be the dummy SMTP engine that logs the rejected client, helo, sender, recipient. Right now I don't miss that at all. Wietse
Re: DNS Whitelisting
On 8/25/2010 4:27 PM, Wietse Venema wrote: Noel Jones: As I see it, there are two complementary paths we can take with DNS whitelists, each with a slightly different purpose. While these are both useful, neither depends on the other, so postfix can implement either or both. I'll read the entire proposal later. Would this notation work: dnswl1.example.com=127.0.0.2*weight1, dnswl2.example.com=127.0.0.1*weight2 dnsbl3.example.com=127.0.0.3*weight3, dnsbl4.example.com=127.0.0.1*weight4 It would reduce the number of config parameters. Yes. Both the A filter and the weight would need to be optional. ie, if you want any result to have a custom weight, it must accept dnsbl1.example.com=*weight This would eliminate one lookup table and make list usage more obvious. Good. Do we want to allow mixing DNSWLs and DNSBLs in one list? I see them as being the same thing; just different weights. Default to blacklist weight of 1; the user must specify a negative weight for a whitelist. I think it would unnecessarily clutter the user interface to have separate lists. But maybe that's just me. Currently, postscreen does not look up the client hostname, that is something that can be added later when there is time. I know. I was just hoping the hostname would be available in the new-and-improved postscreen discussed elsewhere. Wietse -- Noel Jones
Re: DNS Whitelisting
Noel Jones: > As I see it, there are two complementary paths we can take > with DNS whitelists, each with a slightly different purpose. > While these are both useful, neither depends on the other, so > postfix can implement either or both. I'll read the entire proposal later. Would this notation work: dnswl1.example.com=127.0.0.2*weight1, dnswl2.example.com=127.0.0.1*weight2 dnsbl3.example.com=127.0.0.3*weight3, dnsbl4.example.com=127.0.0.1*weight4 It would reduce the number of config parameters. Do we want to allow mixing DNSWLs and DNSBLs in one list? Currently, postscreen does not look up the client hostname, that is something that can be added later when there is time. Wietse
Re: DNS Whitelisting
As I see it, there are two complementary paths we can take
with DNS whitelists, each with a slightly different purpose.
While these are both useful, neither depends on the other, so
postfix can implement either or both.
My proposals:
A) scoring in postscreen
A dns whitelist/blacklist scoring system in postscreen,
whereby a dnswl hit can overrule dnsbl hits and other
postscreen tests.
Since this is in postscreen, the only data available would be
the client IP and hopefully the hostname. This would be
automatically safe from open-relay abuse since the client gets
no more rights than other "screened" clients. The transaction
could still be rejected by other tests done in the real SMTP
server. The user interface would consist of
- postscreen_dnsl_client_sites (default empty); a list of
dnsbl and dnswl domains for client IP lookup.
Specify a list of dns blacklist and whitelist sites with an
optional A result: dnsl_site=d.d.d.d, dnsl_site=d.d.d.d, ...
Count one hit when the A record =d.d.d.d is listed. If no
=d.d.d.d is specified, count one hit if one or more A records
is listed.
- postscreen_dnsl_hostname_sites (default empty); a list of
dnsbl and dnswl sites for hostname lookup using the verified
client hostname. This is recommended for domain-based whitelists.
Specify a list of dns blacklist and whitelist sites with an
optional A result, dnsl_site=d.d.d.d, dnsl_site=d.d.d.d, ...
Count one hit when the A record =d.d.d.d is listed. If no
=d.d.d.d is specified, count one hit if one or more A records
is listed.
- postscreen_dnsl_reverse_hostname_sites (default empty); a
list of dnsbl and dnswl sites for hostname lookup using the
unverified client hostname. This is recommended for
domain-based blacklists.
Specify a list of dns blacklist and whitelist sites with an
optional A result, dnsl_site=d.d.d.d, dnsl_site=d.d.d.d, ...
Count one hit when the A record =d.d.d.d is listed. If no
=d.d.d.d is specified, count one hit if one or more A records
is listed.
- postscreen_dnsl_site_score_default (default 1); default
score for any dns list result. I suppose this could be a
non-user-visible default, since it doesn't seem to make much
sense to adjust this; any adjustments would be made in the
score map.
- postscreen_dnsl_site_score_maps (no default); an optional
table that overrides postscreen_dnsl_site_score_default,
indexed by the list name. The key is the exact dnsl site
name as specified in the dnsl sites maps, including any
optional =d.d.d.d result filter. The result is a positive
number for blacklists, negative number for whitelist.
This table is required when using dns whitelists.
- postscreen_dnsl_whitelist_score (default=-1); a "pass"
threshold score. clients scoring at or BELOW this value
trigger the postscreen_dnsl_whitelist_action.
- postscreen_dnsl_blacklist_score (default=1) a "drop"
threshold score. Clients scoring at or ABOVE this value
trigger the postscreen_dnsl_blacklist_action.
- postscreen_dnsl_whitelist_action (default continue); the
action postscreen takes when a client matches the
postscreen_dnsl_whitelist_score. Specify one of:
continue; perform additional postscreen tests to determine
disposition.
pass; exempt the client from further postscreen tests and pass
it to a real SMTP server process
- postscreen_dnsl_blacklist_action (default continue); the
action postscreen takes when a client exceeds the
postscreen_dnsl_blacklist_score. Specify one of:
continue; perform additional postscreen tests to determine
disposition.
drop; drop the connection with a 521 SMTP reply
There are a lot of parameters, but sane defaults would allow a
simple "drop everything using this one blacklist" config,
while still allowing much more complex multi-list scoring. A
minimal config would be simply:
postscreen_dnsl_client_sites = zen.spamhaus.org
postscreen_dnsl_blacklist_action = drop
The downside of this method is it will require a lot of new code.
B) a "permit" based system, a mirror of reject_rbl_client.
This would have a user interface similar to the existing
reject_rbl_client with expected usage similar to access(5)
based whitelists.
Seems to me that checks using sender-supplied info such as
{helo, sender domain, recipient domain} are unsafe -- give
whitelist control to unverified information -- and probably
shouldn't be implemented.
To prevent open-relay accidents, this would need to return
permit_auth_destination rather than a blanket permit. Maybe
the result action will need to be configurable? Nah.
The user interface would be familiar to anyone using rbl
checks. Sample documentation under the appropriate
smtpd_mumble_restrictions section:
- permit_dnswl_client dnswl_domain=d.d.d.d
Accept the request when the reversed client IP network
address is listed with an A record of d.d.d.d under
dnswl_domain. If no =d.d.d.d is given, accept the request
with any A record under dnswl_domain. For safety, on
Re: DNS Whitelisting
Steve Linford put forth on 8/25/2010 8:27 AM:
> Just to add to the mix if Postfix is working on whitelist implementation...
> Spamhaus has assigned 127.0.2.0/24 for whitelist return codes. The new
> Spamhaus Whitelist ("SWL") due out very shortly will return 127.0.2.2 and
> 127.0.2.3 and Spamhaus' new Domain Whitelist ("DWL") will return 127.0.2.12
> and 127.0.2.13. We will explain what these codes mean closer to the release
> date (sometime in September).
>
> In the case of the Spamhaus Whitelist ("SWL"), the implementation we want
> people to use is that a message from an IP on the SWL should be allowed
> immediately past all spam filtering and all content filtering and passed
> straight to the virus filter if any.
Thanks for the heads up Steve. :) Great timing.
--
Stan
Re: DNS Whitelisting
On 24 Aug 2010, at 21:37, Wietse Venema wrote:
> Stan Hoeppner:
>> Wietse Venema put forth on 8/23/2010 10:11 AM:
>>> Noel Jones:
>>
>>> (Might be time to revisit DNS whitelists in
postfix.)
>>>
>>> Maybe someone can draft a strawman user interface:
>>>
>>> - what is the configuration syntax
>>>
>>> - what does that syntax mean
>>>
>>> - how to make it safe ( we don't want "open relay" problems)
>>>
>>> I'm currently doing this for postscreen, and won't have time for
>>> other Postfix features.
>>
>> accept_dnswl_client (default: 0)
>>
>> 0 - accept all messages
>> 1 - accept messages with trust level 1-3
>> 2 - accept messages with trust level 2-3
>> 3 - accept messages with trust level 3
>
> This looks somewhat like RFC 5782, with reputation scores and
> confidence values encoded in the lower octets as numbers in the
> range 0-255.
>
> With reject_rbl_client etc. Postfix can use different DNSXLs names
> in different access lists, and filter the result. For example, to
> select responses from some.example.com with value 127.0.0.4:
>
> smtpd_mumble_restrictions =
> ...
>reject_rbl_client some.example.com=127.0.0.4
> ...
>
> I suppose that similar selection would be help with whitelists.
Just to add to the mix if Postfix is working on whitelist implementation...
Spamhaus has assigned 127.0.2.0/24 for whitelist return codes. The new Spamhaus
Whitelist ("SWL") due out very shortly will return 127.0.2.2 and 127.0.2.3 and
Spamhaus' new Domain Whitelist ("DWL") will return 127.0.2.12 and 127.0.2.13.
We will explain what these codes mean closer to the release date (sometime in
September).
In the case of the Spamhaus Whitelist ("SWL"), the implementation we want
people to use is that a message from an IP on the SWL should be allowed
immediately past all spam filtering and all content filtering and passed
straight to the virus filter if any.
Steve Linford
The Spamhaus Project
http://www.spamhaus.org
Re: DNS Whitelisting
Wietse Venema put forth on 8/24/2010 2:37 PM: > With reject_rbl_client etc. Postfix can use different DNSXLs names > in different access lists, and filter the result. For example, to > select responses from some.example.com with value 127.0.0.4: > > smtpd_mumble_restrictions = > ... > reject_rbl_client some.example.com=127.0.0.4 > ... > > I suppose that similar selection would be help with whitelists. Yes, I believe the dnswl.org implementation is based on RFC5782. Regarding "safety" against the "list world" and other scenarios, see section 5, and paragraph 2 of section 7. The tests are the same for dnsbls and dnswls. You simply check for responses to make sure a dnsXl is running on the host you're querying before sending real queries. I'm guessing you already have code for this in the reject_rbl_client code. >> I assume postscreen processes or passes this data to smtpd in a way that >> smtpd will automatically bypass all checks normally performed during the >> CONNECT phase. > > Postscreen passes no session information to the SMTP server. The > file handle is all the SMTP server gets. We're talking about a > really tight development budget here. Darn. With all candor and humility Wietse, I don't think postscreen is the right place to implement dnswl whitelisting. Or, I should say, it's not a complete dns whitelisting solution, but only a small first step. If I understand correctly, all this will do is shoot such a whitelisted client past all the postscreen checks. Virtually all MTAs at IPs listed by dnswl servers are going to pass the postscreen checks anyway since postscreen is looking for bots and other ill behaved SMTP clients. Thus there is little gain in dns whitelisting here here but maybe some extra performance on loaded MX'en. IMHO, for dnswl whitelisting to be implemented "properly", or "the way most people will expect it to behave", it should be implemented almost exactly like reject_rbl_client abc.dnsbl.tld n.n.n.n but with the opposite action--immediately queuing the message for mailbox delivery--but still executing recipient address verification, virus checks, and possibly content filter checks etc. For a full up dnswl implementation, if we can assume some but not all dnswl servers have multiple responses, then I'm thinking we should do something just like your reject_rbl_client example, so an OP can configure the level of "trust/accept" behavior based on his/her knowledge of a particular configured dnswl server's responses: accept_dnswl_client abc.dnsbl.tld [n.n.n.n] As with reject_rbl_client the n.n.n.n would be optional and the default behavior would be to accept or "OK" a message with any positive response code from the dnswl server, in the same manner as we would process a local access table whitelist OK action. We obviously still want recipient verification, virus scanning, maybe content filtering. So if I understand Postfix design correctly, we'd implement this in smtpd and skip all smtpd checks on a positive dnswl hit, assuming the OP inserts this parameter appropriately close to the top of the restrictions list. As with reject_rbl_client an OP would be free to insert this dnswl check in the most appropriate place in the OP's processing order. -- Stan
Re: DNS Whitelisting
Stan Hoeppner: > Noel Jones put forth on 8/24/2010 2:18 PM: > > > - This is specific for dnswl.org. Postfix needs a general mechanism. > > Other whitelists are not required to follow dnswl.org's 127.0.x.y > > mechanism. > > Yeah, I used this example as dnswl is, afaik, the most "established" of > the dns whitelists. I haven't yet looked at the return codes the others > use. > > > - what do you mean by "accept the message"; OK? suppress further rbl > > lookups? Postfix has smtpd_mumble_restrictions with a large number of reject-like features and a smaller number of permit-like features. A reject terminates evaluation for all smtpd_mumble_restrictions; a permit terminates evaluation only within one smtpd_mumble_restriction. If whitelisting were to be used as a permit-like feature (which has dangerous failure modes as discussed next) then it will have to behave like all other permit-like features without exception. DNSWL as a permit-like feature increases the risk of becoming an open relay. I don't think we want Postfix to massively fail wide open and become an open relay just because some DNSWL operator made a bad decision. Besides, I am not convinced that DNSWL is best used as an unconditional "permit" operation. Alternatively, DNSWLs would be safe to use when scores from different lists are added up, and mail is rejected when the total score exceeds some threshold. With DNSXL lookup implemented as a reject-like feature, there is no danger of Postfix massively failing wide open when the DNSWQL operator screws up. Currently the smtpd configuration language does not yet have weighted DNSXL lookups. It would be easy enough to configure a global fixed list with domains and weights. Just copy the code for the deprecated maps_rbl_domains configuration parameter and the no longer documented reject_maps_rbl restriction, and add some syntax to the maps_rbl_domains parser. Wietse
Re: DNS Whitelisting
Noel Jones put forth on 8/24/2010 2:18 PM: > - This is specific for dnswl.org. Postfix needs a general mechanism. > Other whitelists are not required to follow dnswl.org's 127.0.x.y > mechanism. Yeah, I used this example as dnswl is, afaik, the most "established" of the dns whitelists. I haven't yet looked at the return codes the others use. > - what do you mean by "accept the message"; OK? suppress further rbl > lookups? This is something that needs discussion due to various expectations of what a dns "whitelist" entry actually means and how it can best be implemented. I think messages with a "positive" dnswl return code should obviously bypass certain restrictions, mainly dnsbl hits, but not all restrictions--we shouldn't bypass virus checks and content filters. And if an admin wants to locally ban an IP listed in a dnswl for any reason, we shouldn't be bypassing user defined access tables. I think bypassing reject_r*bl_* and most of the inbuilt/standard client, sender, helo, etc sanity checks would be a good idea. Obviously we don't want to bypass something like reject_unlisted_recipient. I think what we really need here is consensus on the use of local whitelists. Should dns whitelists carry the same weight? If so, we may want to bypass all restrictions but virus/content filters and recipient address verification. > - If "accept" means OK, how will you protect postfix from open relay if > the dns whitelist accidentally or intentionally lists the whole internet? We currently run similar risk, but in the opposite direction, using dnsbls. I don't believe there is a technical solution to this potential issue. Every dnsbl comes with a "use at your own risk... no warranty..." disclaimer for this reason and others. Though I think the dnsbl case is worse as it can potentially stop all email flow. In the dnswl case, you'd simply see much more spam. Although, at the inbox level, the "denial of service" result may be equivalent, much like yesteryear when folks had to sift through 100 spams to find 3 legit emails they received on a given day. But at least they still received those 3. Probably difficult to implement, but maybe some kind of analysis on mail flow would help. Keep statistics counters and establish an "average volume" baseline. If volume increases 10x over baseline, throttle all connections. The programming effort may not be worth the payoff here given the odds of "list world" occurrences. > - what would the user interface look like? Is it possible to document > it clearly? I think something like I mentioned before would be good. Simple single line configuration parameter populating one global program variable. How the variable would be used is the tricky part, and up to others. I'm not much of a programmer these days. :( I think it would be wise to allow flexible use of accept_dnswl_client much like reject_r*bl_foo which can today be used within access tables in addition to directly within smtpd_foo_restrictions. If we make its functionality very permissive by default, I probably wouldn't make it globally effective by default. WRT documentation, that will depend on the values we allow for the variable. If we do something like the 0-3 thing similar to dnswl.org, it's not as clear due to complexity. If we make it boolean, it's very clear. In this latter case, lots of bold starred italic documentation needs to inform the OP of the risks of turning it on. -- Stan
Re: DNS Whitelisting
Stan Hoeppner: > Wietse Venema put forth on 8/23/2010 10:11 AM: > > Noel Jones: > > > (Might be time to revisit DNS whitelists in > >> postfix.) > > > > Maybe someone can draft a strawman user interface: > > > > - what is the configuration syntax > > > > - what does that syntax mean > > > > - how to make it safe ( we don't want "open relay" problems) > > > > I'm currently doing this for postscreen, and won't have time for > > other Postfix features. > > accept_dnswl_client (default: 0) > > 0 - accept all messages > 1 - accept messages with trust level 1-3 > 2 - accept messages with trust level 2-3 > 3 - accept messages with trust level 3 This looks somewhat like RFC 5782, with reputation scores and confidence values encoded in the lower octets as numbers in the range 0-255. With reject_rbl_client etc. Postfix can use different DNSXLs names in different access lists, and filter the result. For example, to select responses from some.example.com with value 127.0.0.4: smtpd_mumble_restrictions = ... reject_rbl_client some.example.com=127.0.0.4 ... I suppose that similar selection would be help with whitelists. > I assume postscreen processes or passes this data to smtpd in a way that > smtpd will automatically bypass all checks normally performed during the > CONNECT phase. Postscreen passes no session information to the SMTP server. The file handle is all the SMTP server gets. We're talking about a really tight development budget here. Wietse
Re: DNS Whitelisting
On 8/24/2010 1:36 PM, Stan Hoeppner wrote: Wietse Venema put forth on 8/23/2010 10:11 AM: Noel Jones: (Might be time to revisit DNS whitelists in postfix.) Maybe someone can draft a strawman user interface: - what is the configuration syntax - what does that syntax mean - how to make it safe ( we don't want "open relay" problems) I'm currently doing this for postscreen, and won't have time for other Postfix features. accept_dnswl_client (default: 0) 0 - accept all messages 1 - accept messages with trust level 1-3 2 - accept messages with trust level 2-3 3 - accept messages with trust level 3 Trust level is a numerical 0-3 value returned as the last octet of the 127.0.x.x address, see below. The third octet is more informational than actionable, and IMO would simply add unnecessary complexity if used to determine deliverability. Decisions should be based solely on the last octet data, the "trust" level. I assume postscreen processes or passes this data to smtpd in a way that smtpd will automatically bypass all checks normally performed during the CONNECT phase. Below is how dnswl.org does dns based whitelisting. Other dns whitelist operators may use a different standard, or no standard--I've not investigated the few others at this point. AFAIK dnswl.org is the most popular, and was the original dns whitelist operator. === How to query DNSWL The query must always go to the zone "list.dnswl.org" in standard DNSBL format, ie with a reversed dotted quad IP address. To query whether the IP address "1.2.3.4" is listed, the query would thus be 4.3.2.1.list.dnswl.org The list contains the standard test entry of 127.0.0.2, which you can also test manually matthias:~> host 2.0.0.127.list.dnswl.org 2.0.0.127.list.dnswl.org has address 127.0.10.0 Return codes The return codes are structured as 127.0.x.y, with "x" indicating the category of an entry and "y" indicating how trustworthy an entry has been judged. Categories (127.0.X.y): * 2 - Financial services * 3 - Email Service Providers * 4 - Organisations (both for-profit [ie companies] and non-profit) * 5 - Service/network providers * 6 - Personal/private servers * 7 - Travel/leisure industry * 8 - Public sector/governments * 9 - Media and Tech companies * 10 - some special cases * 11 - Education, academic * 12 - Healthcare * 13 - Manufacturing/Industrial * 14 - Retail/Wholesale/Services * 15 - Email Marketing Providers Trustworthiness / Score (127.0.x.Y): * 0 = none - only avoid outright blocking (eg Hotmail, Yahoo mailservers, -0.1) * 1 = low - reduce chance of false positives (-1.0) * 2 = medium - make sure to avoid false positives but allow override for clear cases (-10.0) * 3 = high - avoid override (-100.0). The scores in parantheses are typical SpamAssassin scores - This is specific for dnswl.org. Postfix needs a general mechanism. Other whitelists are not required to follow dnswl.org's 127.0.x.y mechanism. - what do you mean by "accept the message"; OK? suppress further rbl lookups? - If "accept" means OK, how will you protect postfix from open relay if the dns whitelist accidentally or intentionally lists the whole internet? - what would the user interface look like? Is it possible to document it clearly? -- Noel
Re: DNS Whitelisting
Wietse Venema put forth on 8/23/2010 10:11 AM: > Noel Jones: > (Might be time to revisit DNS whitelists in >> postfix.) > > Maybe someone can draft a strawman user interface: > > - what is the configuration syntax > > - what does that syntax mean > > - how to make it safe ( we don't want "open relay" problems) > > I'm currently doing this for postscreen, and won't have time for > other Postfix features. accept_dnswl_client (default: 0) 0 - accept all messages 1 - accept messages with trust level 1-3 2 - accept messages with trust level 2-3 3 - accept messages with trust level 3 Trust level is a numerical 0-3 value returned as the last octet of the 127.0.x.x address, see below. The third octet is more informational than actionable, and IMO would simply add unnecessary complexity if used to determine deliverability. Decisions should be based solely on the last octet data, the "trust" level. I assume postscreen processes or passes this data to smtpd in a way that smtpd will automatically bypass all checks normally performed during the CONNECT phase. Below is how dnswl.org does dns based whitelisting. Other dns whitelist operators may use a different standard, or no standard--I've not investigated the few others at this point. AFAIK dnswl.org is the most popular, and was the original dns whitelist operator. === How to query DNSWL The query must always go to the zone "list.dnswl.org" in standard DNSBL format, ie with a reversed dotted quad IP address. To query whether the IP address "1.2.3.4" is listed, the query would thus be 4.3.2.1.list.dnswl.org The list contains the standard test entry of 127.0.0.2, which you can also test manually matthias:~ > host 2.0.0.127.list.dnswl.org 2.0.0.127.list.dnswl.org has address 127.0.10.0 Return codes The return codes are structured as 127.0.x.y, with "x" indicating the category of an entry and "y" indicating how trustworthy an entry has been judged. Categories (127.0.X.y): * 2 - Financial services * 3 - Email Service Providers * 4 - Organisations (both for-profit [ie companies] and non-profit) * 5 - Service/network providers * 6 - Personal/private servers * 7 - Travel/leisure industry * 8 - Public sector/governments * 9 - Media and Tech companies * 10 - some special cases * 11 - Education, academic * 12 - Healthcare * 13 - Manufacturing/Industrial * 14 - Retail/Wholesale/Services * 15 - Email Marketing Providers Trustworthiness / Score (127.0.x.Y): * 0 = none - only avoid outright blocking (eg Hotmail, Yahoo mailservers, -0.1) * 1 = low - reduce chance of false positives (-1.0) * 2 = medium - make sure to avoid false positives but allow override for clear cases (-10.0) * 3 = high - avoid override (-100.0). The scores in parantheses are typical SpamAssassin scores -- Stan
