Re: [courier-users] Whitelist filtering, including a draft patch
On Thu 24/Jan/2013 13:08:21 +0100 Sam Varshavchik wrote:
>
> Just looking at the results of trying to meta-program the DNSBL lookup
> logic in couriertcpd, using pcre, doesn't look right to me. PCRE in
> couriertcpd would be ok, it's just all the rest of it looks overly
> complicated.
Yeah, it reminds me of Apache's mod_rewrite, somewhat. I guess regex
programming tends to produce such kind of intricacies.
> Looking at what dnswl.org is doing, probably the best thing to do
> there is not to use BLOCK in conjunction with other DNSBLs, and just
> have a generic setting that records environment variables, and leave
> it up to a custom global mail filtering module to process all the
> lookups and figure out what to do with them.
>
> And, looking at what dnswl.org, I have a healthy dose of skepticism
> about them, anyway.
IME, it's good for SPF: Instead of ?all, one can end an SPF record
with ?exists:%{ir}.list.dnswl.org -all. The latter brings much the
same gain as the former, without being overly permissive. After all,
dnswl.org delivers a domain name, whose abuse team can be targeted.
Given what SPF is, the cost of delaying those rejections until the end
of data, so as to let the global filter handle them, seems to be
acceptable.
It would have been different if spamhausewhitelist had had a success
comparable to that of their BLs. But the lack of participation in
this thread --except for your polite replies-- is itself a gauge of
the apathy that characterizes this subject.
--
--
Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. ON SALE this month only -- learn more at:
http://p.sf.net/sfu/learnnow-d2d
___
courier-users mailing list
courier-users@lists.sourceforge.net
Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Whitelist filtering, including a draft patch
Alessandro Vesely writes: Global filters, however, cannot. My initial idea was to use ctlfile records to pass values to --and possibly among-- filters. It's you who said it's simpler to use the message header (there were traces of that argument quoted in my previous message.) Perhaps, but this was going way over into the other direction. Yes, that was my initial approach as well. Lists deliver their content as A records in the 127.0.0.0/8 range, but each list has its own encoding. Dnswl.org, for example, encodes the score is the last byte, valued 0-3. After a few attempts at defining a suitable syntax, I realized I was reinventing regular expressions, so I turned to the real thing. Some decisions, e.g. SPFALLOW, ought to be made that early. Would PCRE unduly aggravate tcpd's footprint? And how'd you compare its cpu cost to the time/bandwidth cost of the query itself? If that's the case, then this is something that better to be dealt with in the global filter itself. The result of the DNSBL lookup would only need to be preserved in the headers, and a global mail filter module would have the necessary domain knowledge to know how to process the results. Just looking at the results of trying to meta-program the DNSBL lookup logic in couriertcpd, using pcre, doesn't look right to me. PCRE in couriertcpd would be ok, it's just all the rest of it looks overly complicated. Looking at what dnswl.org is doing, probably the best thing to do there is not to use BLOCK in conjunction with other DNSBLs, and just have a generic setting that records environment variables, and leave it up to a custom global mail filtering module to process all the lookups and figure out what to do with them. And, looking at what dnswl.org, I have a healthy dose of skepticism about them, anyway. pgpuq9z4b_h_k.pgp Description: PGP signature -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Whitelist filtering, including a draft patch
On Thu 24/Jan/2013 02:00:15 +0100 Sam Varshavchik wrote: > Alessandro Vesely writes: > >> -allow=zone[,varspec]... >> >> [...] >> Now for passing the results to global filters. In 2010 we thought of > > I still think -allow is ok, but I don't see much need to do anything > with regards to the headers. If the purpose is to allow per-recipient > mail filtering, then environment variables set by couriertcpd should > be accessible in rcptfilter and smtpfilter-based scripts for that > purpose, and it's better to do it there. Global filters, however, cannot. My initial idea was to use ctlfile records to pass values to --and possibly among-- filters. It's you who said it's simpler to use the message header (there were traces of that argument quoted in my previous message.) An advantage, in large systems, is that it makes it possible to pass values from a boundary MTA to an internal MTA. In addition, using a standardized header would allow independently developed filters, e.g. SA, to reuse that result rather than querying the list again on their own. Oh, I forgot to mention to add, say, "opt BOFHARGUARDATOMS=2". That is the number of rightmost "atoms" to be kept from config-me: The lower the safer. For example, if me is host.example.com, 2 yields example.com, and an A-R header purportedly by whatever.example.com gets renamed. That's a possible implementation of RFC 5451. > And I'd rather keep it simple, and avoid the need to bring in PCRE. > I'm not even quite sure why that's necessary. A DNSBL that offers > multiple results should do it by returning pseudo-IP addresses, which > couriertcpd should be able to handle. Yes, that was my initial approach as well. Lists deliver their content as A records in the 127.0.0.0/8 range, but each list has its own encoding. Dnswl.org, for example, encodes the score is the last byte, valued 0-3. After a few attempts at defining a suitable syntax, I realized I was reinventing regular expressions, so I turned to the real thing. Some decisions, e.g. SPFALLOW, ought to be made that early. Would PCRE unduly aggravate tcpd's footprint? And how'd you compare its cpu cost to the time/bandwidth cost of the query itself? -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Whitelist filtering, including a draft patch
Alessandro Vesely writes: Hi Sam and All, let me try and resume this: -allow=zone[,varspec]... By example --so as to solicit comments from more people-- the change enables to configure an additional variable in etc/esmtpd: WHITELISTS='-allow=swl.spamhaus.org' The default behavior is to set an empty BLOCK= so that if one had set BLACKLISTS='-block=...' with the same default variable, when the former lookup succeeds, the latter lookup is preempt(i)ed, so to say. Now for passing the results to global filters. In 2010 we thought of I still think -allow is ok, but I don't see much need to do anything with regards to the headers. If the purpose is to allow per-recipient mail filtering, then environment variables set by couriertcpd should be accessible in rcptfilter and smtpfilter-based scripts for that purpose, and it's better to do it there. And I'd rather keep it simple, and avoid the need to bring in PCRE. I'm not even quite sure why that's necessary. A DNSBL that offers multiple results should to it by returning pseudo-IP addresses, which couriertcpd should be able to handle. pgpyzUy7u09QI.pgp Description: PGP signature -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnnow-d2d___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Whitelist filtering, including a draft patch
Hi Sam and All, let me try and resume this: -allow=zone[,varspec]... By example --so as to solicit comments from more people-- the change enables to configure an additional variable in etc/esmtpd: WHITELISTS='-allow=swl.spamhaus.org' The default behavior is to set an empty BLOCK= so that if one had set BLACKLISTS='-block=...' with the same default variable, when the former lookup succeeds, the latter lookup is preempt(i)ed, so to say. Now for passing the results to global filters. In 2010 we thought of inventing an X-DNSWL: header just for this purpose (see below). That is not needed, as there exists an Authentication-Results: header designed for exactly this purpose. It is to be added at border MTAs and examined by downstream filters and even MUAs. We just need to define what values we may need to know for DNS lists. We certainly need a zone, albeit we can lookup a local copy of it, for example: WHITELISTS='-allow=dnswl.local,' WHITELISTS+='_A_R_DNSWL_ZONE=list.dnswl.org,' # always set this That varspec sets the globally known name of the list. However, we need to examine the value that the list returns, in order to decide if that is a pass or a neutral response. Regular expressions can do that, for example: WHITELISTS+='/[1-3]$/,'# match only low-hi scores WHITELISTS+='_A_R_DNSWL_RESULT:pass,' # set "pass" if matched WHITELISTS+='//,' # match anything WHITELISTS+='_A_R_DNSWL_RESULT:found,' # set "found" if matched WHITELISTS+='_A_R_DNSWL_RESULT=fail,' # set "fail" otherwise In the above examples, the assignments done with ':' require that the record is found and the last regex matched, while '=' sets the variable unconditionally, provided that it is not set already. Of course, regex can also capture substrings. By providing a name to a captured string, it is possible to compose a variable name, like so: WHITELISTS+='_A_R_DNSWL/(?[0-9.]*)/,' WHITELISTS+=\ '_A_R_DNSWL|(?[-a-z0-9.]+)\s*(?https?://\S*)|:t' The last line exemplifies using a delimiter different from "/", so as to avoid backslashes. However, parentheses and question marks create difficulties with the shell, and it may be simpler to leave them in the environment. The trailing ":t" is for requiring the TXT record. I hope that is not overly complicated. The patch attached is a draft, in the sense that it should clarify any detail, but I haven't even tried to compile it, although I function-wise tested some parts of it. The idea is that, given the above setting, couriertcpd will set, for 127.0.0.2: _A_R_DNSWL_ZONE=list.dnswl.org _A_R_DNSWL_RESULT=found _A_R_DNSWL=127.0.10.0 _A_R_DNSWL_VALUE=127.0.10.0 Submit will recognize those particular variable names and write a line like: Authentication-Results: host.example.com; dnswl=found dns.zone=list.dnswl.org policy.value=127.0.10.0 Another variable I added to submit is SPFALLOW for whitelisted senders. (I set my site's SPF record that way, and it works great. However, not all those who publish -all do so, which is why I need SPFALLOW.) In general, if we work out the few issues that remain, -allow can result in an interesting alternative to -access, where rbldnsd or similar tool would play a role equivalent to bdb. What you think? Sam, if you give me the green light, two specific issues are whether to modify argparse, and whether we need rfc2047_encode (it seems only Received: uses that). For All, the properties of the new "dnswl" authentication method: They will have to be eventually added to http://www.iana.org/assignments/email-auth/email-auth.xml On Wed 29/Sep/2010 23:56:50 +0200 Sam Varshavchik wrote: > Alessandro Vesely writes: >> On 28/Sep/10 23:18, Sam Varshavchik wrote: >>> Alessandro Vesely writes: An alternative is to define skiplists explicitly, e.g. -allow=swl.spamhaus.org,ALLOW \ -block=zen.spamhaus.org,BLOCK,"550 Reject ip=@" \ -block=some.other.example,BLOCK2 \ -skipif=ALLOW,BLOCK,BLOCK2 In this case "-allow" is exactly a synonym of "-block", because it feels ugly to say "block" for a whitelist. >>> >>> This looks more reasonable. [...] >>> >>> -skipif looks cleaner. Also, the lookup will be skipped anyway, if its >>> environment variable is already set, so this is backwards compatible. >> >> That's also my opinion. (It would be nice to hear some other user, >> though.) >> Do we need an additional comma to specify whether the list is (supposed to be) IPv6 capable? >>> >>> I don't think it will be necessary. IPv6 addresses should >>> automatically format an IPv6 query. An IPv6-formatted query to an IPv4 >>> list will be quietly ignored, DNSBLs already get a lot of crap >>> queries, and they should return an NXDOMAIN by default. >> >> Matthias Leisi said list.dnswl.org is also planning to do IPv6, "to >> adapt to the changing landscape of whitelisting", and >> "implementation should start towards the end of Q1/2011 the
Re: [courier-users] Whitelist filtering
On 29/Sep/10 23:56, Sam Varshavchik wrote: > Alessandro Vesely writes: >> Renaming can be done conveniently in submit, along with >> "Old-Return-Path" and "Old-Received-SPF". > > Yes, I see that. This would mean that handling of the environment > variables from couriertcpd would need to be done in submit rather than > courieresmtpd, and arrangements will have to be made to preserve the > environment variables for submit. OTOH, if those lines are placed /before/ Courier's "Received", filters can be smart enough to discern their origin. (The only difficulty --if local filtering is enabled-- is to skip any "Received: from localhost", for that determination.) > It makes no sense to have similar functionality in two places, and > to be consisted it would have to be Old-X-DNSBL. The "X-" prefix itself is not needed. That naming convention has been dropped in the passage from RFC 822 to RFC 2822. -- -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Whitelist filtering
Alessandro Vesely writes: On 28/Sep/10 23:18, Sam Varshavchik wrote: Alessandro Vesely writes: An alternative is to define skiplists explicitly, e.g. -allow=swl.spamhaus.org,ALLOW \ -block=zen.spamhaus.org,BLOCK,"550 Reject ip=@" \ -block=some.other.example,BLOCK2 \ -skipif=ALLOW,BLOCK,BLOCK2 In this case "-allow" is exactly a synonym of "-block", because it feels ugly to say "block" for a whitelist. This looks more reasonable. [...] -skipif looks cleaner. Also, the lookup will be skipped anyway, if its environment variable is already set, so this is backwards compatible. That's also my opinion. (It would be nice to hear some other user, though.) Do we need an additional comma to specify whether the list is (supposed to be) IPv6 capable? I don't think it will be necessary. IPv6 addresses should automatically format an IPv6 query. An IPv6-formatted query to an IPv4 list will be quietly ignored, DNSBLs already get a lot of crap queries, and they should return an NXDOMAIN by default. Matthias Leisi said list.dnswl.org is also planning to do IPv6, "to adapt to the changing landscape of whitelisting", and "implementation should start towards the end of Q1/2011 the latest." In order to avoid noise, we may assume as a rule --until a counter example arises-- that white lists do IPv6 while black ones don't, however coincidental it may be. In any case, we cannot do IPv6 queries until they publish their syntax. (I've found an "ugly example" in http://tools.ietf.org/html/rfc5782#section-2.4) Another difference between "-block" and "-allow" may be to change the default variable name to something different from BLOCK, just to avoid rude surprises. The logical choice, of course, would be ALLOW. In that case, the logical default for -skipif would be "BLOCK,ALLOW". I think this provides meaningful defaults and minimizes the number of settings for the most common use cases. This would be syntactical sugar. couriertcpd doesn't know anything about the daemon it spawns for each connection. couriertcpd would use this to set a private list of environment variables: _HEADER1="Blacklist: " _HEADER2="Whitelist: " "" is the content of the relevant environment variable, including substitution of '@' with the IP number, correct? Yes, couriertcpd already constructs the message, either from the DNSBL itself or by providing a stock message of its own. courieresmtpd can read these variables from the environment, prefix each one's contents by "X-DNSBL-" and prefix it to every received message. It would also need to take any existing "X-DNSBL-" header in the received message and rename it to "X-Old-DNSBL-". Renaming can be done conveniently in submit, along with "Old-Return-Path" and "Old-Received-SPF". Yes, I see that. This would mean that handling of the environment variables from couriertcpd would need to be done in submit rather than courieresmtpd, and arrangements will have to be made to preserve the environment variables for submit. It makes no sense to have similar functionality in two places, and to be consisted it would have to be Old-X-DNSBL. pgpGR9kU6aBcS.pgp Description: PGP signature -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Whitelist filtering
On 28/Sep/10 23:18, Sam Varshavchik wrote: > Alessandro Vesely writes: >> An alternative is to define skiplists explicitly, e.g. >> >> -allow=swl.spamhaus.org,ALLOW \ >> -block=zen.spamhaus.org,BLOCK,"550 Reject ip=@" \ >> -block=some.other.example,BLOCK2 \ >> -skipif=ALLOW,BLOCK,BLOCK2 >> >> In this case "-allow" is exactly a synonym of "-block", because it >> feels ugly to say "block" for a whitelist. > > This looks more reasonable. [...] > > -skipif looks cleaner. Also, the lookup will be skipped anyway, if its > environment variable is already set, so this is backwards compatible. That's also my opinion. (It would be nice to hear some other user, though.) >> Do we need an additional comma to specify whether the list is >> (supposed to be) IPv6 capable? > > I don't think it will be necessary. IPv6 addresses should > automatically format an IPv6 query. An IPv6-formatted query to an IPv4 > list will be quietly ignored, DNSBLs already get a lot of crap > queries, and they should return an NXDOMAIN by default. Matthias Leisi said list.dnswl.org is also planning to do IPv6, "to adapt to the changing landscape of whitelisting", and "implementation should start towards the end of Q1/2011 the latest." In order to avoid noise, we may assume as a rule --until a counter example arises-- that white lists do IPv6 while black ones don't, however coincidental it may be. In any case, we cannot do IPv6 queries until they publish their syntax. (I've found an "ugly example" in http://tools.ietf.org/html/rfc5782#section-2.4) Another difference between "-block" and "-allow" may be to change the default variable name to something different from BLOCK, just to avoid rude surprises. 2. >>> >>> I think that the best way is to record this metadata into an >>> additional header, than the filter can check. Thinking it over in my >>> ahead, I think this would be the easiest approach, and all the change >>> can be bundled in courieresmtpd, so that the command line sendmail >>> injection point, or the shared codepath between the two, does not need >>> to be overengineered for that. >> >> At first glance courieresmtpd doesn't seem to delve into >> distinguishing header from body. > > It doesn't. It only needs to write new headers to submit before it > sends the original message body. Ah, I hadn't thought about that. That way, it will correctly place the new fields on the top (right after Courier's "Received", IIRC). > It will need to mask off any existing headers with the same name, > though. This is fairly easy and does not require complicated > parsing. Agreed, using a prefix makes it clear. > I think that similar to -skipif, you can specify variable name/header > name pairs by a new option: > > -header BLOCK,Blacklist \ > -header ALLOW,Whitelist > > This would be syntactical sugar. couriertcpd doesn't know anything > about the daemon it spawns for each connection. couriertcpd would use > this to set a private list of environment variables: > > _HEADER1="Blacklist: " > _HEADER2="Whitelist: " "" is the content of the relevant environment variable, including substitution of '@' with the IP number, correct? > The environment variables would be named _HEADERn, beginning at 1. > > courieresmtpd can read these variables from the environment, prefix > each one's contents by "X-DNSBL-" and prefix it to every received > message. It would also need to take any existing "X-DNSBL-" header in > the received message and rename it to "X-Old-DNSBL-". Renaming can be done conveniently in submit, along with "Old-Return-Path" and "Old-Received-SPF". > A global mail filter could then look for "X-DNSBL-" headers. This looks fine to me. The name will have to be confirmed in the filter's config file anyway. For a nit, the "B" in "X-DNSBL-" is wrong. RFC 5782 uses "DNSxL", but such term doesn't seem to be widely used outside the ASRG. >> A more generic approach could be to dump the whole environment to >> the ctlfile, in submit, just before closing it and only if filters >> are enabled, prepending, say, COMCTLFILE_ENVVAR before each line. >> This would also work for a bunch of corner cases, e.g. a filter >> wondering whether an authsmtp message came in via the MSA port. This >> may result in a size increase of around 1300 bytes, at mines; some >> known variables (PATH, SHELL, SHLVL, PWD, _) may be omitted. > > The environment is sanitized before submit gets invoked. Yup, that's why it may be cool to dump it. Nevertheless, bash adds some variables for its own use. I collected the parenthesized name list above from /proc//environ. >>> I do not understand what the problem is. Wildcard DNS entries will >>> work fine. You just put a wildcard record for the /64. End of story. >> >> IMHO, even assuming that such granularity is not too coarse, the >> remaining 64 bits still provide for a too huge address space for the >> whack-a-mole game. > > An ISP allocates a /64 for each customer. A /64 is a reasonable > starting
Re: [courier-users] Whitelist filtering
Alessandro Vesely writes: Hm... one possibility is to modify the "-block" syntax by appending a list of environment variables that prevent the lookup. E.g. -block=zen.spamhaus.org,BLOCK,"550 Reject ip=@",ALLOW,ALLOW2 \ -allow=swl.spamhaus.org,ALLOW where "-allow" has the same syntax of "-block", but new entries are pushed onto the head of the list rather than appended to its tail. An alternative is to define skiplists explicitly, e.g. -allow=swl.spamhaus.org,ALLOW \ -block=zen.spamhaus.org,BLOCK,"550 Reject ip=@" \ -block=some.other.example,BLOCK2 \ -skipif=ALLOW,BLOCK,BLOCK2 In this case "-allow" is exactly a synonym of "-block", because it feels ugly to say "block" for a whitelist. This looks more reasonable. Appending environment variables after the message string won't work very well. In some case the message string is not specified, and the one returned from DNS gets used. Furthermore, an explicit message string cannot contain commas. Quotes get parsed by the shell, the whole option setting is parsed as a single string, and there are plenty of message strings out there on the order of "550 Go away, and see http://whatever";. Banning a comma in the message strings will break many people. -skipif looks cleaner. Also, the lookup will be skipped anyway, if its environment variable is already set, so this is backwards compatible. service starting in 2011". Do we need an additional comma to specify whether the list is (supposed to be) IPv6 capable? I don't think it will be necessary. IPv6 addresses should automatically format an IPv6 query. An IPv6-formatted query to an IPv4 list will be quietly ignored, DNSBLs already get a lot of crap queries, and they should return an NXDOMAIN by default. 2. A mechanism to pass (some of) these variables to global filters. (Values /from/ global filters can be set via header fields, but variable-passing can also be devised to work both ways.) I think that the best way is to record this metadata into an additional header, than the filter can check. Thinking it over in my ahead, I think this would be the easiest approach, and all the change can be bundled in courieresmtpd, so that the command line sendmail injection point, or the shared codepath between the two, does not need to be overengineered for that. At first glance courieresmtpd doesn't seem to delve into distinguishing header from body. It doesn't. It only needs to write new headers to submit before it sends the original message body. It will need to mask off any existing headers with the same name, though. This is fairly easy and does not require complicated parsing. At any rate, yes, adding to the header may work for this specific case. The additional field, say X-Whitelisted-By: swl.spamhaus.org will remain attached to the message, and thus might also be used by client software to avoid discarding important messages. However, this header field has to be guarded against cheats already present in the original message. If the field name is specified on the command line, e.g. as -allow=swl.spamhaus.org,ALLOW,"X-Whitelisted-By: swl.spamhaus.org" checking for cheats may become quite cumbersome. I think that similar to -skipif, you can specify variable name/header name pairs by a new option: -header BLOCK,Blacklist \ -header ALLOW,Whitelist This would be syntactical sugar. couriertcpd doesn't know anything about the daemon it spawns for each connection. couriertcpd would use this to set a private list of environment variables: _HEADER1="Blacklist: " _HEADER2="Whitelist: " The environment variables would be named _HEADERn, beginning at 1. courieresmtpd can read these variables from the environment, prefix each one's contents by "X-DNSBL-" and prefix it to every received message. It would also need to take any existing "X-DNSBL-" header in the received message and rename it to "X-Old-DNSBL-". A global mail filter could then look for "X-DNSBL-" headers. A more generic approach could be to dump the whole environment to the ctlfile, in submit, just before closing it and only if filters are enabled, prepending, say, COMCTLFILE_ENVVAR before each line. This would also work for a bunch of corner cases, e.g. a filter wondering whether an authsmtp message came in via the MSA port. This may result in a size increase of around 1300 bytes, at mines; some known variables (PATH, SHELL, SHLVL, PWD, _) may be omitted. The environment is sanitized before submit gets invoked. I do not understand what the problem is. Wildcard DNS entries will work fine. You just put a wildcard record for the /64. End of story. IMHO, even assuming that such granularity is not too coarse, the remaining 64 bits still provide for a too huge address space for the whack-a-mole game. An ISP allocates a /64 for each customer. A /64 is a reasonable starting point. pgpAE4cOYEskn.pgp Description: PGP sign
Re: [courier-users] Whitelist filtering
On 28/Sep/10 00:00, Sam Varshavchik wrote: > Alessandro Vesely writes: >> I think we may additionally need two things: >> >> 1. A mechanism to skip performing some lookups in case some other >> ones already succeeded. Currently, only if a given variable is >> already set, the corresponding lookup is skipped. > > This seems trivial. The lookup can also be skipped if some other > variable is also already set. The whitelists can be arranged to be > looked up first, followed by blacklists. You mean "can be arranged" by properly setting the configuration file? Blocklists are currently checked respecting the order they have been defined. > The real challenge here is to come up with a sane syntax to specify > all of this. Hm... one possibility is to modify the "-block" syntax by appending a list of environment variables that prevent the lookup. E.g. -block=zen.spamhaus.org,BLOCK,"550 Reject ip=@",ALLOW,ALLOW2 \ -allow=swl.spamhaus.org,ALLOW where "-allow" has the same syntax of "-block", but new entries are pushed onto the head of the list rather than appended to its tail. An alternative is to define skiplists explicitly, e.g. -allow=swl.spamhaus.org,ALLOW \ -block=zen.spamhaus.org,BLOCK,"550 Reject ip=@" \ -block=some.other.example,BLOCK2 \ -skipif=ALLOW,BLOCK,BLOCK2 In this case "-allow" is exactly a synonym of "-block", because it feels ugly to say "block" for a whitelist. In the future, the SWL may have IPv6 capabilities --according to their techfaq, "IPv6 handling is not yet active. Spamhaus estimates IPv6 service starting in 2011". Do we need an additional comma to specify whether the list is (supposed to be) IPv6 capable? >> 2. A mechanism to pass (some of) these variables to global filters. >> (Values /from/ global filters can be set via header fields, but >> variable-passing can also be devised to work both ways.) > > I think that the best way is to record this metadata into an > additional header, than the filter can check. Thinking it over in my > ahead, I think this would be the easiest approach, and all the change > can be bundled in courieresmtpd, so that the command line sendmail > injection point, or the shared codepath between the two, does not need > to be overengineered for that. At first glance courieresmtpd doesn't seem to delve into distinguishing header from body. At any rate, yes, adding to the header may work for this specific case. The additional field, say X-Whitelisted-By: swl.spamhaus.org will remain attached to the message, and thus might also be used by client software to avoid discarding important messages. However, this header field has to be guarded against cheats already present in the original message. If the field name is specified on the command line, e.g. as -allow=swl.spamhaus.org,ALLOW,"X-Whitelisted-By: swl.spamhaus.org" checking for cheats may become quite cumbersome. A more generic approach could be to dump the whole environment to the ctlfile, in submit, just before closing it and only if filters are enabled, prepending, say, COMCTLFILE_ENVVAR before each line. This would also work for a bunch of corner cases, e.g. a filter wondering whether an authsmtp message came in via the MSA port. This may result in a size increase of around 1300 bytes, at mines; some known variables (PATH, SHELL, SHLVL, PWD, _) may be omitted. > I do not understand what the problem is. Wildcard DNS entries will > work fine. You just put a wildcard record for the /64. End of story. IMHO, even assuming that such granularity is not too coarse, the remaining 64 bits still provide for a too huge address space for the whack-a-mole game. But then you're right that someone may still run an IPv6 DNSBL, and it seems wise to plan a syntax that would allow to use it. > *.00.00.27.0c.03.05.01.20.blacklist.example.com TXT "Whatever" > > This presumes that, for consistency, hex is used at the octet-level > granularity for IPv6. Decimal can be used equally well. Probably that's the kind of decisions Spamhaus still has to make, since they have not given an IPv6 query example yet. -- -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Re: [courier-users] Whitelist filtering
Alessandro Vesely writes: I think we may additionally need two things: 1. A mechanism to skip performing some lookups in case some other ones already succeeded. Currently, only if a given variable is already set, the corresponding lookup is skipped. This seems trivial. The lookup can also be skipped if some other variable is also already set. The whitelists can be arranged to be looked up first, followed by blacklists. The real challenge here is to come up with a sane syntax to specify all of this. 2. A mechanism to pass (some of) these variables to global filters. (Values /from/ global filters can be set via header fields, but variable-passing can also be devised to work both ways.) I think that the best way is to record this metadata into an additional header, than the filter can check. Thinking it over in my ahead, I think this would be the easiest approach, and all the change can be bundled in courieresmtpd, so that the command line sendmail injection point, or the shared codepath between the two, does not need to be overengineered for that. Spamhaus have announced their whitelist as the dawn of a new era. In facts, the IPv4 is on the ropes and DNSBL technology cannot go to IPv6 as-is. (Let me quote just one phrase from John Levine, 26 Aug 2009: At one address per millisecond, it would take 500 million years to run through a /64. http://www.ietf.org/mail-archive/web/asrg/current/msg15743.html) I do not understand what the problem is. Wildcard DNS entries will work fine. You just put a wildcard record for the /64. End of story. Let's blacklist j.root-servers.net: j.root-servers.net. 401788 IN 2001:503:c27::2:30 Expanding this gives 2001:0503:0c27:/64. I can't think of any reason why this cannot be recorded as *.00.00.27.0c.03.05.01.20.blacklist.example.com TXT "Whatever" This presumes that, for consistency, hex is used at the octet-level granularity for IPv6. Decimal can be used equally well. I see no reason why this won't work. pgpJY4gKWHAhO.pgp Description: PGP signature -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev___ courier-users mailing list courier-users@lists.sourceforge.net Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
