Re: serious bug with check_client_access
Le 05/11/2010 10:03, Vincent Lefevre a écrit : [hash/cdb/...] - if parent_domain_matches_subdomains contains smtpd_access: here, the search list is S = ( lab1.lab2.lab3.example.com, lab2.lab3.example.com, lab3.example.com ..., com, 1.2.3.4, 1.2.3, 1.2, 1 ) so postfix will search for each element of this set and stops as soon as a match is found. Testing the tld alone seems to be excluded by the access(5) man page, which only documents domain.tld, i.e. the pattern must contain at least one dot. No dot required. so com is not excluded from the search list. otherwise, hash/cdb and the like can't be used to block mail from *...@*.invalid for example. Is it an error in the man page (which could say domain instead, like in Section Email address extension) or is it intentional? yes, it should be domain instead of domain.tld. you can submit a documentation patch;-p
Re: serious bug with check_client_access
Le 05/11/2010 09:48, Vincent Lefevre a écrit : On 2010-11-04 23:36:04 -0300, Reinaldo de Carvalho wrote: On Thu, Nov 4, 2010 at 11:13 PM, Vincent Lefevrevinc...@vinc17.net wrote: Yes, it will generate *some* lookups, but it doesn't say exactly *which* lookups. That was precisely my question. - client hostname (reverse dns hostname) - client IP address. OK, and mous said in that order (but maybe that's just the current implementation, and the user shouldn't rely on the order for the future). Given all the compatibility efforts invested in postfix (thanks to Wietse and others), I doubt this will change. That said, I believe it's better not to rely on, be it just for clarity. having things like example.comrestriction_class_1 192.0.2 restriction_class_2 (with different restriction classes or results) in the same map is a call for trouble! also, for IP addresses, better use cidr (I mean the map type, not a bottle from Normandy;-p).
Re: serious bug with check_client_access
On 2010-11-04 23:36:04 -0300, Reinaldo de Carvalho wrote: On Thu, Nov 4, 2010 at 11:13 PM, Vincent Lefevre vinc...@vinc17.net wrote: Yes, it will generate *some* lookups, but it doesn't say exactly *which* lookups. That was precisely my question. - client hostname (reverse dns hostname) - client IP address. OK, and mous said in that order (but maybe that's just the current implementation, and the user shouldn't rely on the order for the future). if smtpd_access_maps in parent_domain_matches_subdomains. - compare client hostname without the first part at left by dot - compare client hostname without the first and the second part at left by dot (and recursively at the TDL) but not with a regular expression table. -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Re: serious bug with check_client_access
On 2010-11-05 06:21:20 +0100, mouss wrote: in short, for each map, you have multiple parameters: - the map type - the search context (check_client_access, check_sender_acces, ... transport, virtual_alias_maps, ... etc) - the list of search keys [...] Thanks a lot for this very detailed answer. This was exactly the kind of description I was looking for. I have only one comment: [hash/cdb/...] - if parent_domain_matches_subdomains contains smtpd_access: here, the search list is S = ( lab1.lab2.lab3.example.com, lab2.lab3.example.com, lab3.example.com ..., com, 1.2.3.4, 1.2.3, 1.2, 1 ) so postfix will search for each element of this set and stops as soon as a match is found. Testing the tld alone seems to be excluded by the access(5) man page, which only documents domain.tld, i.e. the pattern must contain at least one dot. Is it an error in the man page (which could say domain instead, like in Section Email address extension) or is it intentional? -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Re: serious bug with check_client_access
Vincent Lefevre put forth on 11/5/2010 4:03 AM: Testing the tld alone seems to be excluded by the access(5) man page, which only documents domain.tld, i.e. the pattern must contain at least one dot. Is it an error in the man page (which could say domain instead, like in Section Email address extension) or is it intentional? If you want to block rDNS TLDs this PCRE works with check_client_access: /^.*?(info|kr|jp|sg|qa)$/i 550 We do not accept mail from .$1 domains You could also use this for check_sender_access, check_helo_access, etc--it should work with any check that passes a string with .tld in it. -- Stan
Re: serious bug with check_client_access
В Срд, 03/11/2010 в 22:16 -0500, Noel Jones пишет: On 11/3/2010 10:00 PM, Vincent Lefevre wrote: On 2010-11-03 21:40:54 -0500, Noel Jones wrote: .domain.tld only works if parent_domain_matches_subdomains does NOT include smtpd_access maps. The man page says nothing like that. So, the documentation should be fixed. The vast majority of readers seem to interpret that section as intended. You're welcome to post a documentation patch in a new thread, but I don't think the behavior or its documentation has changed in ~10 years, so don't hold your breath. Vast majority even believe in strange things... Actually I read that section exactly like Vincent Lefevre. I'll comment: The access(5) man page says: domain.tld Matches domain.tld. The pattern domain.tld also matches subdomains, but only when the string smtpd_access_maps is listed in the Postfix parent_domain_matches_subdomains configuration setting. Otherwise, specify .domain.tld (note the initial dot) in order to match subdomains. I read Otherwise as If you don't like to depend on the value of parent_domain_matches_subdomains specify .domain.tld (note the initial dot) in order to match subdomains means exactly that using .domain.tld form WILL match subdomains with no other condition. Also regarding to what you are telling there is no difference in those two sentences, and it's completelly unclear in what those two notation forms differ and why author has written one idea in two sentences. Check your English ;) And next time behavior doesn't match your expectations, you might get more sympathy if your message starts with please clarify this for me rather than serious bug. If behavior doesn't match your expectations and also doesn't match docs it's a bug, either in soft or in docs. -- Покотиленко Костик cas...@meteor.dp.ua
Re: serious bug with check_client_access
Le 04/11/2010 05:24, Noel Jones a écrit : On 11/3/2010 11:07 PM, Vincent Lefevre wrote: BTW, so, there is no way to match only subdomains (by that, I mean all possible subdomains, but not the domain itself) without changing parent_domain_matches_subdomains? That's correct with indexed tables. With regexp or pcre tables there is no automatic subdomain search; you control the scope of the search with your expression. To answer your other question, when parent_domain_matches_subdomains includes smtpd_access_maps (the default), the form .domain.tld is never searched for. As a result, the entry is silently ignored. -- Noel Jones Good to know, that is not what I expected too. Hopefully, I generaly clear parent_domain_matches_subdomains in my configurations. Emmanuel.
Re: serious bug with check_client_access
Zitat von Покотиленко Костик cas...@meteor.dp.ua: В Срд, 03/11/2010 в 22:16 -0500, Noel Jones пишет: On 11/3/2010 10:00 PM, Vincent Lefevre wrote: On 2010-11-03 21:40:54 -0500, Noel Jones wrote: .domain.tld only works if parent_domain_matches_subdomains does NOT include smtpd_access maps. The man page says nothing like that. So, the documentation should be fixed. The vast majority of readers seem to interpret that section as intended. You're welcome to post a documentation patch in a new thread, but I don't think the behavior or its documentation has changed in ~10 years, so don't hold your breath. Vast majority even believe in strange things... Actually I read that section exactly like Vincent Lefevre. I'll comment: The access(5) man page says: domain.tld Matches domain.tld. The pattern domain.tld also matches subdomains, but only when the string smtpd_access_maps is listed in the Postfix parent_domain_matches_subdomains configuration setting. Otherwise, specify .domain.tld (note the initial dot) in order to match subdomains. I read Otherwise as If you don't like to depend on the value of parent_domain_matches_subdomains Otherwise is clearly related to the last part of the previous statement. If it is unclear, ask for clarification. Crying loud serious bug because your language interpretation is different from others is not helpful at all. Regards Andreas smime.p7s Description: S/MIME Cryptographic Signature
Re: serious bug with check_client_access
В Чтв, 04/11/2010 в 10:44 +0100, lst_ho...@kwsoft.de пишет: Zitat von Покотиленко Костик cas...@meteor.dp.ua: В Срд, 03/11/2010 в 22:16 -0500, Noel Jones пишет: On 11/3/2010 10:00 PM, Vincent Lefevre wrote: On 2010-11-03 21:40:54 -0500, Noel Jones wrote: .domain.tld only works if parent_domain_matches_subdomains does NOT include smtpd_access maps. The man page says nothing like that. So, the documentation should be fixed. The vast majority of readers seem to interpret that section as intended. You're welcome to post a documentation patch in a new thread, but I don't think the behavior or its documentation has changed in ~10 years, so don't hold your breath. Vast majority even believe in strange things... Actually I read that section exactly like Vincent Lefevre. I'll comment: The access(5) man page says: domain.tld Matches domain.tld. The pattern domain.tld also matches subdomains, but only when the string smtpd_access_maps is listed in the Postfix parent_domain_matches_subdomains configuration setting. Otherwise, specify .domain.tld (note the initial dot) in order to match subdomains. I read Otherwise as If you don't like to depend on the value of parent_domain_matches_subdomains Otherwise is clearly related to the last part of the previous statement. If it is unclear, ask for clarification. Actually it's not clear to what condition the Otherwise is a conter part. So it's being decided by a reader's logic wich differs sometimes. I'm myself understood the actual meaning only after clarification on this list. And, pleople would ask for clarification only if they can't understand what is being ment, and not in case they think they unsterstand. Also, it's completelly unstated that .domain.tld notation doesn't work if smtpd_access_maps is listed in parent_domain_matches_subdomains. Crying loud serious bug because your language interpretation is different from others is not helpful at all. List subscriber classified this situation as a serious bug from it's point of view, and I think this is correct. If you don't agree, just reclassify it. BTW, clear docs save much time for both, the user, the list. So, I suggest rewritiing this paragraph as following: The pattern domain.tld also matches subdomains, but only when the string smtpd_access_maps is listed in the Postfix parent_domain_matches_subdomains configuration setting. The pattern .domain.tld (note the initial dot) matches only subdomains and only when the string smtpd_access_maps is NOT listed in the Postfix parent_domain_matches_subdomains configuration setting, otherwise it's ignored. Is this correct? -- Покотиленко Костик cas...@meteor.dp.ua
Re: serious bug with check_client_access
On 2010-11-04 10:44:34 +0100, lst_ho...@kwsoft.de wrote: The access(5) man page says: domain.tld Matches domain.tld. The pattern domain.tld also matches subdomains, but only when the string smtpd_access_maps is listed in the Postfix parent_domain_matches_subdomains configuration setting. Otherwise, specify .domain.tld (note the initial dot) in order to match subdomains. I read Otherwise as If you don't like to depend on the value of parent_domain_matches_subdomains Otherwise is clearly related to the last part of the previous statement. Yes, but it adds information in the case where smtpd_access_maps IS NOT listed in the Postfix parent_domain_matches_subdomains configuration setting. The problem occurs when smtpd_access_maps IS listed in the Postfix parent_domain_matches_subdomains configuration setting. What the man page says in THIS case is: The pattern domain.tld also matches subdomains where the pattern domain.tld can be .twitter.com for instance. I don't think there is anything wrong with my reasoning. If it is unclear, ask for clarification. Crying loud serious bug because your language interpretation is different from others is not helpful at all. As for me, the documentation was clear and didn't match the observed behavior. So, I couldn't say that it was unclear. -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Documentation (was: serious bug with check_client_access)
Vincent Lefevre: On 2010-11-04 10:44:34 +0100, lst_ho...@kwsoft.de wrote: The access(5) man page says: domain.tld Matches domain.tld. The pattern domain.tld also matches subdomains, but only when the string smtpd_access_maps is listed in the Postfix parent_domain_matches_subdomains configuration setting. Otherwise, specify .domain.tld (note the initial dot) in order to match subdomains. I can replace that Otherwise... sentence by a separate list item. domain.tld Matches domain.tld. The pattern domain.tld also matches subdomains, but only when the string smtpd_access_maps is listed in the Postfix parent_domain_matches_subdomains configuration setting. .domain.tld Matches subdomains of domain.tld, but only when the string smtpd_access_maps is not listed in the Postfix parent_domain_matches_subdomains configuration setting. We can afford the space used by the extra bits. Wietse
Re: Documentation (was: serious bug with check_client_access)
On Thu, Nov 04, 2010 at 10:56:57AM -0400, Wietse Venema wrote: Vincent Lefevre: On 2010-11-04 10:44:34 +0100, lst_ho...@kwsoft.de wrote: The access(5) man page says: domain.tld Matches domain.tld. The pattern domain.tld also matches subdomains, but only when the string smtpd_access_maps is listed in the Postfix parent_domain_matches_subdomains configuration setting. Otherwise, specify .domain.tld (note the initial dot) in order to match subdomains. I can replace that Otherwise... sentence by a separate list item. domain.tld Matches domain.tld. The pattern domain.tld also matches subdomains, but only when the string smtpd_access_maps is listed in the Postfix parent_domain_matches_subdomains configuration setting. .domain.tld Matches subdomains of domain.tld, but only when the string smtpd_access_maps is not listed in the Postfix parent_domain_matches_subdomains configuration setting. I like this. I think it's clearer. I, too, once misinterpreted this passage, and whilst it might not qualify as a postfix-users FAQ, it does come up regularly here. Perhaps some more flesh could be added here as well: parent_domain_matches_subdomains (default: see postconf -d output) What Postfix features match subdomains of domain.tld automatically, instead of requiring an explicit .domain.tld pattern. This is planned backwards compatibility: eventually, all Postfix features are expected to require explicit .domain.tld style patterns when you really want to match subdomains. Such as: Possible strings include debug_peer_list, fast_flush_domains, mynetworks, permit_mx_backup_networks, qmqpd_authorized_clients, relay_domains, and smtpd_access_maps. Note: the leading dot .domain.tld pattern is ignored for features which are listed in parent_domain_matches_subdomains. The string list is taken from my own postconf -d, are there others which might be used in some cases? I guess all of those except smtpd_access_maps would be hyperlinked; and maybe that could link to access.5.html. -- Offlist mail to this address is discarded unless /dev/rob0 or not-spam is in Subject: header
Re: Documentation (was: serious bug with check_client_access)
On 2010-11-04 10:28:00 -0500, /dev/rob0 wrote: On Thu, Nov 04, 2010 at 10:56:57AM -0400, Wietse Venema wrote: I can replace that Otherwise... sentence by a separate list item. domain.tld Matches domain.tld. The pattern domain.tld also matches subdomains, but only when the string smtpd_access_maps is listed in the Postfix parent_domain_matches_subdomains configuration setting. .domain.tld Matches subdomains of domain.tld, but only when the string smtpd_access_maps is not listed in the Postfix parent_domain_matches_subdomains configuration setting. I like this. I think it's clearer. I, too, once misinterpreted this passage, and whilst it might not qualify as a postfix-users FAQ, it does come up regularly here. I still think that it's a bit ambiguous, because I was seeing .domain.tld as a subcase of domain.tld (domain being a sequence of allowed characters including dots). Correct me if I'm wrong (I haven't tested and don't know what postfix does internally), but would changing the beginning by domain.tld Matches domain.tld, where domain may contain dots, but must not start with a dot. be what is really intended? -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Re: serious bug with check_client_access
Le 04/11/2010 05:07, Vincent Lefevre a écrit : On 2010-11-03 22:55:59 -0500, Noel Jones wrote: I'm so sorry you lost your twitter post. Actually I might have lost other mail (though this is a bit unlikely) since I was generally using an initial dot. a good idea is to include both dotted and undotted entries: example.comOK .example.comOK unless you have a reason not to do so. if you search the archives for posts I've sent, you'll see that I always include both. also, parent_domain_... is deprecated. See http://www.postfix.org/postconf.5.html#parent_domain_matches_subdomains The recommendation is to empty it and use dots explicitely. here: # postconf parent_domain_matches_subdomains parent_domain_matches_subdomains = The access map format you're looking for is twitter.com OK Thanks for the information. I've corrected the whole access file. BTW, so, there is no way to match only subdomains (by that, I mean all possible subdomains, but not the domain itself) without changing parent_domain_matches_subdomains? see above: it is recommended not to rely on parent_domain_... otherwise, you can do whatever you want with pcre: /\.example\.com$/OK or with sql or ldap. I don't currently need such a feature, but I'm asking just in case...
THREAD KILLED: Documentation (was: serious bug with check_client_access)
On Thu, Nov 04, 2010 at 05:02:25PM +0100, Vincent Lefevre wrote: I still think that it's a bit ambiguous, because I was seeing .domain.tld as a subcase of domain.tld This objection is spurious, and constitutes trolling. Please do not feed the trolls. For the record, elementary logic: If there are two cases: domain.tld and .domain.tld and the documentation is not deliberately obfuscated, then the two cases are distinct. This is clear enough. The current informal style is more readable. There is no need for a BNF grammar. However, there is no point in continuing this thread. The OP has consumed all the bandwidth he deserves and more. -- Viktor.
Re: serious bug with check_client_access
On 2010-11-04 17:18:17 +0100, mouss wrote: otherwise, you can do whatever you want with pcre: /\.example\.com$/OK or with sql or ldap. For pcre, the man page is not clear. It says: Each pattern is a regular expression that is applied to the entire string being looked up. Depending on the application, that string is an entire client hostname, an entire client IP address, or an entire mail address. But where is it described whether the string is an entire client hostname, an entire client IP address, or an entire mail address? According to your example, the string is an entire client hostname. But then, this means that one cannot match IP addresses. -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Re: serious bug with check_client_access
Vincent Lefevre put forth on 11/4/2010 6:04 PM: On 2010-11-04 17:18:17 +0100, mouss wrote: otherwise, you can do whatever you want with pcre: /\.example\.com$/OK or with sql or ldap. For pcre, the man page is not clear. It says: Each pattern is a regular expression that is applied to the entire string being looked up. Depending on the application, that string is an entire client hostname, an entire client IP address, or an entire mail address. But where is it described whether the string is an entire client hostname, an entire client IP address, or an entire mail address? check_client_access pcre:/etc/postfix/filter.pcre check_sender_access pcre:/etc/postfix/filter.pcre check_recipient_access pcre:/etc/postfix/filter.pcre As you can see, this is defined by the smtpd_foo_restriction you target the PCRE table with. What is checked against the table is dependent on the restriction used. Read the documentation for each check_*_access restriction above at: http://www.postfix.org/postconf.5.html -- Stan
Re: serious bug with check_client_access
On 2010-11-04 19:06:57 -0500, Stan Hoeppner wrote: check_client_access pcre:/etc/postfix/filter.pcre check_sender_access pcre:/etc/postfix/filter.pcre check_recipient_accesspcre:/etc/postfix/filter.pcre As you can see, this is defined by the smtpd_foo_restriction you target the PCRE table with. What is checked against the table is dependent on the restriction used. Read the documentation for each check_*_access restriction above at: http://www.postfix.org/postconf.5.html On this page, it is said: check_client_access type:table Search the specified access database for the client hostname, ^^^ parent domains, client IP address, or networks obtained by ^ stripping least significant octets. See the access(5) manual page for details. And in the access(5) manual page: Depending on the application, that string is an entire client hostname, an entire client IP address, or an entire mail address. So, which string is checked when a pcre table is used with check_client_access? The client hostname or the client IP address? -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Re: serious bug with check_client_access
Vincent Lefevre: On 2010-11-04 19:06:57 -0500, Stan Hoeppner wrote: check_client_access pcre:/etc/postfix/filter.pcre check_sender_access pcre:/etc/postfix/filter.pcre check_recipient_access pcre:/etc/postfix/filter.pcre As you can see, this is defined by the smtpd_foo_restriction you target the PCRE table with. What is checked against the table is dependent on the restriction used. Read the documentation for each check_*_access restriction above at: http://www.postfix.org/postconf.5.html On this page, it is said: check_client_access type:table Search the specified access database for the client hostname, ^^^ parent domains, client IP address, or networks obtained by ^ stripping least significant octets. See the access(5) manual page for details. And in the access(5) manual page: Depending on the application, that string is an entire client hostname, an entire client IP address, or an entire mail address. So, which string is checked when a pcre table is used with check_client_access? The client hostname or the client IP address? check_client_access searches the address and domain with ALL lookup table types. It just doesn't do the substring lookups with PCRE, REGEXP and CIDR. Wietse
Re: serious bug with check_client_access
On 11/05/2010 01:26 AM, Vincent Lefevre wrote: On 2010-11-04 19:06:57 -0500, Stan Hoeppner wrote: check_client_access pcre:/etc/postfix/filter.pcre check_sender_access pcre:/etc/postfix/filter.pcre check_recipient_access pcre:/etc/postfix/filter.pcre As you can see, this is defined by the smtpd_foo_restriction you target the PCRE table with. What is checked against the table is dependent on the restriction used. Read the documentation for each check_*_access restriction above at: http://www.postfix.org/postconf.5.html On this page, it is said: check_client_access type:table Search the specified access database for the client hostname, ^^^ parent domains, client IP address, or networks obtained by ^ stripping least significant octets. See the access(5) manual page for details. And in the access(5) manual page: Depending on the application, that string is an entire client hostname, an entire client IP address, or an entire mail address. So, which string is checked when a pcre table is used with check_client_access? The client hostname or the client IP address? *REGULAR EXPRESSION TABLES* This section describes how the table lookups change when the table is given in the form of regular expressions. For a description of regular expression lookup table syntax, see*regexp_table*(5) http://www.postfix.org/regexp_table.5.html or*pcre_table*(5) http://www.postfix.org/pcre_table.5.html. Each pattern is a regular expression that is applied to the entire string being looked up. Depending on the appli- cation, that string is an entire client hostname, an entire client IP address, or an entire mail address. Thus, no parent domain or parent network search is done, /u...@domain/ mail addresses are not broken up into their /user@/ and/domain/ constituent parts, nor is/user+foo/ broken up into/user/ and/foo/. Patterns are applied in the order as specified in the ta- ble, until a pattern is found that matches the search string. Actions are the same as with indexed file lookups, with the additional feature that parenthesized substrings from the pattern can be interpolated as*$1*,*$2* and so on. I copied the entire section detailing PCRE access matches for you, since you seem unable to find it. How many domain names look like IP addresses to you ? If check_client_access matches against both IPs and hostnames, then your regex table will match against both IPs and hostnames. Also read http://www.postfix.org/pcre_table.5.html for more detail on PCRE maps. -- J.
Re: serious bug with check_client_access
On 2010-11-04 20:33:11 -0400, Wietse Venema wrote: check_client_access searches the address and domain with ALL lookup table types. It just doesn't do the substring lookups with PCRE, REGEXP and CIDR. If I understand correctly, there's another difference: in the default table format, the string to be checked depends on the pattern form (e.g. hostname for domain.tld, IP address for net.work.addr.ess), but for pcre, both strings are checked against all patterns? So, with pcre, if I want to check whether the IP address starts with 1.2.3, I need something like: /^1\.2\.3\.[0-9]+$/ because /^1\.2\.3\./ could also match hostnames (I've noticed in my mail archives that hostnames of this form occur in practice). -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Re: serious bug with check_client_access
On 2010-11-05 01:38:37 +0100, Jeroen Geilman wrote: *REGULAR EXPRESSION TABLES* This section describes how the table lookups change when the table is given in the form of regular expressions. For a description of regular expression lookup table syntax, see*regexp_table*(5) http://www.postfix.org/regexp_table.5.html or*pcre_table*(5) http://www.postfix.org/pcre_table.5.html. Each pattern is a regular expression that is applied to the entire string being looked up. Depending on the appli- cation, that string is an entire client hostname, an entire client IP address, or an entire mail address. Thus, no parent domain or parent network search is done, /u...@domain/ mail addresses are not broken up into their /user@/ and/domain/ constituent parts, nor is/user+foo/ broken up into/user/ and/foo/. Patterns are applied in the order as specified in the ta- ble, until a pattern is found that matches the search string. Actions are the same as with indexed file lookups, with the additional feature that parenthesized substrings from the pattern can be interpolated as*$1*,*$2* and so on. I copied the entire section detailing PCRE access matches for you, since you seem unable to find it. Useless answer. If you had read my message, you would have seen that I quoted from it. How many domain names look like IP addresses to you ? If check_client_access matches against both IPs and hostnames, then your regex table will match against both IPs and hostnames. This is not what the documentation says: Depending on the application, that string is an entire client hostname, an entire client IP address, or an entire mail address. ^^ It is said or, and or doesn't mean both. Quite the opposite. -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Re: serious bug with check_client_access
On 11/05/2010 01:57 AM, Vincent Lefevre wrote: On 2010-11-05 01:38:37 +0100, Jeroen Geilman wrote: *REGULAR EXPRESSION TABLES* This section describes how the table lookups change when the table is given in the form of regular expressions. For a description of regular expression lookup table syntax, see*regexp_table*(5)http://www.postfix.org/regexp_table.5.html or*pcre_table*(5)http://www.postfix.org/pcre_table.5.html. Each pattern is a regular expression that is applied to the entire string being looked up. Depending on the appli- cation, that string is an entire client hostname, an entire client IP address, or an entire mail address. Thus, no parent domain or parent network search is done, /u...@domain/ mail addresses are not broken up into their /user@/ and/domain/ constituent parts, nor is/user+foo/ broken up into/user/ and/foo/. Patterns are applied in the order as specified in the ta- ble, until a pattern is found that matches the search string. Actions are the same as with indexed file lookups, with the additional feature that parenthesized substrings from the pattern can be interpolated as*$1*,*$2* and so on. I copied the entire section detailing PCRE access matches for you, since you seem unable to find it. Useless answer. If you had read my message, you would have seen that I quoted from it. And yet you didn't understand what it says. It bears repeating. How many domain names look like IP addresses to you ? If check_client_access matches against both IPs and hostnames, then your regex table will match against both IPs and hostnames. This is not what the documentation says: Depending on the application, that string is an entire client hostname, an entire client IP address, or an entire mail address. ^^ It is said or, and or doesn't mean both. Quite the opposite. If you combine Each pattern is a regular expression that is applied to the entire string being looked up. with * check_client_access /type:table http://www.postfix.org/DATABASE_README.html/* Search the specified access database for the client hostname, parent domains, client IP address, or networks obtained by stripping least significant octets. See the access(5) http://www.postfix.org/access.5.html manual page for details. the result is as explained. Nowhere in the entire documentation is it mentioned that a regex table will ONLY match a domain OR an IP address. If it's not in the manual, then it's not supported. -- J.
Re: serious bug with check_client_access
On 2010-11-05 02:29:53 +0100, Jeroen Geilman wrote: If you combine Each pattern is a regular expression that is applied to the entire string being looked up. with * check_client_access /type:table http://www.postfix.org/DATABASE_README.html/* Search the specified access database for the client hostname, parent domains, client IP address, or networks obtained by stripping least ^^ significant octets. See the access(5) http://www.postfix.org/access.5.html manual page for details. the result is as explained. It is said or, not and. Nowhere in the entire documentation is it mentioned that a regex table will ONLY match a domain OR an IP address. Read again what you quoted above. -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Re: serious bug with check_client_access
On Thu, Nov 4, 2010 at 10:42 PM, Reinaldo de Carvalho reinal...@gmail.com wrote: check_client_access type:table Search the specified access database for the client hostname, parent domains, client IP address, or networks obtained by stripping least significant octets. See the access(5) manual page for details. This means that not import the table type, each check_client_access entry in restriction, will generate some lookups. This means that not matter the table type. I'll change my tombstone words for you: While not fully understand a documentation, don't try to adapt this documentation to the way you work, but rather yourself to the way the documentation works. -- Reinaldo de Carvalho http://korreio.sf.net http://python-cyrus.sf.net While not fully understand a software, don't try to adapt this software to the way you work, but rather yourself to the way the software works (myself)
Re: serious bug with check_client_access
Vincent Lefevre put forth on 11/4/2010 7:49 PM:
On 2010-11-04 20:33:11 -0400, Wietse Venema wrote:
check_client_access searches the address and domain with ALL lookup
table types. It just doesn't do the substring lookups with PCRE,
REGEXP and CIDR.
If I understand correctly, there's another difference: in the default
table format, the string to be checked depends on the pattern form
(e.g. hostname for domain.tld, IP address for net.work.addr.ess), but
for pcre, both strings are checked against all patterns?
So, with pcre, if I want to check whether the IP address starts with
1.2.3, I need something like:
/^1\.2\.3\.[0-9]+$/
because /^1\.2\.3\./ could also match hostnames (I've noticed in my
mail archives that hostnames of this form occur in practice).
This is why you need to use fully qualified patterns when matching
forward/reverse hostnames. For example:
/^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.dyn\.isp\.net$/ REJECT
In practice, most ISPs don't have a /8 worth of dynamically assigned
addresses, usually a /16 or less. So for a specific ISP dynamic range
it would look my like this:
/^201\.33\.[0-9]{1,3}\.[0-9]{1,3}\.dyn\.isp\.net$/ REJECT
That will match a /16 of rDNS patterns only at the ISP isp.net
--
Stan
Re: serious bug with check_client_access
On 2010-11-04 23:06:17 -0300, Reinaldo de Carvalho wrote: On Thu, Nov 4, 2010 at 10:42 PM, Reinaldo de Carvalho reinal...@gmail.com wrote: check_client_access type:table Search the specified access database for the client hostname, parent domains, client IP address, or networks obtained by stripping least significant octets. See the access(5) manual page for details. This means that not import the table type, each check_client_access entry in restriction, will generate some lookups. This means that not matter the table type. Yes, it will generate *some* lookups, but it doesn't say exactly *which* lookups. That was precisely my question. -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Re: serious bug with check_client_access
On Thu, Nov 4, 2010 at 11:13 PM, Vincent Lefevre vinc...@vinc17.net wrote: On 2010-11-04 23:06:17 -0300, Reinaldo de Carvalho wrote: On Thu, Nov 4, 2010 at 10:42 PM, Reinaldo de Carvalho reinal...@gmail.com wrote: check_client_access type:table Search the specified access database for the client hostname, parent domains, client IP address, or networks obtained by stripping least significant octets. See the access(5) manual page for details. This means that not import the table type, each check_client_access entry in restriction, will generate some lookups. This means that not matter the table type. Yes, it will generate *some* lookups, but it doesn't say exactly *which* lookups. That was precisely my question. - client hostname (reverse dns hostname) - client IP address. if smtpd_access_maps in parent_domain_matches_subdomains. - compare client hostname without the first part at left by dot - compare client hostname without the first and the second part at left by dot (and recursively at the TDL) -- Reinaldo de Carvalho http://korreio.sf.net http://python-cyrus.sf.net While not fully understand a software, don't try to adapt this software to the way you work, but rather yourself to the way the software works (myself)
Re: serious bug with check_client_access
Le 05/11/2010 00:04, Vincent Lefevre a écrit :
On 2010-11-04 17:18:17 +0100, mouss wrote:
otherwise, you can do whatever you want with pcre:
/\.example\.com$/OK
or with sql or ldap.
For pcre, the man page is not clear. It says:
Each pattern is a regular expression that is applied to the entire
string being looked up. Depending on the application, that string is an
entire client hostname, an entire client IP address, or an entire mail
address.
But where is it described whether the string is an entire client
hostname, an entire client IP address, or an entire mail address?
According to your example, the string is an entire client hostname.
But then, this means that one cannot match IP addresses.
You need to read BOTH the doc of the map type AND the doc of what it is
used for (access in this case).
in short, for each map, you have multiple parameters:
- the map type
- the search context (check_client_access, check_sender_acces, ...
transport, virtual_alias_maps, ... etc)
- the list of search keys
for each combination, a search list is derived: for each key, sub-keys
are derived (whether this occurs and how depends on the map type
context).
and it is this search list that you need to grasp. so here is an example.
for check_client_access, the search keys are: the hostname and the IP
(in that order).
the sub-keys depend on the map type, so let's look at a few.
we assume the hostname is lab1.lab2.lab3.example.com and the IP is 1.2.3.4
[hash/cdb/...]
- if parent_domain_matches_subdomains contains smtpd_access: here, the
search list is
S = ( lab1.lab2.lab3.example.com, lab2.lab3.example.com,
lab3.example.com ..., com, 1.2.3.4, 1.2.3, 1.2, 1 )
so postfix will search for each element of this set and stops as soon as
a match is found.
- if parent_domain_matches_subdomains does not contains smtpd_access,
then the search list becomes
S = ( lab1.lab2.lab3.example.com, .lab2.lab3.example.com,
.lab3.example.com ..., .com, 1.2.3.4, 1.2.3, 1.2, 1 )
note the leading dot before lab2, lab3, ...
[pcre/regexp]
with such maps, no subkeys are used. this means the search list is
S = { lab1.lab2.lab3.example.com, 1.2.3.4 }
[cidr]
with cidr, only the IP is meaningful, so the set becomes
S = { 1.2.3.4}
now if we were using check_helo_access, then it's as above except that
there is no IP.
and if we were about check_sender_access, then we only have one key (the
email address) but may have many sub-keys.
serious bug with check_client_access
Hi, It seems that I've found a serious bug in check_client_access (or something is missing in the documentation). A message was blocked with the following in the log: Nov 3 21:16:55 ioooi postfix/smtpd[15423]: NOQUEUE: reject: RCPT from mx003.twitter.com[128.121.146.152]: 554 5.7.1 Service unavailable; Client host [128.121.146.152] blocked using bl.spamcop.net; Blocked - see http://www.spamcop.net/bl.shtml?128.121.146.152; from=twitter-follow-xx=xx@postmaster.twitter.com to=xxx...@xx.xxx proto=ESMTP helo=mx003.twitter.com In my /etc/postfix/main.cf file, I have: smtpd_client_restrictions = permit_mynetworks, check_client_access hash:/etc/postfix/rbl_override, reject_rbl_client bl.spamcop.net, reject_rbl_client sbl.spamhaus.org and /etc/postfix/rbl_override contains in particular: .twitter.com OK /etc/postfix/rbl_override.db had been rebuilt with postmap /etc/postfix/rbl_override. As .twitter.com matches subdomains, it should have matched mx003.twitter.com, and due to the .twitter.com OK, the RBL's shouldn't have been checked. The machine is an Ubuntu 9.04 (jaunty), with the postfix 2.5.5-1.1 package. The machine has been rebooted since the latest config changes. -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Re: serious bug with check_client_access
Vincent Lefevre: As .twitter.com matches subdomains, it should have matched What documentation supports this?
Re: serious bug with check_client_access
On 2010-11-03 22:00:21 -0400, Wietse Venema wrote: Vincent Lefevre: As .twitter.com matches subdomains, it should have matched What documentation supports this? The access(5) man page says: domain.tld Matches domain.tld. The pattern domain.tld also matches subdomains, but only when the string smtpd_access_maps is listed in the Postfix parent_domain_matches_subdomains configuration setting. Otherwise, specify .domain.tld (note the initial dot) in order to match subdomains. See the last sentence. -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Re: serious bug with check_client_access
On Thu, Nov 04, 2010 at 03:08:03AM +0100, Vincent Lefevre wrote: On 2010-11-03 22:00:21 -0400, Wietse Venema wrote: Vincent Lefevre: As .twitter.com matches subdomains, it should have matched What documentation supports this? The access(5) man page says: domain.tld Matches domain.tld. The pattern domain.tld also matches subdomains, but only when the string smtpd_access_maps is listed in the Postfix parent_domain_matches_subdomains configuration setting. Otherwise, specify .domain.tld (note the initial dot) in order to match subdomains. See the last sentence. See the one before that. :) What is your setting of parent_domain_matches_subdomains? Repeat after me: Oops, sorry Wietse. :) -- Offlist mail to this address is discarded unless /dev/rob0 or not-spam is in Subject: header
Re: serious bug with check_client_access
On 2010-11-03 21:21:24 -0500, /dev/rob0 wrote: On Thu, Nov 04, 2010 at 03:08:03AM +0100, Vincent Lefevre wrote: On 2010-11-03 22:00:21 -0400, Wietse Venema wrote: Vincent Lefevre: As .twitter.com matches subdomains, it should have matched What documentation supports this? The access(5) man page says: domain.tld Matches domain.tld. The pattern domain.tld also matches subdomains, but only when the string smtpd_access_maps is listed in the Postfix parent_domain_matches_subdomains configuration setting. Otherwise, specify .domain.tld (note the initial dot) in order to match subdomains. See the last sentence. See the one before that. :) What is your setting of parent_domain_matches_subdomains? You should read my message again. The setting of parent_domain_matches_subdomains matters only for domain.tld. But my config file has the form .domain.tld. -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Re: serious bug with check_client_access
On 11/3/2010 9:36 PM, Vincent Lefevre wrote: On 2010-11-03 21:21:24 -0500, /dev/rob0 wrote: On Thu, Nov 04, 2010 at 03:08:03AM +0100, Vincent Lefevre wrote: On 2010-11-03 22:00:21 -0400, Wietse Venema wrote: Vincent Lefevre: As .twitter.com matches subdomains, it should have matched What documentation supports this? The access(5) man page says: domain.tld Matches domain.tld. The pattern domain.tld also matches subdomains, but only when the string smtpd_access_maps is listed in the Postfix parent_domain_matches_subdomains configuration setting. Otherwise, specify .domain.tld (note the initial dot) in order to match subdomains. See the last sentence. See the one before that. :) What is your setting of parent_domain_matches_subdomains? You should read my message again. The setting of parent_domain_matches_subdomains matters only for domain.tld. But my config file has the form .domain.tld. Your interpretation is incorrect. .domain.tld only works if parent_domain_matches_subdomains does NOT include smtpd_access maps.
Re: serious bug with check_client_access
On Thu, Nov 04, 2010 at 03:36:30AM +0100, Vincent Lefevre wrote: On 2010-11-03 21:21:24 -0500, /dev/rob0 wrote: On Thu, Nov 04, 2010 at 03:08:03AM +0100, Vincent Lefevre wrote: On 2010-11-03 22:00:21 -0400, Wietse Venema wrote: Vincent Lefevre: As .twitter.com matches subdomains, it should have matched What documentation supports this? The access(5) man page says: domain.tld Matches domain.tld. The pattern domain.tld also matches subdomains, but only The pattern domain.tld also matches subdomains ... You don't have that in your map. when the string smtpd_access_maps is listed in the Postfix parent_domain_matches_subdomains configuration setting. You failed to provide postconf -n in the OP, and again failed to answer my question about it. Otherwise, specify .domain.tld (note the initial dot) in ^ order to match subdomains. You're not in the otherwise category, I bet. See the last sentence. See the one before that. :) What is your setting of parent_domain_matches_subdomains? You should read my message again. The setting of parent_domain_matches_subdomains matters only for domain.tld. But my config file has the form .domain.tld. http://www.postfix.org/postconf.5.html#parent_domain_matches_subdomains -- Offlist mail to this address is discarded unless /dev/rob0 or not-spam is in Subject: header
Re: serious bug with check_client_access
On Thu, 2010-11-04 at 03:36:30 +0100, Vincent Lefevre wrote: On 2010-11-03 21:21:24 -0500, /dev/rob0 wrote: On Thu, Nov 04, 2010 at 03:08:03AM +0100, Vincent Lefevre wrote: On 2010-11-03 22:00:21 -0400, Wietse Venema wrote: Vincent Lefevre: As .twitter.com matches subdomains, it should have matched What documentation supports this? The access(5) man page says: domain.tld Matches domain.tld. The pattern domain.tld also matches subdomains, but only when the string smtpd_access_maps is listed in the Postfix parent_domain_matches_subdomains configuration setting. Otherwise, specify .domain.tld (note the initial dot) in order to match subdomains. See the last sentence. See the one before that. :) What is your setting of parent_domain_matches_subdomains? You should read my message again. The setting of parent_domain_matches_subdomains matters only for domain.tld. But my config file has the form .domain.tld. No, *you* should re-read the documentation. -- Sahil Tandon sa...@freebsd.org
Re: serious bug with check_client_access
On 2010-11-03 21:44:00 -0500, /dev/rob0 wrote: On Thu, Nov 04, 2010 at 03:36:30AM +0100, Vincent Lefevre wrote: On 2010-11-03 21:21:24 -0500, /dev/rob0 wrote: On Thu, Nov 04, 2010 at 03:08:03AM +0100, Vincent Lefevre wrote: On 2010-11-03 22:00:21 -0400, Wietse Venema wrote: Vincent Lefevre: As .twitter.com matches subdomains, it should have matched What documentation supports this? The access(5) man page says: domain.tld Matches domain.tld. The pattern domain.tld also matches subdomains, but only The pattern domain.tld also matches subdomains ... You don't have that in your map. when the string smtpd_access_maps is listed in the Postfix parent_domain_matches_subdomains configuration setting. You failed to provide postconf -n in the OP, and again failed to answer my question about it. Otherwise, specify .domain.tld (note the initial dot) in ^ order to match subdomains. You're not in the otherwise category, I bet. So what? There's some logic error. Otherwise is a replacement for if smtpd_access_maps is not listed in parent_domain_matches_subdomains but nowhere it is said that domain.tld is not supported if this pattern starts with a dot. -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Re: serious bug with check_client_access
On 2010-11-03 21:40:54 -0500, Noel Jones wrote: .domain.tld only works if parent_domain_matches_subdomains does NOT include smtpd_access maps. The man page says nothing like that. So, the documentation should be fixed. -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Re: serious bug with check_client_access
On 11/3/2010 10:00 PM, Vincent Lefevre wrote: On 2010-11-03 21:40:54 -0500, Noel Jones wrote: .domain.tld only works if parent_domain_matches_subdomains does NOT include smtpd_access maps. The man page says nothing like that. So, the documentation should be fixed. The vast majority of readers seem to interpret that section as intended. You're welcome to post a documentation patch in a new thread, but I don't think the behavior or its documentation has changed in ~10 years, so don't hold your breath. And next time behavior doesn't match your expectations, you might get more sympathy if your message starts with please clarify this for me rather than serious bug. -- Noel Jones
Re: serious bug with check_client_access
On 2010-11-03 22:16:48 -0500, Noel Jones wrote: On 11/3/2010 10:00 PM, Vincent Lefevre wrote: On 2010-11-03 21:40:54 -0500, Noel Jones wrote: .domain.tld only works if parent_domain_matches_subdomains does NOT include smtpd_access maps. The man page says nothing like that. So, the documentation should be fixed. The vast majority of readers seem to interpret that section as intended. I'd say that the vast majority of readers probably haven't tried .domain.tld (with an initial dot) or haven't even read the documentation, so that they didn't have to interpret the sentence on this point. And many people interpret If A then B as also meaning that If not A then not B (against mathematical logic), and this doesn't mean that they are correct. Also, perhaps other users use this form and don't know that it doesn't work, because AFAIK, one doesn't get any feedback (I haven't seen any warning) until the sender says that some mail has been rejected by some later rule. You're welcome to post a documentation patch in a new thread, but I don't think the behavior or its documentation has changed in ~10 years, so don't hold your breath. Something difficult for users who don't know postfix internals. What happens if the .domain.tld form is used? Is it simply ignored? Or does anything else occur? And next time behavior doesn't match your expectations, you might get more sympathy if your message starts with please clarify this for me rather than serious bug. Actually if a documentation is incorrect/incomplete, it is a bug in the documentation. And FYI, the consequence was a lost mail. So, this is quite serious. -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Re: serious bug with check_client_access
On 11/3/2010 10:50 PM, Vincent Lefevre wrote: Actually if a documentation is incorrect/incomplete, it is a bug in the documentation. And FYI, the consequence was a lost mail. So, this is quite serious. I'm so sorry you lost your twitter post. The access map format you're looking for is twitter.com OK
Re: serious bug with check_client_access
On 2010-11-03 22:55:59 -0500, Noel Jones wrote: I'm so sorry you lost your twitter post. Actually I might have lost other mail (though this is a bit unlikely) since I was generally using an initial dot. The access map format you're looking for is twitter.com OK Thanks for the information. I've corrected the whole access file. BTW, so, there is no way to match only subdomains (by that, I mean all possible subdomains, but not the domain itself) without changing parent_domain_matches_subdomains? I don't currently need such a feature, but I'm asking just in case... -- Vincent Lefèvre vinc...@vinc17.net - Web: http://www.vinc17.net/ 100% accessible validated (X)HTML - Blog: http://www.vinc17.net/blog/ Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
Re: serious bug with check_client_access
On 11/3/2010 11:07 PM, Vincent Lefevre wrote: BTW, so, there is no way to match only subdomains (by that, I mean all possible subdomains, but not the domain itself) without changing parent_domain_matches_subdomains? That's correct with indexed tables. With regexp or pcre tables there is no automatic subdomain search; you control the scope of the search with your expression. To answer your other question, when parent_domain_matches_subdomains includes smtpd_access_maps (the default), the form .domain.tld is never searched for. As a result, the entry is silently ignored. -- Noel Jones
