Re: check client ip only if mail has a particular header
On 12/03/2011 17:15, Stan Hoeppner wrote: Wietse Venema put forth on 3/12/2011 8:43 AM: Stan Hoeppner: It appears I did understand Giovanni's need correctly. He should be able to use Sahil's checkdbl.pl daemon with some modifications. He'd simply check that X-custom-header exists. If it doesn't, Wietse: header_checks can't detect missing headers. Stan Hoeppner: Postfix is table driven, meaning it has table-driven mechanisms such as header_checks or aliases, and it has table lookup mechanisms such as hash and pcre. Ok, I think I found my error now that you reminded me exactly how header_checks works. :( This is possible because table lookup is based on a simple (key, value) interface, and because the same interface can be used with all table mechanisms: hash, btree, pcre, cidr, tcp, ldap, *sql. - The key is the search string. This key either used as is with hash, btree, pcre, cidr, tcp; or it is embedded in some blob as with ldap, *sql, but that happens under the universal (key, value) interface level, and is invisible for table-driven mechanisms header_checks or aliases. - The value is the result or an error (not found, database error). So yes, you can implement counters in the code that receives the query, but there exists no query that will retrieve that counter, or that will reset it. Again, Postfix table driven mechanisms must use the same table lookup interface regardless of the underlying table implementation, or else the whole thing is worthless. Yep, I found my error. (slouches in chair) I had confused the SMTP access policy delegation protocol sending a blank line to signal the end of the request with header_checks behavior when using a TCP server. I was using Sahil's header_checks TCP server daemon as a reference, and thinking of how relatively easy it should be to adapt it, but I had the policy protocol in mind for passing the data, instead of header_checks. I guess I was suffering mind lock, thinking of how Giovanni's need could be addressed by an external daemon, and lost track of which Postfix mechanism needed to be used to pass the relevant information to it. Therefore, the only solution is to use a miltet, right? Do you know a simple milter (write in perl or python) that i can use as example for develop my milter?
Re: check client ip only if mail has a particular header
Giovanni Mancuso put forth on 3/15/2011 10:59 AM: Therefore, the only solution is to use a miltet, right? It's looking that way. Do you know a simple milter (write in perl or python) that i can use as example for develop my milter? I do not. You're apparently in uncharted territory. It seems no one else has done something like this. That may be a clue that you should reevaluate the methods you choose to achieve your goal. IIRC, you never stated your actual goal, only one method you wish to use to achieve it, i.e. header_check + dnsbl lookup. That's a method, not a goal. If you tell us the actual goal we may have another method that will allow you to achieve it. If you're simply trying to block spam, there are much better and more widely used methods. -- Stan
Re: check client ip only if mail has a particular header
On 3/15/2011 7:46 PM, Stan Hoeppner wrote: Giovanni Mancuso put forth on 3/15/2011 10:59 AM: Therefore, the only solution is to use a miltet, right? It's looking that way. Do you know a simple milter (write in perl or python) that i can use as example for develop my milter? I do not. You're apparently in uncharted territory. It seems no one else has done something like this. That may be a clue that you should reevaluate the methods you choose to achieve your goal. IIRC, you never stated your actual goal, only one method you wish to use to achieve it, i.e. header_check + dnsbl lookup. That's a method, not a goal. If you tell us the actual goal we may have another method that will allow you to achieve it. If you're simply trying to block spam, there are much better and more widely used methods. Probably a milter implementing SpamAssassin would be the easiest way. Add a negative scoring custom rule matching the magic header, and maybe bump up the score of some of the SA built-in RBL checks. -- Noel Jones
Re: check client ip only if mail has a particular header
Wietse Venema put forth on 3/11/2011 8:57 PM: Stan Hoeppner: It appears I did understand Giovanni's need correctly. He should be able to use Sahil's checkdbl.pl daemon with some modifications. He'd simply check that X-custom-header exists. If it doesn't, header_checks can't detect missing headers. Wietse The headers are sent to the TCP server daemon one line at a time, IIRC. Is it not possible to simply code the daemon to a set variable, say our_custom_header=0, and increment it if the header exists? After processing all headers, if our_custom_header=0, then haven't we detected the absence of the header? -- Stan
Re: check client ip only if mail has a particular header
On 3/12/2011 7:14 AM, Stan Hoeppner wrote: Wietse Venema put forth on 3/11/2011 8:57 PM: Stan Hoeppner: It appears I did understand Giovanni's need correctly. He should be able to use Sahil's checkdbl.pl daemon with some modifications. He'd simply check that X-custom-header exists. If it doesn't, header_checks can't detect missing headers. Wietse The headers are sent to the TCP server daemon one line at a time, IIRC. Is it not possible to simply code the daemon to a set variable, say our_custom_header=0, and increment it if the header exists? After processing all headers, if our_custom_header=0, then haven't we detected the absence of the header? The data the TCP map receives is: GET [space] one header [newline] after which the map can send a reply. Some problems you'll need to address: How do you know which message a header belongs to? How do you know when you've seen all the headers for a particular message? How does a map send a reply when no request is made? -- Noel Jones
Re: check client ip only if mail has a particular header
Stan Hoeppner: It appears I did understand Giovanni's need correctly. He should be able to use Sahil's checkdbl.pl daemon with some modifications. He'd simply check that X-custom-header exists. If it doesn't, Wietse: header_checks can't detect missing headers. Stan Hoeppner: The headers are sent to the TCP server daemon one line at a time, IIRC. Is it not possible to simply code the daemon to a set variable, say our_custom_header=0, and increment it if the header exists? After processing all headers, if our_custom_header=0, then haven't we detected the absence of the header? header_checks sends one query per header. header_checks sends no query for non-headers including the body text that follows after the last header. There are two levels of abstraction in Postfix table lookups that you appear to overlook. Postfix is table driven, meaning it has table-driven mechanisms such as header_checks or aliases, and it has table lookup mechanisms such as hash and pcre. This is possible because table lookup is based on a simple (key, value) interface, and because the same interface can be used with all table mechanisms: hash, btree, pcre, cidr, tcp, ldap, *sql. - The key is the search string. This key either used as is with hash, btree, pcre, cidr, tcp; or it is embedded in some blob as with ldap, *sql, but that happens under the universal (key, value) interface level, and is invisible for table-driven mechanisms header_checks or aliases. - The value is the result or an error (not found, database error). So yes, you can implement counters in the code that receives the query, but there exists no query that will retrieve that counter, or that will reset it. Again, Postfix table driven mechanisms must use the same table lookup interface regardless of the underlying table implementation, or else the whole thing is worthless. Wietse
Re: check client ip only if mail has a particular header
Wietse Venema put forth on 3/12/2011 8:43 AM: Stan Hoeppner: It appears I did understand Giovanni's need correctly. He should be able to use Sahil's checkdbl.pl daemon with some modifications. He'd simply check that X-custom-header exists. If it doesn't, Wietse: header_checks can't detect missing headers. Stan Hoeppner: Postfix is table driven, meaning it has table-driven mechanisms such as header_checks or aliases, and it has table lookup mechanisms such as hash and pcre. Ok, I think I found my error now that you reminded me exactly how header_checks works. :( This is possible because table lookup is based on a simple (key, value) interface, and because the same interface can be used with all table mechanisms: hash, btree, pcre, cidr, tcp, ldap, *sql. - The key is the search string. This key either used as is with hash, btree, pcre, cidr, tcp; or it is embedded in some blob as with ldap, *sql, but that happens under the universal (key, value) interface level, and is invisible for table-driven mechanisms header_checks or aliases. - The value is the result or an error (not found, database error). So yes, you can implement counters in the code that receives the query, but there exists no query that will retrieve that counter, or that will reset it. Again, Postfix table driven mechanisms must use the same table lookup interface regardless of the underlying table implementation, or else the whole thing is worthless. Yep, I found my error. (slouches in chair) I had confused the SMTP access policy delegation protocol sending a blank line to signal the end of the request with header_checks behavior when using a TCP server. I was using Sahil's header_checks TCP server daemon as a reference, and thinking of how relatively easy it should be to adapt it, but I had the policy protocol in mind for passing the data, instead of header_checks. I guess I was suffering mind lock, thinking of how Giovanni's need could be addressed by an external daemon, and lost track of which Postfix mechanism needed to be used to pass the relevant information to it. -- Stan
Re: check client ip only if mail has a particular header
On 10/03/2011 21:39, Stan Hoeppner wrote: Giovanni Mancuso put forth on 3/10/2011 11:19 AM: On 10/03/2011 15:04, Stan Hoeppner wrote: Giovanni Mancuso put forth on 3/10/2011 7:22 AM: Hi, I would configure my postfix to control if client ip is in blacklist only if the mail have a particolar header. I tried to create a header_checks table and with a FILTER i redirect all mails that have a particular header to localhost on port 2500 but in this way the client ip is localhost Anyone have any ideas? You don't want to forward the entire mail to a policy daemon or TCP server via FILTER. You simply want to reject it if you don't like the header contents and the IP is in a dnsbl. The closest thing I know of to this is Sahil Tandon's perl based TCP server. It checks the domains in the from, message-id, and reply-to headers against a configurable list of domain block lists. The default list is: I saw the script, but in this way the check is done with header from, message-id or reply-to that can be a fake. I would that the check is done on* *client_address. I must have misunderstood. It sounds above as if you want to check the header for something, and only after some kind of match query a DNSBL with the client IP. Is this not what you want? What exactly do you want? It's not clear. Hi, I try to explain with an example. I have my postfix configured as MX record of my domain with particular smtpd_recipents_restriction, ecc... If my postfix receives a mail that has X-MyCustomHeader, Postfix accept the mail and delivers it to backend. If my Postfix receives a mail that hasn't X-MyCustomHeader, Postfix checks if client ip is in a blacklist and reject the email if the ip is in blacklist or accept the mail if it isn't in blacklist. I hope to have explained it clearly. Sorry for my bad English Bye
Re: check client ip only if mail has a particular header
On Fri, Mar 11, 2011 at 04:58:15PM +0100, Giovanni Mancuso wrote: I have my postfix configured as MX record of my domain with particular smtpd_recipents_restriction, ecc... If my postfix receives a mail that has X-MyCustomHeader, Postfix accept the mail and delivers it to backend. If my Postfix receives a mail that hasn't X-MyCustomHeader, Postfix checks if client ip is in a blacklist and reject the email if the ip is in blacklist or accept the mail if it isn't in blacklist. Rejection of email can only happen before the mail is queued. Therefore, this can only be done with a proxy-filter or milter. http://www.postfix.org/SMTPD_PROXY_README.html http://www.postfix.org/MILTER_README.html While client addresses can checked via built-in mechanisms in smtpd_end_of_data_restrictions, there is no built-in mechanism to make these checks dependent on the message headers. Adding such a feature would not be easy, the header checks are performed in the cleanup(8) server, while client address checks are performed in the smtpd(8) server, the data flow from smtpd(8) to cleanup(8) is essentially one-way... http://www.postfix.org/OVERVIEW.html -- Viktor.
Re: check client ip only if mail has a particular header
On 11/03/2011 17:09, Victor Duchovni wrote: On Fri, Mar 11, 2011 at 04:58:15PM +0100, Giovanni Mancuso wrote: I have my postfix configured as MX record of my domain with particular smtpd_recipents_restriction, ecc... If my postfix receives a mail that has X-MyCustomHeader, Postfix accept the mail and delivers it to backend. If my Postfix receives a mail that hasn't X-MyCustomHeader, Postfix checks if client ip is in a blacklist and reject the email if the ip is in blacklist or accept the mail if it isn't in blacklist. Rejection of email can only happen before the mail is queued. Therefore, this can only be done with a proxy-filter or milter. http://www.postfix.org/SMTPD_PROXY_README.html http://www.postfix.org/MILTER_README.html Another questions. If i use a MILTER, and if i have smtpd_recipient_restrictions configured, postfix uses this restriction? I did not understand this :-( In your opinion, to do this is better MILTER or SMTPD_PROXY? Do you know a simple milter (write in perl or python) that i can use as example for develop my milter? Thanks While client addresses can checked via built-in mechanisms in smtpd_end_of_data_restrictions, there is no built-in mechanism to make these checks dependent on the message headers. Adding such a feature would not be easy, the header checks are performed in the cleanup(8) server, while client address checks are performed in the smtpd(8) server, the data flow from smtpd(8) to cleanup(8) is essentially one-way... http://www.postfix.org/OVERVIEW.html
Re: check client ip only if mail has a particular header
Victor Duchovni put forth on 3/11/2011 10:09 AM: On Fri, Mar 11, 2011 at 04:58:15PM +0100, Giovanni Mancuso wrote: I have my postfix configured as MX record of my domain with particular smtpd_recipents_restriction, ecc... If my postfix receives a mail that has X-MyCustomHeader, Postfix accept the mail and delivers it to backend. If my Postfix receives a mail that hasn't X-MyCustomHeader, Postfix checks if client ip is in a blacklist and reject the email if the ip is in blacklist or accept the mail if it isn't in blacklist. Rejection of email can only happen before the mail is queued. Therefore, this can only be done with a proxy-filter or milter. http://www.postfix.org/SMTPD_PROXY_README.html http://www.postfix.org/MILTER_README.html While client addresses can checked via built-in mechanisms in smtpd_end_of_data_restrictions, there is no built-in mechanism to make these checks dependent on the message headers. Adding such a feature would not be easy, the header checks are performed in the cleanup(8) server, while client address checks are performed in the smtpd(8) server, the data flow from smtpd(8) to cleanup(8) is essentially one-way... http://www.postfix.org/OVERVIEW.html It appears I did understand Giovanni's need correctly. He should be able to use Sahil's checkdbl.pl daemon with some modifications. He'd simply check that X-custom-header exists. If it doesn't, he'd then query the configured dnsbl for the client IP address, although I'm not sure exactly how one would access the client IP from within header checks. I'm not a perl programmer, but this shouldn't be too difficult for a perl guy, assuming the client IP address is available to the daemon one way or another. Most of the code the OP needs is already here: http://people.freebsd.org/~sahil/scripts/checkdbl.pl.txt -- Stan
Re: check client ip only if mail has a particular header
Stan Hoeppner: It appears I did understand Giovanni's need correctly. He should be able to use Sahil's checkdbl.pl daemon with some modifications. He'd simply check that X-custom-header exists. If it doesn't, header_checks can't detect missing headers. Wietse
Re: check client ip only if mail has a particular header
On 3/10/2011 7:22 AM, Giovanni Mancuso wrote: Hi, I would configure my postfix to control if client ip is in blacklist only if the mail have a particolar header. I tried to create a header_checks table and with a FILTER i redirect all mails that have a particular header to localhost on port 2500 but in this way the client ip is localhost Anyone have any ideas? This will never work. Postfix performs dnsbl checks on the connecting client, not on headers. When postfix does dnsbl checks, the headers have not been received yet. Use a milter or smtpd_proxy_filter for complex tests such as this, maybe in combination with spamassassin. -- Noel Jones
Re: check client ip only if mail has a particular header
Giovanni Mancuso put forth on 3/10/2011 7:22 AM: Hi, I would configure my postfix to control if client ip is in blacklist only if the mail have a particolar header. I tried to create a header_checks table and with a FILTER i redirect all mails that have a particular header to localhost on port 2500 but in this way the client ip is localhost Anyone have any ideas? You don't want to forward the entire mail to a policy daemon or TCP server via FILTER. You simply want to reject it if you don't like the header contents and the IP is in a dnsbl. The closest thing I know of to this is Sahil Tandon's perl based TCP server. It checks the domains in the from, message-id, and reply-to headers against a configurable list of domain block lists. The default list is: dbl.spamhaus.org multi.surbl.org black.uribl.com The TCP server returns REJECT if the domain is listed. Not sure if you're trying to accomplish the same thing or just similar, but you can get it here: http://people.freebsd.org/~sahil/scripts/checkdbl.pl.txt If you know perl you should easily be able to modify this to do what you want, as it's so similar to your requirement. You'll do this in header checks as it won't work via FILTER. Usage instructions are in the top of the file. -- Stan
Re: check client ip only if mail has a particular header
On 10/03/2011 15:04, Stan Hoeppner wrote: Giovanni Mancuso put forth on 3/10/2011 7:22 AM: Hi, I would configure my postfix to control if client ip is in blacklist only if the mail have a particolar header. I tried to create a header_checks table and with a FILTER i redirect all mails that have a particular header to localhost on port 2500 but in this way the client ip is localhost Anyone have any ideas? You don't want to forward the entire mail to a policy daemon or TCP server via FILTER. You simply want to reject it if you don't like the header contents and the IP is in a dnsbl. The closest thing I know of to this is Sahil Tandon's perl based TCP server. It checks the domains in the from, message-id, and reply-to headers against a configurable list of domain block lists. The default list is: I saw the script, but in this way the check is done with header from, message-id or reply-to that can be a fake. I would that the check is done on* *client_address. I'm watching smtpd_milters and i found a software mimedefang. Do you know? Otherwise, Is there another possible solution? Thanks dbl.spamhaus.org multi.surbl.org black.uribl.com The TCP server returns REJECT if the domain is listed. Not sure if you're trying to accomplish the same thing or just similar, but you can get it here: http://people.freebsd.org/~sahil/scripts/checkdbl.pl.txt If you know perl you should easily be able to modify this to do what you want, as it's so similar to your requirement. You'll do this in header checks as it won't work via FILTER. Usage instructions are in the top of the file.
Re: check client ip only if mail has a particular header
Giovanni Mancuso put forth on 3/10/2011 11:19 AM: On 10/03/2011 15:04, Stan Hoeppner wrote: Giovanni Mancuso put forth on 3/10/2011 7:22 AM: Hi, I would configure my postfix to control if client ip is in blacklist only if the mail have a particolar header. I tried to create a header_checks table and with a FILTER i redirect all mails that have a particular header to localhost on port 2500 but in this way the client ip is localhost Anyone have any ideas? You don't want to forward the entire mail to a policy daemon or TCP server via FILTER. You simply want to reject it if you don't like the header contents and the IP is in a dnsbl. The closest thing I know of to this is Sahil Tandon's perl based TCP server. It checks the domains in the from, message-id, and reply-to headers against a configurable list of domain block lists. The default list is: I saw the script, but in this way the check is done with header from, message-id or reply-to that can be a fake. I would that the check is done on* *client_address. I must have misunderstood. It sounds above as if you want to check the header for something, and only after some kind of match query a DNSBL with the client IP. Is this not what you want? What exactly do you want? It's not clear. -- Stan
