Re: request improved logging for postfix.
On 21 Dec 2016, at 5:42, L.P.H. van Belle wrote: Hello Noel, Would you please stop say that im labeling.. im not. Sorry im so bad in explaining things in english. I just trying to explain something based on what i did read here: http://www.postfix.org/postconf.5.html#reject_unknown_helo_hostname reject_unknown_helo_hostname (with Postfix < 2.3: reject_unknown_hostname) Reject the request when the HELO or EHLO hostname has no DNS A or MX record. Here the "POSTFIX MANUAL" stats > "HELO or EHLO hostname" << So I think we misunderstand one eachother. I know a "helo hostname" is just a name with refers to a A, or MX record and the MX must reffer to any A or . I know its not client-hostname or helo-hostname. Its "helo " and maybe that should be better in the manual. As long as its has a DNS A or MX record. ( as stated by RFC 5321 2.3.5 ) Postfix mostly ignores the helo name. You should too. Why? Since in my opionion this is a very bad advice. It is good advice, if you want a robust mail system. A mail system which requires all clients to follow every MUST and MUST NOT in any relevant RFC is *NOT ROBUST* it is *FRAGILE*. This is especially true if that server fails to be as strict in its own behavior. This is my I enforce correct "HELO or EHLO hostname". And its as the postfix manual stats by : Rejecting the request when the HELO or EHLO hostname has no DNS A or MX record. Exacty what i need. I think you are confusing "need" and "wish." Using Postfix's "reject_unknown_helo_hostname" directive will result in rejection of mail from senders whose only problem is how their outbound mail server introduces itself. A significant subset of Microsoft's Office365 outbound mail servers have used EHLO/HELO names that do not resolve for many months at a stretch (and may still be doing so.) A significant minority of small and medium sized businesses who run their own mail systems (they still exist!) use unqualified or unresolvable hostnames on those systems. A rough analysis of some of my recent logs shows that while >99% of mail offered by machines that use unresolvable HELO names are spam, I only can know that because that's how much of such mail is caught by other Postfix restrictions and associated tools. Much of the mail which my systems would have rejected with reject_unknown_helo_hostname is definitely NOT spam, as it is the remote half of conversations in which my users are active participants. The missed-spam reports of the past month from my users include nothing with an non-resolving HELO name. In short: reject_unknown_helo_hostname on the systems I help manage would not improve spam exclusion and would cause substantial rejection of mail which users want and expect to have delivered reliably. Your mail flow may be different, however I suspect that you haven't looked closely enough at it to know. rfc5321 section 2.3.5 stats: The domain name, as described in this document and in RFC 1035 [2], is the entire, fully-qualified name (often referred to as an "FQDN"). A domain name that is not in FQDN form is no more than a local alias. Local aliases MUST NOT appear in any SMTP transaction. Only resolvable, fully-qualified domain names (FQDNs) are permitted when domain names are used in SMTP. In other words, names that can be resolved to MX RRs or address (i.e., A or ) RRs (as discussed in Section 5) are permitted, as are CNAME RRs whose targets can be resolved, in turn, to MX or address RRs. Local nicknames or unqualified names MUST NOT be used. See also section 4.1.4: An SMTP server MAY verify that the domain name argument in the EHLO command actually corresponds to the IP address of the client. However, if the verification fails, the server MUST NOT refuse to accept a message on that basis. Information captured in the verification attempt is for logging and tracing purposes. One can parse that very narrowly to allow reject_unknown_helo_hostname, which does not require a *correct* resolution of the HELO name, just any resolution. I think that degree of RFC-lawyering misses the point: HELO names do not have and never have had a concrete functional role in SMTP, which has resulted in endemic carelessness in making that name correct. There ARE things you can do with the HELO name to reliably detect illicit mail, but the only one that is a simple "set and forget" in Postfix is reject_invalid_helo_hostname. That requires the name to fit a much looser definition: essentially that it could be a resolvable hostname somewhere or is a logically valid IP literal. Everything else you can do with Postfix require more careful thought and attention to the mail you actually get.
Re: request improved logging for postfix.
On 12/21/2016 5:42 AM, L.P.H. van Belle wrote: Hello Noel, Would you please stop say that im labeling.. im not. Noel n'a pas dit que vous êtes "labeling" quelque chose. Il a dit quand Postfix marque ("labels") une addresse IP comme "unknown". Le mot anglais "label" n'est pas toujours une accusation. Sorry im so bad in explaining things in english. Et moi aussi à francais. (Or, pardon my French, as we say in English.) -- Larry Kuenning la...@qhpress.org
RE: request improved logging for postfix.
Hello Noel, Would you please stop say that im labeling.. im not. Sorry im so bad in explaining things in english. I just trying to explain something based on what i did read here: http://www.postfix.org/postconf.5.html#reject_unknown_helo_hostname reject_unknown_helo_hostname (with Postfix < 2.3: reject_unknown_hostname) Reject the request when the HELO or EHLO hostname has no DNS A or MX record. Here the "POSTFIX MANUAL" stats >>>>> "HELO or EHLO hostname" <<<<<< So I think we misunderstand one eachother. I know a "helo hostname" is just a name with refers to a A, or MX record and the MX must reffer to any A or . I know its not client-hostname or helo-hostname. Its "helo " and maybe that should be better in the manual. As long as its has a DNS A or MX record. ( as stated by RFC 5321 2.3.5 ) > Postfix mostly ignores the helo name. You should too. Why? Since in my opionion this is a very bad advice. This is my I enforce correct "HELO or EHLO hostname". And its as the postfix manual stats by : Rejecting the request when the HELO or EHLO hostname has no DNS A or MX record. Exacty what i need. rfc5321 section 2.3.5 stats: The domain name, as described in this document and in RFC 1035 [2], is the entire, fully-qualified name (often referred to as an "FQDN"). A domain name that is not in FQDN form is no more than a local alias. Local aliases MUST NOT appear in any SMTP transaction. Only resolvable, fully-qualified domain names (FQDNs) are permitted when domain names are used in SMTP. In other words, names that can be resolved to MX RRs or address (i.e., A or ) RRs (as discussed in Section 5) are permitted, as are CNAME RRs whose targets can be resolved, in turn, to MX or address RRs. Local nicknames or unqualified names MUST NOT be used. Now i just was not happy with some logging parts, but you explained all and for me its ok. I know what todo now to make things better in my logs for my colleges So they can take over some things when im on holiday. Thanks all for the replies. And sorry the the badly choosen words and misunderstandings. Best regards, Louis > -Oorspronkelijk bericht- > Van: njo...@megan.vbhcs.org [mailto:owner-postfix-us...@postfix.org] > Namens Noel Jones > Verzonden: dinsdag 20 december 2016 17:50 > Aan: postfix-users@postfix.org > Onderwerp: Re: request improved logging for postfix. > > On 12/20/2016 3:17 AM, L.P.H. van Belle wrote: > > > > postfix/ [smtp/smtpd/postscreen] show [client-hostname or unknown] IP > > > > (*always unknown if A/PTR mismatches in client hostname OR helo > > hostname) > > Labeling a client as unknown has nothing to do with the helo name. > > See the description for reject_unknown_client_hostname for the > conditions when a client is labeled unknown. > http://www.postfix.org/postconf.5.html#reject_unknown_client_hostname > > Postfix mostly ignores the helo name. You should too. > > > -- Noel Jones
Re: request improved logging for postfix.
On 12/20/2016 3:17 AM, L.P.H. van Belle wrote: > > postfix/ [smtp/smtpd/postscreen] show [client-hostname or unknown] IP > > (*always unknown if A/PTR mismatches in client hostname OR helo > hostname) Labeling a client as unknown has nothing to do with the helo name. See the description for reject_unknown_client_hostname for the conditions when a client is labeled unknown. http://www.postfix.org/postconf.5.html#reject_unknown_client_hostname Postfix mostly ignores the helo name. You should too. -- Noel Jones
RE: request improved logging for postfix.
Thank you Noel, again :-) Based on my loglines i found that; postfix/ [smtp/smtpd/postscreen] show [client-hostname or unknown] IP (*always unknown if A/PTR mismatches in client hostname OR helo hostname) postfix/ cleanup (header Received) show from helo-hostname (client-hostname [IP]) Any i missed? Thank your for this one. check_client_access static:INFO Thats very usefull for me. Now, big thread for a small thing, i hope lots of others profit from it. :-) Greetings, Louis > -Oorspronkelijk bericht- > Van: njo...@megan.vbhcs.org [mailto:owner-postfix-us...@postfix.org] > Namens Noel Jones > Verzonden: maandag 19 december 2016 17:43 > Aan: postfix-users@postfix.org > Onderwerp: Re: request improved logging for postfix. > > On 12/19/2016 3:31 AM, L.P.H. van Belle wrote: > > > > > So when everything is setup correct the helo and hostname ares shown > > in the logs, > > On a normal, accepted connection, the HELO name is never shown in > the logs. The client is identified by the source IP and port and > verified client hostname if available. The HELO name is only logged > with a rejection or error. > > The HELO name is recorded in the Received: header added to mail. > > If you want to always see the HELO in the logs, you can force a log > entry with "check_client_access static:INFO" in your > smtpd_recipient_restrictions. > > something like: > # main.cf > smtpd_recipient_restrictions = > check_client_access static:INFO > ... other checks ... > > > > > -- Noel Jones
Re: request improved logging for postfix.
On 12/19/2016 3:31 AM, L.P.H. van Belle wrote: > > So when everything is setup correct the helo and hostname ares shown > in the logs, On a normal, accepted connection, the HELO name is never shown in the logs. The client is identified by the source IP and port and verified client hostname if available. The HELO name is only logged with a rejection or error. The HELO name is recorded in the Received: header added to mail. If you want to always see the HELO in the logs, you can force a log entry with "check_client_access static:INFO" in your smtpd_recipient_restrictions. something like: # main.cf smtpd_recipient_restrictions = check_client_access static:INFO ... other checks ... -- Noel Jones
RE: request improved logging for postfix.
Hai, Well, Thank you Noel, This makes much more sence now. I was mislead due to the log messages of postfix. My own server has an A/PTR to the hostname and A/MX for helo name. This is the confusing part, at least it was for me. The logs showed me: postfix/smtpd[29331]: connect from core.van-belle.nl[149.210.206.148] and Dec 19 09:46:36 mailhopper postfix/cleanup[29334]: 451A6FF071: hold: header Received: from mail.van-belle.nl (core.van-belle.nl [149.210.206.148]) ... etc ??(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))??(Client did not present a certificate)??by mailhopper.ba from core.van-belle.nl[149.210.206.148]; from=<lo...@van-belle.nl> to=<be...@bazuin.nl> proto=ESMTP helo= The : connect from hostname.fqdn[ip] and : hold: header Received: from mail.van-belle.nl (core.van-belle.nl [149.210.206.148]) and here is also shows mail.van-belle.nl, the helo name and the host.fqdn[ip] since i always did see : mail.van-belle.nl (core.van-belle.nl [149.210.206.148]) i was in the understanding postfix was loggin helo hostnames also, like the client name. Which explains all the confusion at my side. > No fixes are necessary, other than maybe I should write a tutorial > on reading logs. Very good idea, the part you explained is a good one, and that wil help others also. Due to this logging i am/was having discusions. Now..this helps a lot. Thanks you so much. So when everything is setup correct the helo and hostname ares shown in the logs, but when with errors it referes only back to the client name. Why is this? Best regards, Louis > -Oorspronkelijk bericht- > Van: njo...@megan.vbhcs.org [mailto:owner-postfix-us...@postfix.org] > Namens Noel Jones > Verzonden: vrijdag 16 december 2016 16:56 > Aan: postfix-users@postfix.org > Onderwerp: Re: request improved logging for postfix. > > On 12/16/2016 5:13 AM, L.P.H. van Belle wrote: > > > Maybe im totaly incorrect here so correct me if needed. > > Yes. > > > Now, Im running Debian Wheezy, postfix ( debian backport ) > > 2.11.2-1~bpo70+1. Kernel : 3.2.82-1 > > > > I’ve increased the debug level in postfix for the domains. > > Don't use debug logging. Everything you need is in the normal > logging, and the extra noise just confuses you. > > > > Dec 16 08:47:31 mailhopper postfix/smtpd[16089]: warning: hostname > > sweeper.stater.com does not resolve to address 193.172.8.206: Name > > or service not known > > > > Dec 16 08:47:32 mailhopper postfix/smtpd[16089]: NOQUEUE: reject: > > RCPT from unknown[193.172.8.206]: 554 5.7.1 : > > Helo command rejected: Host not found; from=<serviced...@stater.nl> > > to=<be...@bazuin.nl> proto=ESMTP helo= > > > > > > > > This part : > > > > hostname sweeper.stater.com does not resolve to address > > 193.172.8.206 which is totaly correct. > > > > > No, the warning: message always refers to the CLIENT hostname, and > is giving you the reason the CLIENT is labeled as "unknown". > > > > The line (part of the rejected incomming ) > > > > ... NOQUEUE: reject: RCPT from unknown[193.172.8.206]: 554 5.7.1 > > > > > > More consistant would be : > > > > unknown([193.172.8.206]): 554 5.7.1 > > > > Or with correct A/PTR but incorrect helo > > But the A/PTR is not correct, as logged earlier. That is the reason > the client is labeled unknown. > > > > To many people are confused by the “unknown” since it can be 2 things: > > > > Unknown CLIENT hostname > > > > Unknown HELO hostname > > No, the "unknown" always refers to the client, unless it's in the > descriptive text of a reject message. > > > ... reject: {smtp stage} from {client hostname/unknown}[{ipaddr]}: > {reject code} {extended code}; {descriptive text} > > Notice the HELO name is never listed other than in the descriptive > text if HELO is the reason for rejection. > > > > > > Which give discusions on the fixes. > > No fixes are necessary, other than maybe I should write a tutorial > on reading logs. > > > > -- Noel Jones
Re: request improved logging for postfix.
On 12/16/2016 10:27 AM, /dev/rob0 wrote: > On Fri, Dec 16, 2016 at 09:56:26AM -0600, Noel Jones wrote: >> No fixes are necessary, other than maybe I should write a tutorial >> on reading logs. > > Oh, a LOG_README, an excellent idea! Later it can branch out into > the various configuration knobs we might eventually see. > > Do you think you could start a draft sometime soon? I'd be happy to > review and comment if you like. > I'll start getting something together, but probably not until sometime next month. I don't think an exhaustive breakdown of all possible log messages is required; just a fairly short tutorial on what the different items in a log entry mean and how different entries relate to each other should be sufficient for now. If someone else wants to contribute a draft, please do. -- Noel Jones
Re: request improved logging for postfix.
On Fri, Dec 16, 2016 at 09:56:26AM -0600, Noel Jones wrote: > No fixes are necessary, other than maybe I should write a tutorial > on reading logs. Oh, a LOG_README, an excellent idea! Later it can branch out into the various configuration knobs we might eventually see. Do you think you could start a draft sometime soon? I'd be happy to review and comment if you like. -- http://rob0.nodns4.us/ Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:
RE: request improved logging for postfix.
> No fixes are necessary, other than maybe I should write a tutorial > on reading logs. > > -- Noel Jones +1 In particular, your writing style is exceptionally clear! Michael
Re: request improved logging for postfix.
On 12/16/2016 5:13 AM, L.P.H. van Belle wrote: > Maybe im totaly incorrect here so correct me if needed. Yes. > Now, Im running Debian Wheezy, postfix ( debian backport ) > 2.11.2-1~bpo70+1. Kernel : 3.2.82-1 > > I’ve increased the debug level in postfix for the domains. Don't use debug logging. Everything you need is in the normal logging, and the extra noise just confuses you. > Dec 16 08:47:31 mailhopper postfix/smtpd[16089]: warning: hostname > sweeper.stater.com does not resolve to address 193.172.8.206: Name > or service not known > > Dec 16 08:47:32 mailhopper postfix/smtpd[16089]: NOQUEUE: reject: > RCPT from unknown[193.172.8.206]: 554 5.7.1 : > Helo command rejected: Host not found; from=> to= proto=ESMTP helo= > > > > This part : > > hostname sweeper.stater.com does not resolve to address > 193.172.8.206 which is totaly correct. > No, the warning: message always refers to the CLIENT hostname, and is giving you the reason the CLIENT is labeled as "unknown". > The line (part of the rejected incomming ) > > ... NOQUEUE: reject: RCPT from unknown[193.172.8.206]: 554 5.7.1 > > > More consistant would be : > > unknown([193.172.8.206]): 554 5.7.1 > > Or with correct A/PTR but incorrect helo But the A/PTR is not correct, as logged earlier. That is the reason the client is labeled unknown. > To many people are confused by the “unknown” since it can be 2 things: > > Unknown CLIENT hostname > > Unknown HELO hostname No, the "unknown" always refers to the client, unless it's in the descriptive text of a reject message. ... reject: {smtp stage} from {client hostname/unknown}[{ipaddr]}: {reject code} {extended code}; {descriptive text} Notice the HELO name is never listed other than in the descriptive text if HELO is the reason for rejection. > > Which give discusions on the fixes. No fixes are necessary, other than maybe I should write a tutorial on reading logs. -- Noel Jones
Re: request improved logging for postfix.
>Now, here is an inconistany of logging ( i think ) by postfix. > >I point to this line,: ?sweeper2.stater.com[193.172.8.206]:25: >220-sweeper.stater.com ESMTP ? > >More consistand would be (sweeper2.stater.com[193.172.8.206]):25: >220-sweeper.stater.com ESMTP ? The form: client: request from client server: response from server is consistent with the widely-used convention to show a protocol transcript in Internet RFC documents. Wietse
request improved logging for postfix.
Hello, After the message from yesterday, im asking if the postfix logging can be changed. To improve the loggings and a better more clear reject message. A small change maybe, i dont know, i’ll show what i mean below. Maybe im totaly incorrect here so correct me if needed. Now, Im running Debian Wheezy, postfix ( debian backport ) 2.11.2-1~bpo70+1. Kernel : 3.2.82-1 I’ve increased the debug level in postfix for the domains. Im seeing the following : Time : 08:34 : me be...@bazuin.nl sending to serviced...@stater.com Dec 16 08:34:39 mailhopper postfix/smtp[15288]: < sweeper2.stater.com[193.172.8.206]:25: 220-sweeper.stater.com ESMTP Dec 16 08:34:39 mailhopper postfix/smtp[15288]: < sweeper2.stater.com[193.172.8.206]:25: 220 Connection is logged and abuse will be reported... Dec 16 08:34:39 mailhopper postfix/smtp[15288]: > sweeper2.stater.com[193.172.8.206]:25: EHLO mailhopper.bazuin.nl Dec 16 08:34:39 mailhopper postfix/smtp[15288]: < sweeper2.stater.com[193.172.8.206]:25: 250-sweeper.stater.com Dec 16 08:34:39 mailhopper postfix/smtp[15288]: < sweeper2.stater.com[193.172.8.206]:25: 250-8BITMIME Dec 16 08:34:39 mailhopper postfix/smtp[15288]: < sweeper2.stater.com[193.172.8.206]:25: 250-SIZE 52428800 Dec 16 08:34:39 mailhopper postfix/smtp[15288]: < sweeper2.stater.com[193.172.8.206]:25: 250 STARTTLS Dec 16 08:34:39 mailhopper postfix/smtp[15288]: > sweeper2.stater.com[193.172.8.206]:25: STARTTLS Dec 16 08:34:39 mailhopper postfix/smtp[15288]: < sweeper2.stater.com[193.172.8.206]:25: 220 Go ahead with TLS Dec 16 08:34:39 mailhopper postfix/smtp[15288]: send attr cache_id = smtp&193.172.8.206&&4DFEB04581B7B5FE02EE5DA3C09609BF6F53AC5A02666E3BE4556ED143A51345 Dec 16 08:34:39 mailhopper postfix/smtp[15288]: send attr cache_id = smtp&193.172.8.206&&4DFEB04581B7B5FE02EE5DA3C09609BF6F53AC5A02666E3BE4556ED143A51345 Dec 16 08:34:39 mailhopper postfix/smtp[15288]: Untrusted TLS connection established to sweeper2.stater.com[193.172.8.206]:25: TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits) Dec 16 08:34:39 mailhopper postfix/smtp[15288]: > sweeper2.stater.com[193.172.8.206]:25: EHLO mailhopper.bazuin.nl Dec 16 08:34:39 mailhopper postfix/smtp[15288]: < sweeper2.stater.com[193.172.8.206]:25: 250-sweeper.stater.com Dec 16 08:34:39 mailhopper postfix/smtp[15288]: < sweeper2.stater.com[193.172.8.206]:25: 250-8BITMIME Dec 16 08:34:39 mailhopper postfix/smtp[15288]: < sweeper2.stater.com[193.172.8.206]:25: 250 SIZE 52428800 Dec 16 08:34:39 mailhopper postfix/smtp[15288]: > sweeper2.stater.com[193.172.8.206]:25: MAIL FROM:SIZE=19695 Dec 16 08:34:39 mailhopper postfix/smtp[15288]: < sweeper2.stater.com[193.172.8.206]:25: 250 sender ok Dec 16 08:34:39 mailhopper postfix/smtp[15288]: > sweeper2.stater.com[193.172.8.206]:25: RCPT TO: Dec 16 08:34:39 mailhopper postfix/smtp[15288]: < sweeper2.stater.com[193.172.8.206]:25: 250 recipient ok Dec 16 08:34:39 mailhopper postfix/smtp[15288]: > sweeper2.stater.com[193.172.8.206]:25: DATA Dec 16 08:34:39 mailhopper postfix/smtp[15288]: < sweeper2.stater.com[193.172.8.206]:25: 354 go ahead Now, here is an inconistany of logging ( i think ) by postfix. I point to this line,: sweeper2.stater.com[193.172.8.206]:25: 220-sweeper.stater.com ESMTP More consistand would be (sweeper2.stater.com[193.172.8.206]):25: 220-sweeper.stater.com ESMTP Or without a/ptr for the client name: (unknown[193.172.8.206]):25: 220-sweeper.stater.com ESMTP At Time : 08:47 : reply from stater.com to my but rejected as it should. Dec 16 08:47:31 mailhopper postfix/smtpd[16089]: warning: hostname sweeper.stater.com does not resolve to address 193.172.8.206: Name or service not known Dec 16 08:47:32 mailhopper postfix/smtpd[16089]: NOQUEUE: reject: RCPT from unknown[193.172.8.206]: 554 5.7.1 : Helo command rejected: Host not found; from= to= proto=ESMTP helo= This part : hostname sweeper.stater.com does not resolve to address 193.172.8.206 which is totaly correct. But it would be nicer to set : “helo hostname sweeper.stater.com does not resolve to address 193.172.8.206“ The line (part of the rejected incomming ) ... NOQUEUE: reject: RCPT from unknown[193.172.8.206]: 554 5.7.1 More consistant would be : unknown([193.172.8.206]): 554 5.7.1 Or with correct A/PTR but incorrect helo unknown(sweeper2.stater.com[193.172.8.206]): 554 5.7.1 You see the small () changes all together.. : unknown[193.172.8.206]: 554 5.7.1 unknown([193.172.8.206]): 554 5.7.1 unknown(sweeper2.stater.com[193.172.8.206]): 554 5.7.1 To many people are confused by the “unknown” since it can be 2 things: Unknown CLIENT hostname Unknown HELO hostname Which give discusions on the fixes. Also what i dont get here is the postfix message . NOQUEUE: reject: RCPT