Re: Conditional milter_header_checks?
Please keep replies on-list only. Duplicates of anything sent to the list are just a nuisance. On 2021-07-14 at 20:51:03 UTC-0400 (Thu, 15 Jul 2021 10:51:03 +1000) raf is rumored to have said: On Wed, Jul 14, 2021 at 09:07:54AM -0400, Bill Cole wrote: On 2021-07-14 at 03:43:57 UTC-0400 (Wed, 14 Jul 2021 17:43:57 +1000) raf is rumored to have said: Here's a (silly) thing that wrong with DMARC: :-) I've sent two messages to this mailing list so far, and I've received 52 DMARC forensic/failure report emails as a result! :-) There are 2 different and contradictory DMARC records in DNS for raf.org. That guarantees breakage. I think it's in the process of propagating. I don't know what's taking it so long. Your primary nameserver is returning 2 TXT records for _dmarc.raf.org, so this is not a propagation issue. [...] SPF is intended to be used with the envelope sender address and (secondarily) the HELO/EHLO hostname argument. If those do not 'align' with the header From address, DMARC will not use SPF. When you say "DMARC will not use SPF", does that mean that a difference between the envelope address and the From: address constitutes a DMARC+SPF failure? Yes. Best explanation I've seen: https://mxtoolbox.com/dmarc/spf/spf-alignment And specifically, a failure relating to the From: domain? DMARC is always relating to the From header address. If the envelope sender (often: Return-Path) is verified by SPF and aligns with the From header address, DMARC passes. If there is a valid DKIM signature for a domain which aligns with the From header address, DMARC passes. Is it a DMARC+SPF failure relating to the envelope domain as well? i.e. could failure reports be sent to both domains if both "reporting policies" requested it? Have you considered reading the RFC for yourself? https://datatracker.ietf.org/doc/html/rfc7489 DMARC is designed to break the traditional practices of both simple transparent forwarding and discussion mailing lists. To do so, it allows the use of SPF in a manner it was never intended to be used, to "align" with the header From address. Since mailing lists properly use their own domain in the envelope, SPF for a mailing list delivery will never align with the author's From header. If you want to post to discussion mailing lists, you should either use a From address in a domain without any DMARC record or publish one with a p=none policy and sign your messages with DKIM, even though they are likely to be broken by the mailing list. My policy is p=none. Hopefully, that'll be sufficient to limit any damage. Based on other traffic here in a collateral subthread in the past day, it is not. At least one person running a mail server and discussing their choices in public is convinced that if your message is reformatted in transit in any way or if mailing list software adds anything to the body or Subject, your now-broken signature is a sound reason to reject your message arriving via a mailing list, because "there is no reason why such messages should pollute the receiving systems." The resulting damage should be isolated to his system. You obviously misunderstood the sub-thread you mention. Cheers, K.
Re: Conditional milter_header_checks?
You can certainly take a pedantic view, that's contrary to the DKIM RFCs and common sense, there's no Internet police to stop you. Just keep in mind that rejecting failed DKIM signatures has no security benefit. Hm, there is always the possibility that I misunderstood the specifications. Corrections are always welcome :). I do think, however, that my view is actually in line with the DKIM specs, although I wouldn't call it quite pedantic :) . A DKIM signature conveys the origin domain in a DKIM-Signature header that most users don't see, and that (DMARC aside) need not bear any relationship to either the envelope sender address or the RFC2822.From address. Yes, you are correct. Nowhere have I stated otherwise. * Apart from conveying potentially positive reputation information, What security benefit do you see in such a signature? * If a bad actor can choose between not signing (neutral outcome) or signing with a key that fails to verify (negative outcome), what would you expect the bad actor to do? * Given the above, what is the value of rejecting signatures that fail to verify? What class of attacks are you stopping? "It is beyond the scope of this specification to describe what actions an Identity Assessor can make, but mail carrying a validated SDID presents an opportunity to an Identity Assessor that unauthenticated email does not." From this, one can conclude that the DKIM specification actually makes no formal recommendations on whether to reject or accept a message. Even if there is no additional security benefit, there is no reason why such messages should pollute the receiving systems. Well, it clearly (top of page 51) suggests that DKIM validation failure SHOULD NOT it itself cause a message to be rejected. "In general, modules that consume DKIM verification output SHOULD NOT determine message acceptability based solely on a lack of any signature or on an *unverifiable signature*; such rejection would cause severe interoperability problems." Now, *unverifiable signature* is not the same as "signature present but not valid". Actually, it is in this case. No, it is not. Meaning of words does matter. The "sender domain" is free to choose not to sign its messages and to not publish a DKIM record. There isn't "a DKIM record", there's one per selector, and until you see a signed message with a particular selector, there's no mechanism for locating any or all of a domain's DKIM signing public keys. You are correct again. And again, nowhere have I stated that such a locator mechanism is available. You can nitpick at the unfortunately chosen "a DKIM record" expression if you want but the idea still stands. Also, loosely speaking, "one per selector" is still "a DKIM record". But *if it does* sign them, *the signature should be valid*. It is common sense. It is a common mistake. Some of a domain's mail (some transactional or opted-in marketing) traffic may be signed under various selectors, and the rest might not. The RFC2822.From of signed messages may differ from the DKIM "d=" domain. Absent DMARC, a DKIM signature just tells which domain potentially takes *responsibility* for sending the message. When the signatuer check fails, you can't make that connection. That's all. The message may be transformed in some manner in transit that invalidates the signature, sometimes right from the start if the sending domain has software that first signs, and then does 8-bit to 7-bit downgrade, or adds a footer, ... Sure they should not do that, but this is not a good reason to seek to "punish" them for this. Publishing a DKIM record by the "sender domain" does in fact constitute an "implicit prior agreement" that the "sender domain" sends signed messages. So, *if present*, the signature should be valid. Actually, no. It just makes it possible to take responsibility for *some* messages. They might for example not choose to take responsibility for bounces (whose returned body they did not author). "If the email cannot be verified, then it SHOULD be treated the same as all unverified email, regardless of whether or not it looks like it was signed. See Section 8.15 for additional discussion." might seem like a recommendation for non-rejection to always occur when an email can not be verified, Section 8.15 shows that they are cases when delivery can be refused: That's the only sensible, non-pedantic, policy. "It is up to the Identity Assessor or some other subsequent agent to act on such messages as needed, such as degrading the trust of the message (or, indeed, of the Signer), warning the recipient, *or even refusing delivery*." Again, I believe that mail handling software, such as mailing lists or intermediate relays, should fix their issues and be standard compliant instead of putting the burden on the
Re: Conditional milter_header_checks?
It is a really bad idea to reject messages whose DKIM signature is invalid. DO NOT DO THIS. Why exactly is it a really bad idea :) ? Could you give us some more practical details/examples? The point is that absent DMARC policy that promises DKIM signatures aligned with the RFC2822.From domain, there is no sane threat model in which rejecting invalid DKIM signatures yields any security benefit. An attacker (spammer if you like), can always sign the mail with some throw-away domain, or not sign it at all. So a failed DKIM signature conveys nothing other than perhaps an operator error on the legitimate sending system, or an unexpected message transformation in transit. No spammer wastes bandwidth sending messages with broken DKIM signatures, they either sign correctly, or don't sign at all. In my opinion if a signature is present is should be valid. Always. Otherwise it loses it's whole purpose. You can certainly take a pedantic view, that's contrary to the DKIM RFCs and common sense, there's no Internet police to stop you. Just keep in mind that rejecting failed DKIM signatures has no security benefit. Hm, there is always the possibility that I misunderstood the specifications. Corrections are always welcome :). I do think, however, that my view is actually in line with the DKIM specs, although I wouldn't call it quite pedantic :) . Here is how I see it: Section 6.3 "Interpret Results/Apply Local Policy" states: "It is beyond the scope of this specification to describe what actions an Identity Assessor can make, but mail carrying a validated SDID presents an opportunity to an Identity Assessor that unauthenticated email does not." From this, one can conclude that the DKIM specification actually makes no formal recommendations on whether to reject or accept a message. The choice is up to the receiving system. Whatever the decision might be, it will still be in line with the specs and with common sense. Next, it does give some general guidelines: "In general, modules that consume DKIM verification output SHOULD NOT determine message acceptability based solely on a lack of any signature or on an *unverifiable signature*; such rejection would cause severe interoperability problems." Now, *unverifiable signature* is not the same as "signature present but not valid". A signature can be unverifiable for multiple reasons, such as the temporary inability to access the key server. The "sender domain" is free to choose not to sign its messages and to not publish a DKIM record. But *if it does* sign them, *the signature should be valid*. It is common sense. "If an MTA does wish to reject such messages during an SMTP session (for example, when communicating with a peer who, by prior agreement, agrees to only send signed messages), and a signature is missing or does not verify, the handling MTA SHOULD use a 550/5.7.x reply code." Publishing a DKIM record by the "sender domain" does in fact constitute an "implicit prior agreement" that the "sender domain" sends signed messages. So, *if present*, the signature should be valid. While this: "If the email cannot be verified, then it SHOULD be treated the same as all unverified email, regardless of whether or not it looks like it was signed. See Section 8.15 for additional discussion." might seem like a recommendation for non-rejection to always occur when an email can not be verified, Section 8.15 shows that they are cases when delivery can be refused: "It is up to the Identity Assessor or some other subsequent agent to act on such messages as needed, such as degrading the trust of the message (or, indeed, of the Signer), warning the recipient, *or even refusing delivery*." Again, I believe that mail handling software, such as mailing lists or intermediate relays, should fix their issues and be standard compliant instead of putting the burden on the end recipient system. Spammers are often early adopters of various email security standards. On some receiving systems there's a positive correlation between a message having a valid DKIM signature and the message being spam! I wold even go so far as to require DKIM signatures from everybody. But unfortunately that is not quite possible since there are still many who, for various reasons, can't provide a DKIM signature at all :) . Your network, your rules. I am just trying to give rational advice. On a more practical note: Indeed, our organization does handle quite a large amount of messages and transactions are very closely monitored for this kind of issues. So far, the only DKIM related issues we ever had were with a couple of poorly configured or outdated mailing list software and spammers. Lots and lots of spammers. However, depending on their configuration, the situation might be different for
Re: Conditional milter_header_checks?
The DKIM standards are quite emphatically clear that bad signature == no signature, and that receiving systems MUST NOT reject a message just because a signature is missing or fails to match. The treatment of messages that lack a signature is covered by DMARC (and ARC). It is a really bad idea to reject messages whose DKIM signature is invalid. DO NOT DO THIS. Why exactly is it a really bad idea :) ? Could you give us some more practical details/examples? It is true that DKIM does not convey a sender domain policy, but that should not limit or impose decision restriction on the receiving end. I don't see why should the receiver base its decisions of how to handle bad signatures on the wishes of the sender domain. By the way, I don't use DMARC. In my opinion if a signature is present is should be valid. Always. Otherwise it loses it's whole purpose. I wold even go so far as to require DKIM signatures from everybody. But unfortunately that is not quite possible since there are still many who, for various reasons, can't provide a DKIM signature at all :) . If a mail handling software, such as a mailing list one, changes a message in a way that breaks a signature, it should instead encapsulate the original message in a completely new message with a valid signature. If opendkim supports "On-BadSignature reject", that's a disservice to its users. Yes, OpenDKIM does support this and I find that to be perfectly fine since it gives the user an option to decide how to handle this kind of situations. By default the action is set to "accept" anyway. Cheers, K.
Re: [ENHANCEMENT] Log SMTP protocol errors to SYSLOG
For example the following transaction will not show any errors in SYSLOG: In: AUTH LOGIN Out: 503 5.5.1 Error: authentication not enabled In: QUIT Out: 221 2.0.0 Bye You can use the existing notify_classes based mechamism and pipe that into syslog. notify_classes = protocol, ... error_notice_recipient = syslog@localhost syslog@localhost can be a transport_maps entry for a pipe(8) service invokes a script like this to log the body of the message: #!/bin/sh PATH=/bin:/usr/bin:/usr/local/bin sed -n '/^$/,$p' | postlog -p info -t transcript Wietse This can do the trick to some extent, but it can still be very verbose since (by default) the entire transaction is included in the mail and not just the relevant errors. This 'entire transaction' is only a few lines: 220 greeting ehlo command + response mail from + response rcpt to + response data + response quit + response Also, at a quick glance, it seems that this approach would be missing relevant client information, such as the client host/IP. You have enough information in the maillog file. Postfix logs the ehlo, mail from, rcpt to, and more. That same info is also in the session transcript, therefore connecting them is not difficult. With the enhancement I was suggesting a more "tightly coupled" approach, like in the case of a "reject" log message. For example, like this one: reject: RCPT from unknown[X.X.X.X]: 550 5.7.25 Client host rejected: cannot find your hostname, [X.X.X.X]; ... That is not a protocol error. Correct. It is not. The mentioned "reject" log message was just a very loose example of how the protocol error log message might look like. Logging every individual command+error would require major changes to the SMTP server code. Oh, I see. I was not aware of that :) . Thank you for the clarification. Cheers, K.
Re: [ENHANCEMENT] Log SMTP protocol errors to SYSLOG
For example the following transaction will not show any errors in SYSLOG: In: AUTH LOGIN Out: 503 5.5.1 Error: authentication not enabled In: QUIT Out: 221 2.0.0 Bye You can use the existing notify_classes based mechamism and pipe that into syslog. notify_classes = protocol, ... error_notice_recipient = syslog@localhost syslog@localhost can be a transport_maps entry for a pipe(8) service invokes a script like this to log the body of the message: #!/bin/sh PATH=/bin:/usr/bin:/usr/local/bin sed -n '/^$/,$p' | postlog -p info -t transcript Wietse This can do the trick to some extent, but it can still be very verbose since (by default) the entire transaction is included in the mail and not just the relevant errors. Also, at a quick glance, it seems that this approach would be missing relevant client information, such as the client host/IP. With the enhancement I was suggesting a more "tightly coupled" approach, like in the case of a "reject" log message. For example, like this one: reject: RCPT from unknown[X.X.X.X]: 550 5.7.25 Client host rejected: cannot find your hostname, [X.X.X.X]; ... Cheers, K. Although, true, the client information could be obtained from the subject of the message sent to syslog@localhost if we customize the proposed script a bit. But I still think it would be nice to have a configuration option and a more "tightly coupled" and "cleaner" approach for this. Cheers, K.
Re: [ENHANCEMENT] Log SMTP protocol errors to SYSLOG
Kevin N.: For example the following transaction will not show any errors in SYSLOG: In: AUTH LOGIN Out: 503 5.5.1 Error: authentication not enabled In: QUIT Out: 221 2.0.0 Bye You can use the existing notify_classes based mechamism and pipe that into syslog. notify_classes = protocol, ... error_notice_recipient = syslog@localhost syslog@localhost can be a transport_maps entry for a pipe(8) service invokes a script like this to log the body of the message: #!/bin/sh PATH=/bin:/usr/bin:/usr/local/bin sed -n '/^$/,$p' | postlog -p info -t transcript Wietse This can do the trick to some extent, but it can still be very verbose since (by default) the entire transaction is included in the mail and not just the relevant errors. Also, at a quick glance, it seems that this approach would be missing relevant client information, such as the client host/IP. With the enhancement I was suggesting a more "tightly coupled" approach, like in the case of a "reject" log message. For example, like this one: reject: RCPT from unknown[X.X.X.X]: 550 5.7.25 Client host rejected: cannot find your hostname, [X.X.X.X]; ... Cheers, K.
[ENHANCEMENT] Log SMTP protocol errors to SYSLOG
It would be nice to have an option to enable logging to SYSLOG the SMTP protocol errors that occur during a SMTP session, along with the SMTP commands that caused them. As far as I know, currently these errors can be logged to SYSLOG only by one of the following methods: 1. By making the SMTP service more verbose. In this case a large amount of additional data is being logged, which might not always be desirable. 2. By adding "protocol" to the list of notify_classes. In this case an transcript email is sent to "postmaster" (or to a configured recipient) for every SMTP session that contains errors, which again might not always be desirable and might be an overkill. For example the following transaction will not show any errors in SYSLOG: In: AUTH LOGIN Out: 503 5.5.1 Error: authentication not enabled In: QUIT Out: 221 2.0.0 Bye Cheers, K.
Re: Stopping backscatter spam to a specific domain
This might help: http://www.postfix.org/BACKSCATTER_README.html Cheers, K. On Jul 11, 2021, at 9:58 AM, Wietse Venema wrote: Ron Garret: I have recently come under a backscatter spam attack from one specific domain. This domain has blacklisted my server?s IP address, and so bounce replies sent to this domain are piling up in my mail queue and I have to go through periodically and manually delete them. I don?t want to disable bounce messages in general because I don?t want incoming messages with typos in the destination address to just vanish into the cosmic void. Is there a way to disable bounce replies for a specific domain? Why is your server sending bounces (or any other email) to that domain? Because spammers are sending messages with forged return-path headers to invalid addresses on my server. It’s called backscatter: https://en.wikipedia.org/wiki/Backscatter_(email) It’s actually possible that I’m sending backscatter spam to other domains, but only one has blacklisted me so far. rg
Re: policy_service protocol_state with smtpd_delay_reject
Is there a way to reuse the same instance of the script, not spawn two instances, and some how have the script know which restriction it was called from? Not sure if this helps, but maybe you could try to implement your policy server as a standalone network server instead of calling it through the spawn service. Cheers, K.
Re: policy_service protocol_state with smtpd_delay_reject
I was curious so I did a quick test :) . As suspected, it does work.
Having a setup like:
smtpd_delay_reject = yes
smtpd_helo_required = yes
smtpd_helo_restrictions =
...
check_policy_service { unix:private/policy-service,
policy_context=helo_restrictions_value }
...
smtpd_relay_restrictions =
...
check_policy_service { unix:private/policy-service,
policy_context=relay_restrictions_value }
...
will allow you to pass the desired value to the policy server.
1st call result, for smtpd_helo_restrictions :
Read line: "request=smtpd_access_policy"
Read line: "protocol_state=RCPT"
...
Read line: "policy_context=helo_restrictions_value"
Read line: ""
2nd call result, for smtpd_relay_restrictions :
Read line: "request=smtpd_access_policy"
Read line: "protocol_state=RCPT"
...
Read line: "policy_context=relay_restrictions_value"
Read line: ""
Cheers,
K.
Haven't tried it, but this might be what you are looking for.
http://www.postfix.org/SMTPD_POLICY_README.html#advanced
check_policy_service { inet:host:port,
timeout=10s, default_action=DUNNO
policy_context=submission }
...
From the SMTPD_POLICY_README:
The "policy_context" attribute provides a way to pass information that
is not available via other attributes (Postfix version 3.1 and later).
I notice when using SMTPD_DELAY_REJECT=yes and calling a
CHECK_POLICY_SERVICE inside SMTPD_HELO_RESTRICTIONS it will report
"protocol_state = RCPT", same as when you call the policy service from
inside SMTPD_RECIPIENT_RESTRICTIONS.
Is there a way to pass a value from the CHECK_POLICY_SERVICE line (or
any other method), to know which restriction section the policy
service is being ran inside of?
Could you give us a bit more details about what are you trying to do? :)
Cheers,
K.
Re: policy_service protocol_state with smtpd_delay_reject
Haven't tried it, but this might be what you are looking for.
http://www.postfix.org/SMTPD_POLICY_README.html#advanced
check_policy_service { inet:host:port,
timeout=10s, default_action=DUNNO
policy_context=submission }
...
From the SMTPD_POLICY_README:
The "policy_context" attribute provides a way to pass information that
is not available via other attributes (Postfix version 3.1 and later).
I notice when using SMTPD_DELAY_REJECT=yes and calling a
CHECK_POLICY_SERVICE inside SMTPD_HELO_RESTRICTIONS it will report
"protocol_state = RCPT", same as when you call the policy service from
inside SMTPD_RECIPIENT_RESTRICTIONS.
Is there a way to pass a value from the CHECK_POLICY_SERVICE line (or
any other method), to know which restriction section the policy service
is being ran inside of?
Could you give us a bit more details about what are you trying to do? :)
Cheers,
K.
Re: smtpd_reject_unlisted_recipient = yes
Also, the "Delayed evaluation of SMTP access restriction lists" section from the SMTPD_ACCESS_README page, might give you some answers. http://www.postfix.org/SMTPD_ACCESS_README.html#timing Cheers, K My educated guess would be it is checked at the end of the supplied options for smtpd_recipient_restrictions, is that correct? On a very short glance at the source code, your guess does seem to be correct. src/smtpd/smtpd_check.c: /* * If the "reject_unlisted_recipient" restriction still needs to be * applied, validate the recipient here. */ if (var_smtpd_rej_unl_rcpt && status != SMTPD_CHECK_REJECT && state->recipient_rcptmap_checked == 0 && state->discard == 0) status = check_recipient_rcpt_maps(state, recipient); However, I am not very familiar with the Postfix source code. Maybe somebody closer to the code can give you a more absolute confirmation. Cheers, K.
Re: smtpd_reject_unlisted_recipient = yes
My educated guess would be it is checked at the end of the supplied options for smtpd_recipient_restrictions, is that correct? On a very short glance at the source code, your guess does seem to be correct. src/smtpd/smtpd_check.c: /* * If the "reject_unlisted_recipient" restriction still needs to be * applied, validate the recipient here. */ if (var_smtpd_rej_unl_rcpt && status != SMTPD_CHECK_REJECT && state->recipient_rcptmap_checked == 0 && state->discard == 0) status = check_recipient_rcpt_maps(state, recipient); However, I am not very familiar with the Postfix source code. Maybe somebody closer to the code can give you a more absolute confirmation. Cheers, K.
Re: Clarify reject_* for smtpd_helo_restrictions
reject_invalid_helo_hostname Would reject an invalid host name such as "ho+st", but a valid hostname such as "host" would pass (https://datatracker.ietf.org/doc/html/rfc2821#section-2.3.5): 501 5.5.2 : Helo command rejected: Invalid name reject_non_fqdn_helo_hostname Would reject a non fully-qualified hostname such as "host", but a fully-qualified one such as "host.domain.tld" would pass: 504 5.5.2 : Helo command rejected: need fully-qualified hostname Cheers, K.
Re: Illegal address syntax in MAIL command
It seems that in the MAIL command the IP address is still not between []. should be On a quick look, it seems that you could try setting resolve_numeric_domain = yes in your Postfix configuration and see if that changes anything. From http://www.postfix.org/postconf.5.html resolve_numeric_domain (default: no) Resolve "user@ipaddress" as "user@[ipaddress]", instead of rejecting the address as invalid. Cheers, K. On 07/07/2021 18:08, j...@wrightthisway.com wrote: I believe you are correct, but again I have no control over that part. Also, I mistakenly attached the log attempt from the telnet session I tried, the actual systems having issues have the from address within brackets, here is the system in question: Jul 6 15:18:42 localhost postfix/smtpd[40342]: warning: Illegal address syntax from unknown[100.67.10.122] in MAIL command: On 2021-07-07 09:59, Kevin N. wrote: When using IP addresses in the email address, shouldn't the IP be enclosed between []? For example: noreply@[100.67.10.122] instead of noreply@100.67.10.122 Cheers, K. On 07/07/2021 17:49, j...@wrightthisway.com wrote: Hello folks. I have set up a fresh instance of Postfix at my office to help do some troubleshooting on another issue. There is a relay upstream that is having issues forwarding mail from some devices here, and this seemed the easiest way to get some data to help them troubleshoot. Install is Redat 8.4 using the postfix install from YUM. Everything is pretty much default settings. This is what I'm seeing in the logs: Jul 6 15:36:02 localhost postfix/smtpd[40841]: connect from desktop-204qpi1.example.net[100.67.2.4] Jul 6 15:36:20 localhost postfix/smtpd[40841]: warning: Illegal address syntax from desktop-204qpi1.example.net[100.67.2.4] in MAIL command: noreply@100.67.10.122 Jul 6 15:36:23 localhost postfix/smtpd[40841]: warning: Illegal address syntax from desktop-204qpi1.example.net[100.67.2.4] in MAIL command: noreply@100.67.10.122. Jul 6 15:38:11 localhost postfix/smtpd[40841]: warning: Illegal address syntax from desktop-204qpi1.example.net[100.67.2.4] in MAIL command: Jul 6 15:38:48 localhost postfix/smtpd[40841]: disconnect from desktop-204qpi1.example.net[100.67.2.4] mail=1/4 quit=1 unknown=0/1 commands=2/6 If I telnet to this postfix and use a mail from with an IP literal, it fails, but a DNS name works. I can't seem to locate the proper command to allow such emails to be received. These emails would be generated from Dell servers via their iDrac (system management), temperature probes, etc, so I have little control over how these devices send mail. Mail delivery would be targeted to system admins needing to monitor alerts from such systems. Below is my postconf output: [root@localhost postfix]# postconf -n alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases command_directory = /usr/sbin compatibility_level = 2 daemon_directory = /usr/libexec/postfix data_directory = /var/lib/postfix debug_peer_level = 2 debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd $daemon_directory/$process_name $process_id & sleep 5 html_directory = no inet_interfaces = all inet_protocols = all mail_owner = postfix mailq_path = /usr/bin/mailq.postfix manpage_directory = /usr/share/man meta_directory = /etc/postfix mydestination = $myhostname, localhost.$mydomain, localhost mynetworks = 100.67.0.0/16 mynetworks_style = subnet newaliases_path = /usr/bin/newaliases.postfix queue_directory = /var/spool/postfix readme_directory = /usr/share/doc/postfix/README_FILES sample_directory = /usr/share/doc/postfix/samples sendmail_path = /usr/sbin/sendmail.postfix setgid_group = postdrop shlib_directory = /usr/lib64/postfix smtp_tls_CAfile = /etc/pki/tls/certs/ca-bundle.crt smtp_tls_CApath = /etc/pki/tls/certs smtp_tls_security_level = may smtpd_tls_cert_file = /etc/pki/tls/certs/postfix.pem smtpd_tls_key_file = /etc/pki/tls/private/postfix.key smtpd_tls_security_level = may unknown_local_recipient_reject_code = 550
Re: Illegal address syntax in MAIL command
When using IP addresses in the email address, shouldn't the IP be enclosed between []? For example: noreply@[100.67.10.122] instead of noreply@100.67.10.122 Cheers, K. On 07/07/2021 17:49, j...@wrightthisway.com wrote: Hello folks. I have set up a fresh instance of Postfix at my office to help do some troubleshooting on another issue. There is a relay upstream that is having issues forwarding mail from some devices here, and this seemed the easiest way to get some data to help them troubleshoot. Install is Redat 8.4 using the postfix install from YUM. Everything is pretty much default settings. This is what I'm seeing in the logs: Jul 6 15:36:02 localhost postfix/smtpd[40841]: connect from desktop-204qpi1.example.net[100.67.2.4] Jul 6 15:36:20 localhost postfix/smtpd[40841]: warning: Illegal address syntax from desktop-204qpi1.example.net[100.67.2.4] in MAIL command: noreply@100.67.10.122 Jul 6 15:36:23 localhost postfix/smtpd[40841]: warning: Illegal address syntax from desktop-204qpi1.example.net[100.67.2.4] in MAIL command: noreply@100.67.10.122. Jul 6 15:38:11 localhost postfix/smtpd[40841]: warning: Illegal address syntax from desktop-204qpi1.example.net[100.67.2.4] in MAIL command: Jul 6 15:38:48 localhost postfix/smtpd[40841]: disconnect from desktop-204qpi1.example.net[100.67.2.4] mail=1/4 quit=1 unknown=0/1 commands=2/6 If I telnet to this postfix and use a mail from with an IP literal, it fails, but a DNS name works. I can't seem to locate the proper command to allow such emails to be received. These emails would be generated from Dell servers via their iDrac (system management), temperature probes, etc, so I have little control over how these devices send mail. Mail delivery would be targeted to system admins needing to monitor alerts from such systems. Below is my postconf output: [root@localhost postfix]# postconf -n alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases command_directory = /usr/sbin compatibility_level = 2 daemon_directory = /usr/libexec/postfix data_directory = /var/lib/postfix debug_peer_level = 2 debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd $daemon_directory/$process_name $process_id & sleep 5 html_directory = no inet_interfaces = all inet_protocols = all mail_owner = postfix mailq_path = /usr/bin/mailq.postfix manpage_directory = /usr/share/man meta_directory = /etc/postfix mydestination = $myhostname, localhost.$mydomain, localhost mynetworks = 100.67.0.0/16 mynetworks_style = subnet newaliases_path = /usr/bin/newaliases.postfix queue_directory = /var/spool/postfix readme_directory = /usr/share/doc/postfix/README_FILES sample_directory = /usr/share/doc/postfix/samples sendmail_path = /usr/sbin/sendmail.postfix setgid_group = postdrop shlib_directory = /usr/lib64/postfix smtp_tls_CAfile = /etc/pki/tls/certs/ca-bundle.crt smtp_tls_CApath = /etc/pki/tls/certs smtp_tls_security_level = may smtpd_tls_cert_file = /etc/pki/tls/certs/postfix.pem smtpd_tls_key_file = /etc/pki/tls/private/postfix.key smtpd_tls_security_level = may unknown_local_recipient_reject_code = 550
Re: Postfix / Dovecot Not able to authenticate
Hi Keith, You should really *not* post sensitive information on a public mailing list especially since your server is accessible from the internet. From what I can see, your posted auth logs contain the Base64 encoded password which can be very easily decoded. Cheers, Kevin On 05/07/2021 22:25, techli...@phpcoderusa.com wrote: Hi My name is Keith Smith. I am a PHP developer and have been a freelance PHP developer since 2006. I have been working on a PHP web server in the hope of learning more about Linux hosting server administration. Over the years I have had the opportunity to create many PHP virtual hosts. I have not worked with or configured BIND, Postfix, and Dovecot before about 3 weeks ago. My home server is connected to the Internet via my home office business cable account. No blocked ports and I am able run one or more servers. I am running Ubuntu 20.04lts / Apache / MySql (or a clone) / PHP / BIND9 / Postfix / Dovecot / Let's Encrypt SSL. My server’s FQDN is : soho.keiththewebguy.com as is my reverse lookup on my IP : 98.191.108.149. The purpose is for learning. I intend to run one such configured server out of my home office running one domain or more (virtual hosts) and would like to configure Postfix/Dovecot to service one or more email addresses per virtual hosting. I have BIND working. At least enough that a simple web page can be displayed. I've installed Postfix and Dovecot and am getting an authentication issue that shows up in the logs. I do not know all that you might need. Please let me know. I admit I find Postfix difficult to understand. I am at the newbie stage. I've scoured the Internet looking for answers. After I configured Postfix and Dovecot I issued the commend : telnet keiththewebguy.com 25 on my web server. It returned: Trying 98.191.108.149... Connected to keiththewebguy.com. Escape character is '^]'. 220 soho.keiththewebguy.com ESMTP Postfix I issued : ehlo soho.keiththewebguy.com 250-soho.keiththewebguy.com 250-PIPELINING 250-SIZE 1024 250-VRFY 250-ETRN 250-STARTTLS 250-AUTH PLAIN LOGIN 250-ENHANCEDSTATUSCODES 250-8BITMIME 250-DSN 250 CHUNKING I issued : quit 221 2.0.0 Bye Connection closed by foreign host. --- I assume this means things are configured correctly. I am looking at both logs: 1) /var/log/mail.err which contains no content. 2) /var/log/mail.log which is verbose. To /etc/dovecot/dovecot.conf I added the following for debugging: auth_verbose = yes auth_verbose_passwords = no auth_debug = yes auth_debug_passwords = yes mail_debug = yes verbose_ssl = yes I am trying to connect using Thunderbird. Config: IMAP port 993 / SSL/TLS / Normal Password / username : ke...@keiththewebguy.com and password that is in /etc/dovecot/dovecot-users SMTP port 25 / STARTTLS / Normal Password / username : ke...@keiththewebguy.com and password that is in /etc/dovecot/dovecot-users Thunderbird tests these configurations and reports them as found on the server. When I press the done button to create the email account I get a message "Unable to log into the server. Probably wrong configuration, username, or password.". The output when trying to create the Thunderbird account: /var/log/mail.log === Jul 5 18:58:35 soho dovecot: imap-login: Debug: SSL: where=0x10, ret=1: before SSL initialization Jul 5 18:58:35 soho dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: before SSL initialization Jul 5 18:58:35 soho dovecot: imap-login: Debug: SSL: where=0x2002, ret=-1: before SSL initialization Jul 5 18:58:35 soho dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: before SSL initialization Jul 5 18:58:35 soho dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3/TLS read client hello Jul 5 18:58:35 soho dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3/TLS write server hello Jul 5 18:58:35 soho dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3/TLS write certificate Jul 5 18:58:35 soho dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3/TLS write key exchange Jul 5 18:58:35 soho dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3/TLS write server done Jul 5 18:58:35 soho dovecot: imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3/TLS write server done Jul 5 18:58:35 soho dovecot: message repeated 2 times: [ imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3/TLS write server done] Jul 5 18:58:35 soho dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3/TLS write server done Jul 5 18:58:35 soho dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3/TLS read client key exchange Jul 5 18:58:35 soho dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3/TLS read change cipher spec Jul 5 18:58:35 soho dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3/TLS read finished Jul 5 18:58:35 soho dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3/TLS write session ticket Jul 5
Re: Postfix / Dovecot SASL
% postconf -A (SASL support in the SMTP+LMTP client) I might be wrong, but I think that part of the document is actually referring to the LMTP protocol in general and not necesarily to Dovecot's LMTPD server. https://en.wikipedia.org/wiki/Local_Mail_Transfer_Protocol Cheers, K.
Re: Postconf and postmap in check_policy_service scripts
Matus UHLAR - fantomas: I was curious if I could do a script that would do the same, with the same possible issues. I can do perl, but it looks neither python nor perl have interface to postfix what could e.g. expand maps without calling external commands. Among other things, it mainly acted as a proxy between Postfix and Dovecot's quota-status to make sure that the quota query was done for a real user instead of an alias. One solution is when the table is in a real database (sql, etc.) then you could use perl/pythobn/etc bindings. Accessing Berkeley DB from perl or python may be possible but they should adhere to the locking protocols that Postfix uses, typically FCNTL-style shared locks for reading. Wietse I was told it will be migrated to a real database but since it is used only internally it wasn't high on the priorities list :) . Cheers, K.
Re: Postconf and postmap in check_policy_service scripts
Hi Viktor, Thank you for the suggestion. Are there any other general areas that I should be looking out for in this kind of situations? Cheers, K. It appears that some care has been taken to do it right. In principle something like this should be sufficient. You'll need to review the code and API documentation in detail to convince yourself that there are no loose ends.
Re: Postconf and postmap in check_policy_service scripts
Hi Wietse, Thank you for the detailed explanation. This will limit scalability, but can work with low request rates. However, there is an inherent danger to using arbitrary email addresses from the internet in a shell command line. Depending on how the commands are run, there may be shell command injection opportunities when an email address contains semicolon, backslash, quote, or other special characters. Postfix does not allow addresses that start with '-'. That's what I was afraid of. The script is a Python script and it is executed as user nobody through Postfix's spawn service, whenever check_policy_service needs it. From what I can see postconf and postmap are called using Python's subprocess.Popen, like so: subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, encoding='utf-8', shell=False) where: args = ['/usr/sbin/postconf', '-xh', 'virtual_alias_maps'] and args = ['/usr/sbin/postmap', '-q', 'recipient@from-postfix-check-policy-service-call', 'hash:/etc/postfix/virtual_aliases'] With shell=False and assuming that Python doesn't have some nasty bug in this area, is it safe to assume that shell command injection would not be possible in this case? A famous example of shell command injection was CVE-1999-0067 (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-1999-0067) which took input from the internet and pasted it into a shell command line. For an exploit, see https://xkcd.com/327/ - the specific case is for an SQL-based server, but it is easily adapted to UNIX shell. Wietse That comic is priceless :) Cheers, K.
Postconf and postmap in check_policy_service scripts
Hello everybody, On one of our internal Postfix system I noticed that one of the check_policy_service script is using postconf and postmap to perform some alias lookups. It uses postconf to get the virtual_alias_maps parameter, which is then used by postmap to perform the lookups. Now, the load on the system is relatively low, and everything seems to be working quite well. Only reading is involved. But it still got me thinking. Could there be any hidden, unexpected behavior when using them this way? How about if the load gets higher? Were they designed from the start to handle higher loads? Cheers, K.
Re: "Authentication-Results" header order
Super. Thank you for all the info :) Cheers, Kevin On 28/06/2021 00:04, David Bürgin wrote: Kevin N.: Milters decide themselves where they want to insert headers, by index. Depending on the order in which milters run, insertion done by one milter can shift the insertion point of the next milter. The sendmail milter API that milters use to insert headers has a bit of an oddity when using index 0 and 1 to insert: Index 0 inserts *before* the MTA’s ‘Received’ header, index 1 *after*. When all milters use index 1, headers will be inserted in (reverse) order after the ‘Received’ header. However, when just one milter uses index 0, all subsequent milters using index 1 also insert *before* the MTA’s ‘Received’ header. (For details see doc for ‘smfi_insheader’.) This is what I would guess is happening in your case. I definitely need to take a closer look at the 'smfi_insheader' docs. I forgot the main bit of my explanation. So: If your spf-milter inserts at index 0 and your dkim-milter inserts at index 1, then the header order behaviour that you showed is exactly as expected. By the way, RFC 8601 says that ‘Authentication-Results’ headers should be inserted *before* the MTA’s ‘Received’ header. I totally missed this part while I was skimming through the RFC. So, just to make sure that I understand this correctly, the order of the "Authentication-Results" headers do matter. Correct? RFC 8601 seems to give significance to the relative ordering of ‘Authentication-Results’ and ‘Received’ headers. If it is OpenDKIM you’re talking about, you may be interested in this recent change request to fix this and make it consistent: https://github.com/trusteddomainproject/OpenDKIM/pull/126 Yes, I was talking about OpenDKIM. I forgot to mention that in my initial mail. I'll take a look at the pull request. Thanks for pointing this out :) Personally I prefer to do SPF before DKIM. Because SPF looks at envelope information, which comes before the data, it seems more logical to check that first. This actually makes a lot of sense now that you mentioned it :) . But in this case, can there be a situation in which the "Authentication-Results" header added by the SPF check could mess up the DKIM signature check? From what I read, in certain situations, milters running before the milter that does the DKIM check, could add headers that would mess up the DKIM signature check. Is it safe to assume that the "Authentication-Results" header added by the SPF check is *not* such a case? Or am I misunderstanding this completely :) ? I hadn’t thought about this in detail but checked quickly. RFC 6376, sections 5.4.1 and 5.4.2 makes it clear that this is not a problem. Cheers,
Re: "Authentication-Results" header order
Hi David, Thank you for the detailed explanation. Milters decide themselves where they want to insert headers, by index. Depending on the order in which milters run, insertion done by one milter can shift the insertion point of the next milter. The sendmail milter API that milters use to insert headers has a bit of an oddity when using index 0 and 1 to insert: Index 0 inserts *before* the MTA’s ‘Received’ header, index 1 *after*. When all milters use index 1, headers will be inserted in (reverse) order after the ‘Received’ header. However, when just one milter uses index 0, all subsequent milters using index 1 also insert *before* the MTA’s ‘Received’ header. (For details see doc for ‘smfi_insheader’.) This is what I would guess is happening in your case. I definitely need to take a closer look at the 'smfi_insheader' docs. By the way, RFC 8601 says that ‘Authentication-Results’ headers should be inserted *before* the MTA’s ‘Received’ header. I totally missed this part while I was skimming through the RFC. So, just to make sure that I understand this correctly, the order of the "Authentication-Results" headers do matter. Correct? If it is OpenDKIM you’re talking about, you may be interested in this recent change request to fix this and make it consistent: https://github.com/trusteddomainproject/OpenDKIM/pull/126 Yes, I was talking about OpenDKIM. I forgot to mention that in my initial mail. I'll take a look at the pull request. Thanks for pointing this out :) Personally I prefer to do SPF before DKIM. Because SPF looks at envelope information, which comes before the data, it seems more logical to check that first. This actually makes a lot of sense now that you mentioned it :) . But in this case, can there be a situation in which the "Authentication-Results" header added by the SPF check could mess up the DKIM signature check? From what I read, in certain situations, milters running before the milter that does the DKIM check, could add headers that would mess up the DKIM signature check. Is it safe to assume that the "Authentication-Results" header added by the SPF check is *not* such a case? Or am I misunderstanding this completely :) ? Cheers, Kevin
Re: "Authentication-Results" header order
Thank you for the clarification, Cheers, Kevin On 27/06/2021 02:36, Wietse Venema wrote: Kevin N.: 3. From what I've read, the milters are called in the order they are specified. But does that mean that for each SMTP event Postfix will call the milters in the specified order? Yes. Each event is given to the first milter, then the second milter, and so on. Not: all events to the first milter, then all events to the second milter, and so on. Wietse
"Authentication-Results" header order
Hello everybody, I am using two milters to check incoming mail for DKIM signatures and SPF records. They are specified in main.cf using the "smtpd_milters" parameter. Now, when I place the DKIM milter before the SPF milter, like so: smtpd_milters = inet:dkim-milter-host:port, inet:spf-milter-host:port the final delivered message headers will look like: Received: from ... Authentication-Results: ... Received: from ... Authentication-Results: ... Authentication-Results: auth=pass (login) (note the "Received" header between the two "Authentication-Results" headers) When I place the SPF milter before the DKIM milter, like so: smtpd_milters = inet:spf-milter-host:port, inet:dkim-milter-host:port the final delivered message headers will look like: Received: from ... Authentication-Results: ... Authentication-Results: ... Received: from ... Authentication-Results: auth=pass (login) (no "Received" header between the two "Authentication-Results" headers) 1. Is there a situation in which the order of the "Authentication-Results" header matters? I tend to think not, since the ones set by the remote MTA and the ones set by my milter should be distinguishable based on the "authserv-id" field. Is this correct? 2. For incoming mail, I like to place the DKIM milter first, before any other milter has the chance to change relevant headers. But I think in this particular case it would not matter if SPF is performed before DKIM, since as far as I know the Authentication-Results header is generally not included in the DKIM signature. So basically the SPF authentication header added by my milter should not affect the DKIM signature check on the incoming message. Is this correct? 3. From what I've read, the milters are called in the order they are specified. But does that mean that for each SMTP event Postfix will call the milters in the specified order? Or does it mean that it will call and wait until the first milter finishes processing all SMTP events and then it moves on to the next milter from the list? As far as I can tell it is the first case (otherwise, i guess that in my particular case, when the SPF milter is placed after the DKIM milter this should be reflected in the order of the auth results headers. But in my case the SPF auth results header is always places before the DKIM auth results header). I'm not sure the second case would even make sense with the SMTP protocol :) . Do I understand this correctly? Cheers, Kevin.
