Re: SMTP over TLS fails only when using local email clients (logs attached)

[Bill Cole]

On 6 Sep 2018, at 22:06 (-0400), eaerhaerhaehae aehraerhaeha wrote:

I can send emails over port 465 using smtper.net  just fine. It's the 
clients (thunderbird, k-9,..) that cause an error when there is 
supposed to be EHLO.
STARTTLS works perfectly for both, dovecot and postfix. TLS works 
perfectly for dovecot. Only postfix TLS is giving me trouble.

What could be the problem here?
Thanks!

 
NOT WORKING CONNECTION FROM MY PC/PHONE ("Thunderbird failed to find 
the settings for your email account.")


That sounds like a TBird problem. Postfix has nothing to do with 
providing settings for TBird or any other MUA.



-

Sep  7 02:42:49 myserver postfix/smtpd[20128]: 
xsasl_dovecot_server_mech_filter: keep mechanism: PLAIN
Sep  7 02:42:49 myserver postfix/smtpd[20128]: 
xsasl_dovecot_server_mech_filter: keep mechanism: LOGIN
Sep  7 02:42:49 myserver postfix/smtpd[20128]: < my.isp.com[1.2.3.4]: 
???
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ??? ~? connect
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ??? ~? get
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ??? ~? post
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_list_match: ???: 
no match
Sep  7 02:42:49 myserver postfix/smtpd[20128]: > my.isp.com[1.2.3.4]: 
502 5.5.2 Error: command not recognized
Sep  7 02:42:49 myserver postfix/smtpd[20128]: < my.isp.com[1.2.3.4]: 
??
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ? ~? connect
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ? ~? get
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ? ~? post
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_list_match: ?: 
no match
Sep  7 02:42:49 myserver postfix/smtpd[20128]: > my.isp.com[1.2.3.4]: 
502 5.5.2 Error: command not recognized
Sep  7 02:42:49 myserver postfix/smtpd[20128]: < my.isp.com[1.2.3.4]: 
?
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ? ~? connect
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ? ~? get
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ? ~? post
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_list_match: ?: 
no match
Sep  7 02:42:49 myserver dovecot: imap-login: Aborted login (no auth 
attempts in 1 secs): user=<>, rip=1.2.3.4, lip=my.server.ip.here, TLS, 
session=<2ZdzST117MJXe3ri>


This looks like one of 2 common problems:

1. The MUA is trying to use immediate TLS ("smtps" or "wrappermode" in 
postfix-ese) on port 25 or 587, rather than on port 465, which is the 
only place where it is usable.


2. You have a very dumb firewall (e.g. Cisco ASA or ancient Cisco PIX) 
misconfigured to "protect" your mail server.


This looks MUCH more like (1) to me...

Solution: fix your client settings. Don't use wrappermode on anything 
but port 465 *configured* for wrappermode.




WORKING CONNECTION FROM SMTPER

[...]
Sep  7 02:46:52 myserver postfix/smtpd[20169]: < 
ns513574.ip-192-99-9.net[192.99.9.142]: AUTH login 
ZW1haWxAbXktZW1haWwtc2VydmVyLmNvbQ==
Sep  7 02:46:52 myserver postfix/smtpd[20169]: 
xsasl_dovecot_server_first: sasl_method login, init_response 
ZW1haWxAbXktZW1haWwtc2VydmVyLmNvbQ==
Sep  7 02:46:52 myserver postfix/smtpd[20169]: 
xsasl_dovecot_handle_reply: auth reply: CONT?1?UGFzc3dvcmQ6
Sep  7 02:46:52 myserver postfix/smtpd[20169]: > 
ns513574.ip-192-99-9.net[192.99.9.142]: 334 UGFzc3dvcmQ6
Sep  7 02:46:52 myserver postfix/smtpd[20169]: < 
ns513574.ip-192-99-9.net[192.99.9.142]: MTIzNDU=
Sep  7 02:46:52 myserver postfix/smtpd[20169]: 
xsasl_dovecot_handle_reply: auth reply: 
OK?1?user=em...@my-email-server.com?

-


If you didn't munge the above to make it look like you use a supremely 
bad password, you need to stop using such a supremely bad password...



For further assistance, you should provide the information noted in the 
last section Postfix DEBUG_README documentation.




--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Currently Seeking Steadier Work: https://linkedin.com/in/billcole

Re: SMTP over TLS fails only when using local email clients (logs attached)

[Miwa Susumu]

Hi.

2018-09-07 11:06 GMT+09:00 eaerhaerhaehae aehraerhaeha :
> I can send emails over port 465 using smtper.net  just fine. It's the clients 
> (thunderbird, k-9,..) that cause an error when there is supposed to be EHLO.
> STARTTLS works perfectly for both, dovecot and postfix. TLS works perfectly 
> for dovecot. Only postfix TLS is giving me trouble.
> What could be the problem here?
> Thanks!
>

Can you capture e-mail packets from client to server ? (use wireshark etc...)

-- 
miwarin

SMTP over TLS fails only when using local email clients (logs attached)

[eaerhaerhaehae aehraerhaeha]

I can send emails over port 465 using smtper.net  just fine. It's the clients 
(thunderbird, k-9,..) that cause an error when there is supposed to be EHLO.
STARTTLS works perfectly for both, dovecot and postfix. TLS works perfectly for 
dovecot. Only postfix TLS is giving me trouble.
What could be the problem here?
Thanks!

 
NOT WORKING CONNECTION FROM MY PC/PHONE ("Thunderbird failed to find the 
settings for your email account.")
-

Sep  7 02:42:49 myserver postfix/smtpd[20128]: 
xsasl_dovecot_server_mech_filter: keep mechanism: PLAIN
Sep  7 02:42:49 myserver postfix/smtpd[20128]: 
xsasl_dovecot_server_mech_filter: keep mechanism: LOGIN
Sep  7 02:42:49 myserver postfix/smtpd[20128]: < my.isp.com[1.2.3.4]: ???
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ??? ~? connect
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ??? ~? get
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ??? ~? post
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_list_match: ???: no match
Sep  7 02:42:49 myserver postfix/smtpd[20128]: > my.isp.com[1.2.3.4]: 502 5.5.2 
Error: command not recognized
Sep  7 02:42:49 myserver postfix/smtpd[20128]: < my.isp.com[1.2.3.4]: ??
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ? ~? connect
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ? ~? get
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ? ~? post
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_list_match: ?: no match
Sep  7 02:42:49 myserver postfix/smtpd[20128]: > my.isp.com[1.2.3.4]: 502 5.5.2 
Error: command not recognized
Sep  7 02:42:49 myserver postfix/smtpd[20128]: < my.isp.com[1.2.3.4]: ?
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ? ~? connect
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ? ~? get
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_string: 
smtpd_forbidden_commands: ? ~? post
Sep  7 02:42:49 myserver postfix/smtpd[20128]: match_list_match: ?: no match
Sep  7 02:42:49 myserver dovecot: imap-login: Aborted login (no auth attempts 
in 1 secs): user=<>, rip=1.2.3.4, lip=my.server.ip.here, TLS, 
session=<2ZdzST117MJXe3ri>
Sep  7 02:42:54 myserver postfix/smtpd[20128]: > my.isp.com[1.2.3.4]: 502 5.5.2 
Error: command not recognized
Sep  7 02:42:59 myserver postfix/smtpd[20128]: > my.isp.com[1.2.3.4]: 421 4.7.0 
mx.my-email-server.com Error: too many errors
-
 

WORKING CONNECTION FROM SMTPER
-

Sep  7 02:46:51 myserver postfix/smtpd[20169]: 
xsasl_dovecot_server_mech_filter: keep mechanism: PLAIN
Sep  7 02:46:51 myserver postfix/smtpd[20169]: 
xsasl_dovecot_server_mech_filter: keep mechanism: LOGIN
Sep  7 02:46:51 myserver postfix/smtpd[20169]: < 
ns513574.ip-192-99-9.net[192.99.9.142]: EHLO SkyWeb
Sep  7 02:46:51 myserver postfix/smtpd[20169]: match_list_match: 
ns513574.ip-192-99-9.net: no match
Sep  7 02:46:51 myserver postfix/smtpd[20169]: match_list_match: 192.99.9.142: 
no match
Sep  7 02:46:51 myserver postfix/smtpd[20169]: > 
ns513574.ip-192-99-9.net[192.99.9.142]: 250-mx.my-email-server.com
Sep  7 02:46:51 myserver postfix/smtpd[20169]: > 
ns513574.ip-192-99-9.net[192.99.9.142]: 250-PIPELINING
Sep  7 02:46:51 myserver postfix/smtpd[20169]: > 
ns513574.ip-192-99-9.net[192.99.9.142]: 250-SIZE 1
Sep  7 02:46:51 myserver postfix/smtpd[20169]: > 
ns513574.ip-192-99-9.net[192.99.9.142]: 250-ETRN
Sep  7 02:46:51 myserver postfix/smtpd[20169]: > 
ns513574.ip-192-99-9.net[192.99.9.142]: 250-AUTH PLAIN LOGIN
Sep  7 02:46:51 myserver postfix/smtpd[20169]: > 
ns513574.ip-192-99-9.net[192.99.9.142]: 250-AUTH=PLAIN LOGIN
Sep  7 02:46:51 myserver postfix/smtpd[20169]: > 
ns513574.ip-192-99-9.net[192.99.9.142]: 250-ENHANCEDSTATUSCODES
Sep  7 02:46:51 myserver postfix/smtpd[20169]: > 
ns513574.ip-192-99-9.net[192.99.9.142]: 250-8BITMIME
Sep  7 02:46:51 myserver postfix/smtpd[20169]: > 
ns513574.ip-192-99-9.net[192.99.9.142]: 250 DSN
Sep  7 02:46:52 myserver postfix/smtpd[20169]: < 
ns513574.ip-192-99-9.net[192.99.9.142]: AUTH login 
ZW1haWxAbXktZW1haWwtc2VydmVyLmNvbQ==
Sep  7 02:46:52 myserver postfix/smtpd[20169]: xsasl_dovecot_server_first: 
sasl_method login, init_response ZW1haWxAbXktZW1haWwtc2VydmVyLmNvbQ==
Sep  7 02:46:52 myserver postfix/smtpd[20169]: xsasl_dovecot_handle_reply: auth 
reply: CONT?1?UGFzc3dvcmQ6
Sep  7 02:46:52 myserver postfix/smtpd[20169]: > 
ns513574.ip-192-99-9.net[192.99.9.142]: 334 UGFzc3dvcmQ6
Sep  7 02:46:52 myserver postfix/smtpd[20169]: < 
ns513574.ip-192-99-9.net[192.99.9.142]: MTIzNDU=
Sep  7 02:46:52 myserver 

Re: What is Postfix telling me?

[Bastian Blank]

On Thu, Sep 06, 2018 at 05:04:43PM -0400, James B. Byrne wrote:
> Sep  6 12:36:42 mx31 postgrey[85107]: action=pass, reason=client AWL,
> client_name=malton22-1176258451.sdsl.bell.ca,
> client_address=70.28.71.147, sender=c...@airportcargo.ca,
> recipient=impo...@harte-lyne.ca

This is from postgrey, which is not postfix.  Disable it if you don't
understand what is does.

> Sep  6 12:36:48 mx31 postfix-p25/smtpd[66636]: proxy-reject:
> END-OF-MESSAGE: 451 4.5.0 Error in processing, id=29937-07, quar+notif
> FAILED: mail_dispatch: no recognized protocol name: -2 at
> /usr/local/sbin/amavisd line 9638.; from=
> to= proto=ESMTP helo=

This is a postfix log message, but providing the output of an amavis.
You missconfigured it, search for -2.  Disable if you don't understand
what it does.

> If someone could clue me in as to what is happening then I would be
> most grateful.

Check your complete log.  Read
http://www.postfix.org/DEBUG_README.html#mail

Regards,
Bastian

-- 
It is more rational to sacrifice one life than six.
-- Spock, "The Galileo Seven", stardate 2822.3

What is Postfix telling me?

[James B. Byrne]

Starting shortly after midnight 20180906 our maillog file began to
record this sort of message pair every six minutes or so.

Sep  6 12:36:42 mx31 postgrey[85107]: action=pass, reason=client AWL,
client_name=malton22-1176258451.sdsl.bell.ca,
client_address=70.28.71.147, sender=c...@airportcargo.ca,
recipient=impo...@harte-lyne.ca

Sep  6 12:36:48 mx31 postfix-p25/smtpd[66636]: proxy-reject:
END-OF-MESSAGE: 451 4.5.0 Error in processing, id=29937-07, quar+notif
FAILED: mail_dispatch: no recognized protocol name: -2 at
/usr/local/sbin/amavisd line 9638.; from=
to= proto=ESMTP helo=

We are not getting the error message for any other domain and we do
not get it for every message from airportcargo.ca.  For example:

Sep  6 15:06:21 mx31 postgrey[85107]: action=pass, reason=client AWL,
client_name=toroondcmxzomta01.bellnexxia.net,
client_address=67.69.168.80, sender=c...@airportcargo.ca,
recipient=impo...@harte-lyne.ca

Sep  6 15:06:21 mx31 policyd-spf[68870]: prepend X-Comment: SPF
skipped for whitelisted relay domain - client-ip=67.69.168.80;
helo=toroondcmxzomta01-srv.bellnexxia.net;
envelope-from=c...@airportcargo.ca; receiver=

Sep  6 15:06:22 mx31 postfix/qmgr[79845]: E64931EBF7:
from=, size=3786, nrcpt=1 (queue active)
Sep  6 15:06:22 mx31 postfix-p25/smtpd[64693]: proxy-accept:
END-OF-MESSAGE: 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0
Ok: queued as E64931EBF7; from=
to= proto=ESMTP
helo=


Since SPF appears to figure in the successful receipt I checked the
DNS RR:

drill airportcargo.ca txt

;; ANSWER SECTION:
airportcargo.ca.11072   IN  TXT
"v=spf1 a mx include:mail.airportcargo.ca include:airportcargo.ca
include:home.zetwork.ca ~all"

drill airportcargo.ca mx

;; ANSWER SECTION:
airportcargo.ca.4552 IN MX 30 lastmx.spamexperts.net.
airportcargo.ca.4552 IN MX 20 fallbackmx.spamexperts.eu.
airportcargo.ca.4552 IN MX 10 mx.spamexperts.com.

But this only tells me that any SPF failure for airportcargo.ca
messages should be treated as a softfail.  Our policyd-spf.conf has
these options set:

HELO_reject = Fail - Reject on HELO Fail
Mail_From_reject = Fail
Domain_Whitelist = bellnexxia.net,lcbo.com

Which, to me, indicates that mail arriving via bellnexxia.net is not
checked for SPF compliance or at least messages delivered by that
route do not fail regardless of the SPF settings for the sender's
domain.

If someone could clue me in as to what is happening then I would be
most grateful.

-- 
***  e-Mail is NOT a SECURE channel  ***
Do NOT transmit sensitive data via e-Mail
 Do NOT open attachments nor follow links sent by e-Mail

James B. Byrnemailto:byrn...@harte-lyne.ca
Harte & Lyne Limited  http://www.harte-lyne.ca
9 Brockley Drive  vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada  L8E 3C3

Re: Patch: eliminate postfix-script warnings about symlinks

[Viktor Dukhovni]

> On Sep 6, 2018, at 2:19 PM, Luc Pardon  wrote:
> 
> However, although symlinks inside the Postfix dirs were not needed in
> the past, that has changed by now. They have become necessary because
> OpenSSL needs them to find its certificates, so we can't just tell the
> admin to get rid of them.

There is no need to put your certificate trust store in /etc/postfix/
What's wrong with /etc/ssl/postfix/ or similar?  The /etc/postfix
directory does not presently support symlinks.

-- 
Viktor.

Re: strict_rfc821_envelopes possibly broken on postfix-3.3.1

[Tuomo Soini]

On Wed, 5 Sep 2018 13:05:02 -0400 (EDT)
Wietse Venema  wrote:

> Your observation is incorrect. I just did a test with
> 
> mail from:wietse

Thank you, I digged the issue deeper and found out pre-queue content
filter which postfix runs as proxy setup was too picky. Problem
resolved by patching the smtp filter software to be as non-strict as
postfix and now things work properly with these broken clients.

-- 
Tuomo Soini 
Foobar Linux services
+358 40 5240030
Foobar Oy 

Re: Patch: eliminate postfix-script warnings about symlinks

[Luc Pardon]

On 05-09-18 18:18, Wietse Venema wrote:
> Luc Pardon:
>> The first question is obviously: can we disallow symlinks to the outside
>> world by definition? I'd say the answer is yes, but $(whoami) ?
> 
> Here is some background on pathname safety.
> 

Thanks for that. Also, the paper - at a first cursory reading - is very
helpful, and thought-provoking as well.

But we have to be clear about the problem that we are trying to solve
here in this particular case.

So allow me to re-iterate for a moment.

* My problem is that, after pointing "smtpd_tls_CApath" and
"smtp_tls_CApath" to a subdir of /etc/postfix, and installing a bunch of
root CA certificates in it, my logs are flooded with something like 150
(!) bogus warnings about the symlinks to these certs having the wrong
permission.

* You correctly observed that my proposed "-L" patch does too much: it
suppresses _all_ warnings about _all_ symlinks, safe or not. I do agree
that this is an unacceptable loss of functionality.

However, you propose to compensate for it by adding - necessarily
elaborate - safety checks on symlinks.

I don't think we need to go that far.


For starters, I'd venture to say that, in the past, symlinks inside the
safe_for_X Postfix dirs were never needed nor used in the first place.

Neither were they expected, it seems. In any case the postfix-script
doesn't "see" that it is dealing with a symlink. It treats it just like
a regular file - and trips over its permissions (which are 0777 by
definition, hence wrong by definition inside a safe_for_X dir).

That also means that _all_ symlinks get warned about - as I found out.

The only way for the admin to get rid of such a warning is to remove the
symlink altogether, since, unlike a regular file, chmod won't help here.

Conclusion: any symlink, that happened to trespass into Postfix
territory, must in the past have been systematically removed by those
admins that read their logs and heed the warnings (meaning, of course,
_all_ admins ).



However, although symlinks inside the Postfix dirs were not needed in
the past, that has changed by now. They have become necessary because
OpenSSL needs them to find its certificates, so we can't just tell the
admin to get rid of them.

Also, with initiatives like "Let's Encrypt" and "STARTTLS Everywhere"
gaining ground, more and more Postfix users will start making real use
of certificates, and they too will eventually start complaining about
the bogus warnings - or worse: ignore them, together with the important
ones.


We could of course sidestep the issue by patching the docs, telling
people to store their certificates _outside_ /etc/postfix.

However, that doesn't feel right.

One reason is that, if they are used for handling the mail, they are
effectively part of the Postfix config, and /etc/postfix is their proper
home.

Also, those of us who run SELinux would have to invent and add some
custom rules, just to allow Postfix to access those certificates and
their dirs. That is cumbersome, tricky to get it right, and - above all
- not needed when we store them in /etc/postfix, where they belong.



Fortunately, I think that we can remain safe by continuing to warn about
symlinks, much as before, except that we would from now on remain silent
about symlinks that:

 a) live inside a "safe_for_X" directory structure, and
 b) point to a target inside that very same dir structure.

And by "safe_for_X dir structure", I do mean of course one of the dirs
that are monitored by postfix-script.

Such symlinks are inherently safe_for_X, simply because they point to a
safe_for_X target. No need to pick them apart to find that out.

We can indeed rely on the target being safe, because if it is not, the
regular file check of postfix-script would issue a "wrong permission"
warning for this target, and the admin would step in to fix that with
chmod. And when the target is made safe, the symlink pointing to it
automatically becomes safe as well.

Case in point: the OpenSSL symlinks happen to live in the same dir as
the certificate they point to, and the certs themselves are monitored by
postfix-script because they are in $config_directory.



It is a different story with symlinks that point to a target  _outside_
one of the "safe_for_X" Postfix dirs. Here we know nothing about the
target, so we would normally have to go out and investigate closely - as
you propose to do.

However, such symlinks, pointing all over the place, are not needed
today, no more than they were needed yesterday. So we can continue to
issue a warning as we did before, and thereby incite the admin to remove
them, as before.

It doesn't matter whether they are good or bad or ugly. They don't
belong in there and they must go, period.


Also, I am not convinced that the elaborate checks would have much added
value. Quite to the contrary - at least for this particular case of Postfix.

For example, if I understand it correctly, a symlink somewhere inside
the safe_for_root