Re: problem with protection.outlook.com released spam getting bounced

[Viktor Dukhovni]

> On Apr 10, 2017, at 4:01 PM, John Stoffel  wrote:
> 
> Since I built 2.11.9 by hand, I'm willing to do this hack as well I
> think.  It's a total hack too... and I'm still amazed I'm the only one
> seeing this.  But maybe most people who use Office365 for spam
> filtering also use Exchange on the inside and not the setup we have
> which is due to Lotus Notes, Mailman and Exchange all being valid
> internal targets.  Whee...
> 
> Looks like I just need to hack src/global/mail_copy.c and
> src/global/delivered_hrd.c, or better yet, just change the
> global/header_opts.c to maybe be something like this:
> 
> diff -ur postfix-2.11.9/src/global/header_opts.c 
> postfix-2.11.9.jfs/src/global/header_opts.c
> --- postfix-2.11.9/src/global/header_opts.c 2017-04-10 12:50:34.381884494 
> -0700
> +++ postfix-2.11.9.jfs/src/global/header_opts.c 2008-05-08 13:41:35.0 
> -0700
> @@ -62,7 +62,7 @@
> "Content-Length", HDR_CONTENT_LENGTH, HDR_OPT_DROP,
> "Content-Transfer-Encoding", HDR_CONTENT_TRANSFER_ENCODING, HDR_OPT_MIME,
> "Content-Type", HDR_CONTENT_TYPE, HDR_OPT_MIME,
> -"Delivered-To", HDR_DELIVERED_TO, 0,
> +"My-Delivered-To", HDR_DELIVERED_TO, 0,
> "Disposition-Notification-To", HDR_DISP_NOTIFICATION, HDR_OPT_SENDER,
> "Date", HDR_DATE, 0,
> "Errors-To", HDR_ERRORS_TO, HDR_OPT_SENDER,
> 
> 
> Which certainly can't hurt as a test.

All three need to be changed to a consistent replacement:

src/global/header_opts.c:"Delivered-To", HDR_DELIVERED_TO, 0,
src/global/mail_copy.c: vstream_fprintf(dst, "Delivered-To: %s%s", 
vstring_str(buf), eol);
src/local/forward.c:rec_fprintf(info->cleanup, REC_TYPE_NORM, 
"Delivered-To: %s",

-- 
Viktor.

Re: Postfix error message to Postmaster

[Juri Haberland]

On 10.04.2017 18:33, James B. Byrne wrote:
> 
> On Mon, April 10, 2017 11:29, Wietse Venema wrote:
>> James B. Byrne:

>>> The issue seems to be some sort of time-out with the Amavis proxy.

>>> 66  Apr  8 10:50:27 inet08 postfix-p25/smtpd[18374]: warning:
>>> timeout talking to proxy 127.0.0.1:10024
>>> 67  Apr  8 10:50:27 inet08 postfix-p25/smtpd[18374]:
>>> proxy-reject:
>>> END-OF-MESSAGE: 451 4.3.0 Error: queue file write error; from=
>>> to= proto=ESMTP helo=
>>
>> The spam proxy is taking a long time to do something. To find out what
>> that is, see the spam proxy's log.

> This is all that I see in /var/log/spamd for the relevant time period.
> 
> # cat /var/log/spamd.log-20170409
> . . .
> Sat Apr  8 00:06:25 2017 [3845] info: spamd: server started on port
> 783/tcp (running version 3.3.1)

You are talking about amavis - this is spamassassin. Please check what you
have configured.

Is amavis running (ps aux|grep amavis)?
Does amavis listen on port 10024 (netstat -lntp|grep 10024)?
What does your mail.log say about amavis?

  Juri

Re: problem with protection.outlook.com released spam getting bounced

[John Stoffel]

> "Viktor" == Viktor Dukhovni  writes:

>> On Apr 4, 2017, at 5:26 PM, John Stoffel  wrote:
>> 
>> But I only want this replamcent to happen for email that comes from a
>> specific set of outside servers.  I think I might have to run my own
>> milter here to do this.   I really can't depend on the headers not
>> being forged somehow, but I can depend on the host which connects to
>> me being who it says.
>> 
>> To a degree.
>> 
>> Am I making sense?

Viktor> Yes.  Another option is to build a custom version of Postfix in which
Viktor> the Delivered-To: header is renamed.  Perhaps this should be 
configurable,
Viktor> and maybe even forgery-protected, but an obscure alternative header 
would
Viktor> suffice, if you're willing to run Postfix binaries built on site.

Since I built 2.11.9 by hand, I'm willing to do this hack as well I
think.  It's a total hack too... and I'm still amazed I'm the only one
seeing this.  But maybe most people who use Office365 for spam
filtering also use Exchange on the inside and not the setup we have
which is due to Lotus Notes, Mailman and Exchange all being valid
internal targets.  Whee...

Looks like I just need to hack src/global/mail_copy.c and
src/global/delivered_hrd.c, or better yet, just change the
global/header_opts.c to maybe be something like this:

diff -ur postfix-2.11.9/src/global/header_opts.c 
postfix-2.11.9.jfs/src/global/header_opts.c
--- postfix-2.11.9/src/global/header_opts.c 2017-04-10 12:50:34.381884494 
-0700
+++ postfix-2.11.9.jfs/src/global/header_opts.c 2008-05-08 13:41:35.0 
-0700
@@ -62,7 +62,7 @@
 "Content-Length", HDR_CONTENT_LENGTH, HDR_OPT_DROP,
 "Content-Transfer-Encoding", HDR_CONTENT_TRANSFER_ENCODING, HDR_OPT_MIME,
 "Content-Type", HDR_CONTENT_TYPE, HDR_OPT_MIME,
-"Delivered-To", HDR_DELIVERED_TO, 0,
+"My-Delivered-To", HDR_DELIVERED_TO, 0,
 "Disposition-Notification-To", HDR_DISP_NOTIFICATION, HDR_OPT_SENDER,
 "Date", HDR_DATE, 0,
 "Errors-To", HDR_ERRORS_TO, HDR_OPT_SENDER,


Which certainly can't hurt as a test.

John

Re: Postfix error message to Postmaster

[Wietse Venema]

James B. Byrne:
[ Charset ISO-8859-1 converted... ]
> 
> On Mon, April 10, 2017 11:29, Wietse Venema wrote:
> > James B. Byrne:
> >> 66  Apr  8 10:50:27 inet08 postfix-p25/smtpd[18374]: warning:
> >> timeout talking to proxy 127.0.0.1:10024
> >> 67  Apr  8 10:50:27 inet08 postfix-p25/smtpd[18374]:
> >> proxy-reject:
> >> END-OF-MESSAGE: 451 4.3.0 Error: queue file write error; from=
> >> to= proto=ESMTP helo=
> >
> > The spam proxy is taking a long time to do something. To find out what
> > that is, see the spam proxy's log.
> >
> > Wietse
> >
> 
> This is all that I see in /var/log/spamd for the relevant time period.
> 
> # cat /var/log/spamd.log-20170409
> . . .
> Sat Apr  8 00:06:25 2017 [3845] info: spamd: server started on port
> 783/tcp (running version 3.3.1)
> Sat Apr  8 00:06:25 2017 [3845] info: spamd: server pid: 3845
> Sat Apr  8 00:06:25 2017 [3845] info: spamd: server successfully
> spawned child process, pid 3851
> Sat Apr  8 00:06:25 2017 [3845] info: spamd: server successfully
> spawned child process, pid 3852
> Sat Apr  8 00:06:25 2017 [3845] info: prefork: child states: IS
> Sat Apr  8 00:06:25 2017 [3845] info: prefork: child states: II
> Sun Apr  9 03:29:27 2017 [3845] info: spamd: server killed by SIGTERM,
> shutting down
> 
> There are no entries at all between midnight on the 8th and a
> maintenance reboot on the 9th.  Or have I misunderstood your
> reference?

If this is all the logging, then I suggest that you ask the people
who created this product how to get more insight into the handling
of individual mail transactions.

Wietse

Re: Postfix error message to Postmaster

[James B. Byrne]

On Mon, April 10, 2017 11:29, Wietse Venema wrote:
> James B. Byrne:
>> 66  Apr  8 10:50:27 inet08 postfix-p25/smtpd[18374]: warning:
>> timeout talking to proxy 127.0.0.1:10024
>> 67  Apr  8 10:50:27 inet08 postfix-p25/smtpd[18374]:
>> proxy-reject:
>> END-OF-MESSAGE: 451 4.3.0 Error: queue file write error; from=
>> to= proto=ESMTP helo=
>
> The spam proxy is taking a long time to do something. To find out what
> that is, see the spam proxy's log.
>
>   Wietse
>

This is all that I see in /var/log/spamd for the relevant time period.

# cat /var/log/spamd.log-20170409
. . .
Sat Apr  8 00:06:25 2017 [3845] info: spamd: server started on port
783/tcp (running version 3.3.1)
Sat Apr  8 00:06:25 2017 [3845] info: spamd: server pid: 3845
Sat Apr  8 00:06:25 2017 [3845] info: spamd: server successfully
spawned child process, pid 3851
Sat Apr  8 00:06:25 2017 [3845] info: spamd: server successfully
spawned child process, pid 3852
Sat Apr  8 00:06:25 2017 [3845] info: prefork: child states: IS
Sat Apr  8 00:06:25 2017 [3845] info: prefork: child states: II
Sun Apr  9 03:29:27 2017 [3845] info: spamd: server killed by SIGTERM,
shutting down

There are no entries at all between midnight on the 8th and a
maintenance reboot on the 9th.  Or have I misunderstood your
reference?


-- 
***  e-Mail is NOT a SECURE channel  ***
Do NOT transmit sensitive data via e-Mail
 Do NOT open attachments nor follow links sent by e-Mail

James B. Byrnemailto:byrn...@harte-lyne.ca
Harte & Lyne Limited  http://www.harte-lyne.ca
9 Brockley Drive  vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada  L8E 3C3

Re: Postfix error message to Postmaster

[Wietse Venema]

James B. Byrne:
> 66  Apr  8 10:50:27 inet08 postfix-p25/smtpd[18374]: warning:
> timeout talking to proxy 127.0.0.1:10024
> 67  Apr  8 10:50:27 inet08 postfix-p25/smtpd[18374]: proxy-reject:
> END-OF-MESSAGE: 451 4.3.0 Error: queue file write error; from=
> to= proto=ESMTP helo=

The spam proxy is taking a long time to do something. To find out what
that is, see the spam proxy's log.

Wietse

Postfix error message to Postmaster

[James B. Byrne]

We continue to receive messages addressed to Postmaster from our MX
host.  All appear to be related to a single original transmission. 
The issue seems to be some sort of time-out with the Amavis proxy.  
Is there some way of determining exactly what is wrong with this
specific sender's message?

The Postmaster receives this messages:

Transcript of session follows.

 Out: 220 inet08.hamilton.harte-lyne.ca ESMTP Postfix
 In:  EHLO mail-it0-f47.google.com
 Out: 250-inet08.hamilton.harte-lyne.ca
 Out: 250-PIPELINING
 Out: 250-SIZE 2048
 Out: 250-ETRN
 Out: 250-STARTTLS
 Out: 250-ENHANCEDSTATUSCODES
 Out: 250-8BITMIME
 Out: 250 DSN
 In:  STARTTLS
 Out: 220 2.0.0 Ready to start TLS
 In:  EHLO mail-it0-f47.google.com
 Out: 250-inet08.hamilton.harte-lyne.ca
 Out: 250-PIPELINING
 Out: 250-SIZE 2048
 Out: 250-ETRN
 Out: 250-AUTH LOGIN PLAIN
 Out: 250-AUTH=LOGIN PLAIN
 Out: 250-ENHANCEDSTATUSCODES
 Out: 250-8BITMIME
 Out: 250 DSN
 In:  MAIL FROM: SIZE=1701895
 Out: 250 2.1.0 Ok
 In:  RCPT TO:
 Out: 250 2.1.5 Ok
 In:  DATA
 Out: 354 End data with .
 Out: 451 4.3.0 Error: queue file write error

Session aborted, reason: lost connection

For other details, see the local mail logfile

The maillog contains these types of entries:

66  Apr  8 10:50:27 inet08 postfix-p25/smtpd[18374]: warning:
timeout talking to proxy 127.0.0.1:10024
67  Apr  8 10:50:27 inet08 postfix-p25/smtpd[18374]: proxy-reject:
END-OF-MESSAGE: 451 4.3.0 Error: queue file write error; from=
to= proto=ESMTP helo=
68  Apr  8 10:50:27 inet08 postfix/cleanup[18523]: AE20261ED8:
message-id=<20170408145027.ae20261...@inet08.hamilton.harte-lyne.ca>
69  Apr  8 10:50:27 inet08 postfix-p25/smtpd[18374]: disconnect
from mail-it0-f49.google.com[209.85.214.49]
70  Apr  8 10:50:27 inet08 postfix/qmgr[17396]: AE20261ED8:
from=, size=1429,
nrcpt=1 (queue active)
71  Apr  8 10:50:28 inet08 postfix/smtp[18524]: AE20261ED8:
to=,
orig_to=,
relay=inet07.hamilton.harte-lyne.ca[216.185.71.27]:25, delay=0.56,
delays=0.07/0.05/0.32/0.12, dsn=2.0.0, status=sent (250 2.0.0 Ok:
queued as 2FA418A69E)
72  Apr  8 10:50:28 inet08 postfix/qmgr[17396]: AE20261ED8: removed



-- 
***  e-Mail is NOT a SECURE channel  ***
Do NOT transmit sensitive data via e-Mail
 Do NOT open attachments nor follow links sent by e-Mail

James B. Byrnemailto:byrn...@harte-lyne.ca
Harte & Lyne Limited  http://www.harte-lyne.ca
9 Brockley Drive  vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada  L8E 3C3

Re: Postfix TLS/SSL with wildcard SSL certificate

[Viktor Dukhovni]

> On Apr 10, 2017, at 6:02 AM, Selcuk Yazar  wrote:
> 
> I have a wildcard SSL certificate file in pfx format.

More accurately, you have a PKCS#12 file, which contains a password-
protected copy of the private key and the certificate chain.

> I'm little confuse with smtpd_tls_cert_file ,smtpd_tls_key_file settings.

Postfix reads the certificates and private key in PEM format.

> How can I prepare these cert_file and key_file files with openssl
> command.

To place both the private key and the certificate chain in a single file:

   # umask 077
   # openssl pkcs12 -nodes -in /some/where/keypair.pfx \
-out /etc/postfix/certkey.pem.tmp
   # mv /etc/postfix/certkey.pem.tmp /etc/postfix/certkey.pem
   # postconf -e "smtpd_tls_cert_file = /etc/postfix/certkey.pem"

To use separate files:

   # date=$(date "%Y-%m-%d-%H")

   # umask 077
   # key="/etc/postfix/key-${date}.pem"
   # openssl pkcs12 -nodes -nocerts -in /some/where/keypair.pfx -out "$key"

   # umask 022
   # cert="/etc/postfix/cert-${date}.pem"
   # openssl pkcs12 -nodes -nokeys -clcerts -in /some/where/keypair.pfx \
-out /dev/stdout > "$cert"
   # openssl pkcs12 -nodes -nokeys -cacerts -in /some/where/keypair.pfx \
-out /dev/stdout >> "$cert"
   # postconf -e "smtpd_tls_key_file = ${key}" \
 "smtpd_tls_cert_file = ${cert}"

In either case you'll be prompted for the PKCS#12 (aka pfx) file decryption 
password.

-- 
Viktor.

Re: Disabling SMTPUTF8 per destination

[Philip Paeps]

On 2017-04-10 09:51:42 (-0400), Wietse Venema  wrote:
> Philip Paeps:
> > My system is configured with default SMTPUTF8 settings [...]
> > 
> > This works perfectly fine (probably because, sadly, SMTPUTF8 is still
> > quite rare in the wild) except occasionally I'll get an NDR for a
> > locally submitted message:
> > 
> > SMTPUTF8 is required, but was not offered by host
> > 
> > This happens when I "bounce"/"resend" a message with UTF8 in one of the
> > headers.  Pre-SMTPUTF8 Postfix would not care about UTF8 in e.g. From:
> > or Subject: but in the new world order, such messages submitted locally
> > bounce.
> > 
> > I'm cool with that.  The world needs to move on.
> > 
> > Except ... I know that some parts of the world will take a while before
> > they move on.  I couldn't find anything in postconf(5) or in the mailing
> > list archives about disabling SMTPUTF8 per destination.
> 
> The simplest solution is to not enable SMTPUTF8 auto-detection in
> the Postfix sendmail command (smtputf8_autodetect_classes = verify).
> That is the root cause of the problem, after all.

That had occurred to me but it happens rarely enough that I've not been
too bothered (it pretty much only happens when I bounce Asian spam to
spamcop.net).  When it does happen, it reminds me to check if anything
new is happening in the world of SMTPUTF8. :)

If it starts bothering me, I'll do as you suggest.

> > If a per-destination safety net existed, I would likely consider
> > setting ``smtputf8_autodetect_classes`` to all.  If others feel
> > the same, maybe it would advance adoption of SMTPUTF8 in the wild.
> 
> What should happen with TRANSIT EMAIL, after the Postfix SMTP server
> has promised that it will deliver such email according to the rules
> for SMTPUTF8?

Argh.  I see what you mean.  I hadn't considered mail in transit.  I was
thinking only of mail coming in through submission (which, come to think
of it, is actually also in transit).  I should think before writing
email rather than during.

> > Prior art in Postfix is ``smtp_tls_policy_maps`` which allow
> > overriding main.cf TLS settings per destination.
> 
> There is no promise that email received with TLS will be forwarded
> with TLS. With SMTPUTF8 (and 8BITMIME), the receiving MTA actually
> makes such a promise. The MTA is required to respect the promise
> that it made when it announced SMTPUTF8 (or 8BITMIME) support, and
> if it can't keep that promise, to return email as undeliverable.

You are correct of course.  I had forgotten that the SMTPUTF8 promise
applies to the entire message and all headers not simply the envelope.

> Perhaps SMTPUTF8 autodetection could be more granular: UTF8 in the
> envelope is definitely problematic for a receiver that does not
> support SMTPUTF8, while UTF8 in a message header is less so.

An option to accept/forward a message that arrives with SMTPUTF8 but
only has UTF8 in the message headers but not the envelope to a nexthop
that does not support SMTPUTF8 would only "fix" the problem for that one
nexthop and still violates the end-to-end promise of SMTPUTF8.  We can't
(always) know that the nexthop configured like this is the final
destination for the message.

The more I think about this, the more I realise that an option like this
would create a lot more problems than it solves.

The rest of the world just needs to adopt SMTPUTF8. :-)

Philip

-- 
Philip Paeps
Senior Reality Engineer
Ministry of Information

Re: Disabling SMTPUTF8 per destination

[Wietse Venema]

Philip Paeps:
> My system is configured with default SMTPUTF8 settings, i.e.:
> 
> root@rincewind:~ # postconf -d | grep UTF8
> smtputf8_autodetect_classes = sendmail, verify
> smtputf8_enable = ${{$compatibility_level} < {1} ? {no} : {yes}}
> strict_smtputf8 = no
> root@rincewind:~ # postconf -n | grep UTF8
> root@rincewind:~ # postconf compatibility_level
> compatibility_level = 2
> 
> This works perfectly fine (probably because, sadly, SMTPUTF8 is still
> quite rare in the wild) except occasionally I'll get an NDR for a
> locally submitted message:
> 
> SMTPUTF8 is required, but was not offered by host
> 
> This happens when I "bounce"/"resend" a message with UTF8 in one of the
> headers.  Pre-SMTPUTF8 Postfix would not care about UTF8 in e.g. From:
> or Subject: but in the new world order, such messages submitted locally
> bounce.
> 
> I'm cool with that.  The world needs to move on.
> 
> Except ... I know that some parts of the world will take a while before
> they move on.  I couldn't find anything in postconf(5) or in the mailing
> list archives about disabling SMTPUTF8 per destination.

The simplest solution is to not enable SMTPUTF8 auto-detection in
the Postfix sendmail command (smtputf8_autodetect_classes = verify).
That is the root cause of the problem, after all.

> If a per-destination safety net existed, I would likely consider
> setting ``smtputf8_autodetect_classes`` to all.  If others feel
> the same, maybe it would advance adoption of SMTPUTF8 in the wild.

What should happen with TRANSIT EMAIL, after the Postfix SMTP server
has promised that it will deliver such email according to the rules
for SMTPUTF8?

> Prior art in Postfix is ``smtp_tls_policy_maps`` which allow
> overriding main.cf TLS settings per destination.

There is no promise that email received with TLS will be forwarded
with TLS. With SMTPUTF8 (and 8BITMIME), the receiving MTA actually
makes such a promise. The MTA is required to respect the promise
that it made when it announced SMTPUTF8 (or 8BITMIME) support, and
if it can't keep that promise, to return email as undeliverable.

Perhaps SMTPUTF8 autodetection could be more granular: UTF8 in the
envelope is definitely problematic for a receiver that does not
support SMTPUTF8, while UTF8 in a message header is less so.

Wietse

Disabling SMTPUTF8 per destination

[Philip Paeps]

My system is configured with default SMTPUTF8 settings, i.e.:

root@rincewind:~ # postconf -d | grep utf8
smtputf8_autodetect_classes = sendmail, verify
smtputf8_enable = ${{$compatibility_level} < {1} ? {no} : {yes}}
strict_smtputf8 = no
root@rincewind:~ # postconf -n | grep utf8
root@rincewind:~ # postconf compatibility_level
compatibility_level = 2

This works perfectly fine (probably because, sadly, SMTPUTF8 is still
quite rare in the wild) except occasionally I'll get an NDR for a
locally submitted message:

SMTPUTF8 is required, but was not offered by host

This happens when I "bounce"/"resend" a message with UTF8 in one of the
headers.  Pre-SMTPUTF8 Postfix would not care about UTF8 in e.g. From:
or Subject: but in the new world order, such messages submitted locally
bounce.

I'm cool with that.  The world needs to move on.

Except ... I know that some parts of the world will take a while before
they move on.  I couldn't find anything in postconf(5) or in the mailing
list archives about disabling SMTPUTF8 per destination.

If a per-destination safety net existed, I would likely consider setting
``smtputf8_autodetect_classes`` to all.  If others feel the same, maybe
it would advance adoption of SMTPUTF8 in the wild.

Prior art in Postfix is ``smtp_tls_policy_maps`` which allow overriding
main.cf TLS settings per destination.

Any views?  Does a per-destination override exist and did I miss it in
the documentation and the archives?  Has this been discussed before?

Thanks.
Philip

-- 
Philip Paeps
Senior Reality Engineer
Ministry of Information

Postfix TLS/SSL with wildcard SSL certificate

[Selcuk Yazar]

Hi,

I have a wildcard SSL certificate file in pfx format (Include private key
export from Windows Server). I'm little confuse with smtpd_tls_cert_file
,smtpd_tls_key_file settings. How can i prepare these cert_file and
key_file files with openssl command.  Actually  i know how i create private
key file but, i don't understand clearly tls_cert_file format

thanks in advance.

-- 
Selçuk YAZAR