postfix-3.4-20180619 updated connection reuse
postfix-3.4-20180619 fixes today's segfault in the connection reuse logic. I have been unable to reproduce the problem on my own systems so I'll depend on other people for confirmation. Wietse
Re: PATCH: multiple deliveries per TLS-encrypted connection
Viktor Dukhovni: > > > > On Jun 19, 2018, at 3:07 PM, Wietse Venema wrote: > > > > Viktor Dukhovni: > >> On Jun 19, 2018, at 2:38 PM, Wietse Venema wrote: > >>> Or alternatively, > >>> we should save the original "DANE candidate" level for recording > >>> in the session cache for nexthop entries, and then use the actual > >>> level and TLS properties for the per-address entries. > > > > That assumes that the TLS policy for a given (delivery agent, nexthop, ...) > > is stable, which is a reasonable assumption. > > That's my take also, on the timescale of the connection reuse > maximum TTL. > > >> In which case when looking for some connection to the nexthop, > >> I think it could be simpler to assume that if the connection > >> match the policy when it was recently created (we have a > >> connection re-use TTL) then it should still be good enough > >> without salting in the TLS details. > > > > Which is the same thing as fixing it to "may" as in your > > previous patch. > > Yes, but perhaps better to just use the pre-MX policy > without modification. Thinking ahead to MTA-STS, when > a first message triggers background MTA-STS policy > discovery, we might want messages that arrive after > that's available to immediately upgrade to STS. smtp_reuse_session() is called if SMTP_MISC_FLAG_CONN_LOAD is set, whereas smtp_save_session() is called if SMTP_MISC_FLAG_CONN_STORE is set. These guards are different for a reason. Therefore we cannot assume that smtp_reuse_session() will be able to determine the pre-MX policy to be used by for smtp_save_session(), unless I mis-understand what you mean with pre-MX policy (I assume the TLS policy based on the nexhop info only). I'll keep it simple for now. and use an manifest constant for nexthop-level connection cache lookups and stores. We can refine the logic later. Wietse > This suggests that the base (nexthop-based) level should > be used and checked. If that changes, we make a new > connection. > > [ In actuality we'll need some new way to enable either or > both of MTA-STS and DANE, and when both are enabled prefer > DANE for those MX hosts that have TLSA records, but otherwise > do MTA-STS if that's provisioned at the remote domain. But > that's for 2019. ] > > > >> So I don't think the below is correct. One might > >> argue that my patch should simply leave the TLS > >> level alone (rather than set it to "may"), and > >> that the cache entry for the nexthop should reflect > >> that preliminary nexthop security level, before it > >> is potentially modified for MX hosts that lack TLSA > >> records... > > > > But then we'd have to keep that preliminary nexthop security level > > around until scache_save_mumble are called. It may be clearer to > > use a dummy level to indicate that this is not really used as a TLS > > level. > > Instead of a dummy, I think for now just keep the > level that comes out of TLS policy lookup, before > DANE messes with it on a per-MX basis. > > We could later consider refactoring, so that the DANE > level always remains unchanged, but is treated as "may" > dynamically, when no TLSA data accompanies the client > policy. > > -- > Viktor. > >
Re: PATCH: multiple deliveries per TLS-encrypted connection
Wietse Venema: > Viktor Dukhovni: > > On Jun 19, 2018, at 2:38 PM, Wietse Venema wrote: > > > > > > It would not crash, but I don't think it would help. > > > > > > First, the scache is indexed with keys that include the TLS security > > > level for a connection, so that we will never reuse a low-security > > > connection to deliver mail for a high-security destination. > > > > The problem is that with DANE we don't know the security level > > until we're working with a particular MX host. Or alternatively, > > we should save the original "DANE candidate" level for recording > > in the session cache for nexthop entries, and then use the actual > > level and TLS properties for the per-address entries. > > That assumes that the TLS policy for a given (delivery agent, nexthop, ...) > is stable, which is a reasonable assumption. > > > IIRC the cache key also includes the delivery agent name, > > so we never mix delivery agents with different CA stores, > > TLS policy tables, ... > > Right. We trust in belts, suspenders, and paranoia. > > > In which case when looking for some connection to the nexthop, > > I think it could be simpler to assume that if the connection > > match the policy when it was recently created (we have a > > connection re-use TTL) then it should still be good enough > > without salting in the TLS details. > > Which is the same thing as fixing it to "may" as in your > previous patch. > > > Salting in TLS details is of course needed for the per-peer > > address cache lookups, where the same host might serve > > multiple domains. > > > [...] One might > > argue that my patch should simply leave the TLS > > level alone (rather than set it to "may"), and > > that the cache entry for the nexthop should reflect > > that preliminary nexthop security level, before it > > is potentially modified for MX hosts that lack TLSA > > records... > > But then we'd have to keep that preliminary nexthop security level > around until scache_save_mumble are called. It may be clearer to > use a dummy level to indicate that this is not really used as a TLS > level. And that is what I will try to implement: a dummy security level for nexthop-level connection cache lookup keys. Wietse
Re: PATCH: multiple deliveries per TLS-encrypted connection
Viktor Dukhovni: > On Jun 19, 2018, at 2:38 PM, Wietse Venema wrote: > > > > It would not crash, but I don't think it would help. > > > > First, the scache is indexed with keys that include the TLS security > > level for a connection, so that we will never reuse a low-security > > connection to deliver mail for a high-security destination. > > The problem is that with DANE we don't know the security level > until we're working with a particular MX host. Or alternatively, > we should save the original "DANE candidate" level for recording > in the session cache for nexthop entries, and then use the actual > level and TLS properties for the per-address entries. That assumes that the TLS policy for a given (delivery agent, nexthop, ...) is stable, which is a reasonable assumption. > IIRC the cache key also includes the delivery agent name, > so we never mix delivery agents with different CA stores, > TLS policy tables, ... Right. We trust in belts, suspenders, and paranoia. > In which case when looking for some connection to the nexthop, > I think it could be simpler to assume that if the connection > match the policy when it was recently created (we have a > connection re-use TTL) then it should still be good enough > without salting in the TLS details. Which is the same thing as fixing it to "may" as in your previous patch. > Salting in TLS details is of course needed for the per-peer > address cache lookups, where the same host might serve > multiple domains. > So I don't think the below is correct. One might > argue that my patch should simply leave the TLS > level alone (rather than set it to "may"), and > that the cache entry for the nexthop should reflect > that preliminary nexthop security level, before it > is potentially modified for MX hosts that lack TLSA > records... But then we'd have to keep that preliminary nexthop security level around until scache_save_mumble are called. It may be clearer to use a dummy level to indicate that this is not really used as a TLS level. Wietse
Re: PATCH: multiple deliveries per TLS-encrypted connection
On Jun 19, 2018, at 2:38 PM, Wietse Venema wrote:
>
> It would not crash, but I don't think it would help.
>
> First, the scache is indexed with keys that include the TLS security
> level for a connection, so that we will never reuse a low-security
> connection to deliver mail for a high-security destination.
The problem is that with DANE we don't know the security level
until we're working with a particular MX host. Or alternatively,
we should save the original "DANE candidate" level for recording
in the session cache for nexthop entries, and then use the actual
level and TLS properties for the per-address entries.
IIRC the cache key also includes the delivery agent name,
so we never mix delivery agents with different CA stores,
TLS policy tables, ...
In which case when looking for some connection to the nexthop,
I think it could be simpler to assume that if the connection
match the policy when it was recently created (we have a
connection re-use TTL) then it should still be good enough
without salting in the TLS details.
Salting in TLS details is of course needed for the per-peer
address cache lookups, where the same host might serve
multiple domains.
> Thus
> the lookups need to specify the security level that was in effect
> when a connection was stored in the cache.
This is not possible with DANE without some further changes,
but I am not convinced it is necessary.
> Second, it wants to look up the scache for the nexthop and primary
> MXes to avoid contacting hosts that we have no cached connection
> for. I think that these are the lookups that are failing because
> state->iter->rr is not set. Would the patch below help?
It would not always result in correct behaviour, the cached
nexthop connection might not be associated with the first
MX host. So this too would not crash, but may not produce
correct results. If, for example, the first MX host has
DANE TLSA records that don't match, we presumably (really
must) close the connection without caching it, and may
make a connection to a backup MX without DANE TLSA RRs
(insecure, but that's what we have). But then the
nexthop connection's security level does not match
the security level of the initial MX.
Note also that DANE policy lookup may fail for the
initial MX (bad DNSSEC...), but we should still
be able to re-use cached connections via the second
MX, ...
So I don't think the below is correct. One might
argue that my patch should simply leave the TLS
level alone (rather than set it to "may"), and
that the cache entry for the nexthop should reflect
that preliminary nexthop security level, before it
is potentially modified for MX hosts that lack TLSA
records...
> *** ./smtp_connect.c- 2018-06-04 19:21:21.0 -0400
> --- ./smtp_connect.c 2018-06-19 14:36:32.0 -0400
> ***
> *** 672,677
> --- 672,678
> * for connection-cache lookup by request nexthop only.
> */
> #ifdef USE_TLS
> + iter->rr = *addr_list;
> if (!smtp_tls_policy_cache_query(why, state->tls, iter)) {
> msg_warn("TLS policy lookup error for %s/%s: %s",
>STR(iter->dest), STR(iter->host), STR(why->reason));
--
Viktor.
Re: PATCH: multiple deliveries per TLS-encrypted connection
Viktor Dukhovni:
>
>
> > On Jun 19, 2018, at 1:29 PM, Viktor Dukhovni
> > wrote:
> >
> > In that case, perhaps the below will work?
> >
> > diff --git a/src/smtp/smtp_tls_policy.c b/src/smtp/smtp_tls_policy.c
> > index 13735b21..b5f72376 100644
> > --- a/src/smtp/smtp_tls_policy.c
> > +++ b/src/smtp/smtp_tls_policy.c
> > @@ -824,6 +824,20 @@ static void dane_init(SMTP_TLS_POLICY *tls,
> > SMTP_ITERATOR *iter)
> > dane_incompat(tls, iter, NONDANE_DEST, "non DNSSEC destination");
> > return;
> > }
> > +
> > +/*
> > + * Without the context of a particular MX host, the nexthop is merely a
> > + * opportunistic "candidate" for DANE policy. Just return "may" for
> > now.
> > + *
> > + * XXX: This state should only be reached from smtp_reuse_session(),
> > + * perhaps add a safety flag in iter that is co-requisite with a NULL
> > + * iter->rr, or else panic?
> > + */
> > +if (!iter->rr) {
> > + tls->level = TLS_LEV_MAY;
> > + return;
> > +}
> > +
> > /* When TLSA lookups fail, we defer the message */
> > if ((dane = tls_dane_resolve(iter->port, "tcp", iter->rr,
> > var_smtp_tls_force_tlsa)) == 0) {
>
> Ralf, please try just this patch against the stock 20180618 snapshot,
> and check as many of the below as you can:
This will stop crashes but it will also not find any connections
that are cached under the nexthop name, and that were stored with
a different TLS security level.
The connection cache storage key contains the TLS security level
that is in effect at the time that the connection is stored, so
that we will never deliver mail for a high-security destination
over a reused low-security connection. BTW it currently not reuse
connections when the security level requires certificate verification,
because the certificate information is currently not stored in the
connection cache.
Not finding connections cached under nexthop, when primary MXes are
unavailable, means Postfix will waste time commecting to unavailable
MX hosts before it finds something that works.
Finally, can anyone tell me how to reproduce these crashes? This
is extremely frustrating. I can do DNSSSEC lookups on freeBSD and
LINUX.
Wietse
Re: PATCH: multiple deliveries per TLS-encrypted connection
Viktor Dukhovni:
> On Tue, Jun 19, 2018 at 01:22:53PM -0400, Wietse Venema wrote:
>
> > Unfortunately, this would be suboptimal when a site has muliple MX hosts
> > (It may end up making connections to each of them).
> >
> > Viktor's suggestion to skip the dane cache makes more sense.
> >
> > Viktor, cache wshould terminate after "postfix reload".
>
> In that case, perhaps the below will work?
It would not crash, but I don't think it would help.
First, the scache is indexed with keys that include the TLS security
level for a connection, so that we will never reuse a low-security
connection to deliver mail for a high-security destination. Thus
the lookups need to specify the security level that was in effect
when a connection was stored in the cache.
Second, it wants to look up the scache for the nexthop and primary
MXes to avoid contacting hosts that we have no cached connection
for. I think that these are the lookups that are failing because
state->iter->rr is not set. Would the patch below help?
Wietse
*** ./smtp_connect.c- 2018-06-04 19:21:21.0 -0400
--- ./smtp_connect.c2018-06-19 14:36:32.0 -0400
***
*** 672,677
--- 672,678
* for connection-cache lookup by request nexthop only.
*/
#ifdef USE_TLS
+ iter->rr = *addr_list;
if (!smtp_tls_policy_cache_query(why, state->tls, iter)) {
msg_warn("TLS policy lookup error for %s/%s: %s",
STR(iter->dest), STR(iter->host), STR(why->reason));
Re: PATCH: multiple deliveries per TLS-encrypted connection
> On Jun 19, 2018, at 1:29 PM, Viktor Dukhovni
> wrote:
>
> In that case, perhaps the below will work?
>
> diff --git a/src/smtp/smtp_tls_policy.c b/src/smtp/smtp_tls_policy.c
> index 13735b21..b5f72376 100644
> --- a/src/smtp/smtp_tls_policy.c
> +++ b/src/smtp/smtp_tls_policy.c
> @@ -824,6 +824,20 @@ static void dane_init(SMTP_TLS_POLICY *tls,
> SMTP_ITERATOR *iter)
> dane_incompat(tls, iter, NONDANE_DEST, "non DNSSEC destination");
> return;
> }
> +
> +/*
> + * Without the context of a particular MX host, the nexthop is merely a
> + * opportunistic "candidate" for DANE policy. Just return "may" for now.
> + *
> + * XXX: This state should only be reached from smtp_reuse_session(),
> + * perhaps add a safety flag in iter that is co-requisite with a NULL
> + * iter->rr, or else panic?
> + */
> +if (!iter->rr) {
> + tls->level = TLS_LEV_MAY;
> + return;
> +}
> +
> /* When TLSA lookups fail, we defer the message */
> if ((dane = tls_dane_resolve(iter->port, "tcp", iter->rr,
>var_smtp_tls_force_tlsa)) == 0) {
Ralf, please try just this patch against the stock 20180618 snapshot,
and check as many of the below as you can:
* The crashes are gone
* DANE is still used when expected
* TLS connection re-use happens under sustained load
We might want to log some sort of connection identifier
with re-used connections, that would make it possible
reliably find the original session to check that it
was authenticated as expected. This is only useful
with TLS, so we could perhaps ask the TLS library for
"channel binding" at session creation, and log it
(on both ends) with the client logging it again on
*connection* re-use. (TLS session resumption is
a separate matter and would produce a separate
channel fingerprint on each connection).
--
Viktor.
Re: PATCH: multiple deliveries per TLS-encrypted connection
Wietse Venema:
> Ralf, does this helpl?
Unfortunately, this would be suboptimal when a site has muliple MX hosts
(It may end up making connections to each of them).
Viktor's suggestion to skip the dane cache makes more sense.
Viktor, cache wshould terminate after "postfix reload".
> Wietse
>
> *** ./smtp_connect.c- 2018-06-04 19:21:21.0 -0400
> --- ./smtp_connect.c 2018-06-19 13:11:30.0 -0400
> ***
> *** 671,676
> --- 671,677
>* matching sessions. Otherwise, request a dummy "TLS disabled" policy
>* for connection-cache lookup by request nexthop only.
>*/
> + return (0);
> #ifdef USE_TLS
> if (!smtp_tls_policy_cache_query(why, state->tls, iter)) {
> msg_warn("TLS policy lookup error for %s/%s: %s",
>
Re: PATCH: multiple deliveries per TLS-encrypted connection
Ralf, does this helpl?
Wietse
*** ./smtp_connect.c- 2018-06-04 19:21:21.0 -0400
--- ./smtp_connect.c2018-06-19 13:11:30.0 -0400
***
*** 671,676
--- 671,677
* matching sessions. Otherwise, request a dummy "TLS disabled" policy
* for connection-cache lookup by request nexthop only.
*/
+ return (0);
#ifdef USE_TLS
if (!smtp_tls_policy_cache_query(why, state->tls, iter)) {
msg_warn("TLS policy lookup error for %s/%s: %s",
Re: PATCH: multiple deliveries per TLS-encrypted connection
> On Jun 19, 2018, at 12:32 PM, Viktor Dukhovni > wrote: > > DANE context initialization needs to know whether the MX hostname > is an alias, and was previously only done per-MX. Now there's > a new call with "iter->rr" still NULL. The code in dane_init() > is not prepared for that. Without the MX hostname and its DNS_RRs > it can't decide whether the security level is DANE (MX host has > TLSA records, ...) or opportunistic TLS. Connection re-use is primarily per-destination, and only secondarily per-address, while DANE policy is per MX host, based on presence or absence of TLSA records for *that* MX host. At the global level, all we know is whether the nexthop domain is a *candidate* for DANE, but not whether DANE will or won't happen. If we have a cached connection for the same nexthop domain (and port), presumably it has a recent policy for one of the MX hosts, and we can use that. And only do fine-graing policy matching when doing per-host resumption. Which means that dane_init() could be optimized out when (iter->rr == NULL)? That would however imply that a change in policy might not kick in quite as fast as one might like. A reload might be needed to make sure that cached connections for a nexthop with a prior policy are not re-used (will a reload cause scache to restart?). -- Viktor.
Re: PATCH: multiple deliveries per TLS-encrypted connection
> On Jun 19, 2018, at 12:21 PM, Wietse Venema wrote:
>
> Argh, the trace ends in the smtp_tls_policy_cache_query which is called from
> more than one place. Investigating...
DANE context initialization needs to know whether the MX hostname
is an alias, and was previously only done per-MX. Now there's
a new call with "iter->rr" still NULL. The code in dane_init()
is not prepared for that. Without the MX hostname and its DNS_RRs
it can't decide whether the security level is DANE (MX host has
TLSA records, ...) or opportunistic TLS.
diff --git a/postfix/src/smtp/smtp_connect.c b/postfix/src/smtp/smtp_connect.c
index 2bf209d9..abccb57c 100644
--- a/postfix/src/smtp/smtp_connect.c
+++ b/postfix/src/smtp/smtp_connect.c
@@ -669,9 +669,12 @@ static int smtp_reuse_session(SMTP_STATE *state, DNS_RR
**addr_list,
*
- * We request a dummy "TLS disabled" policy for connection-cache lookup by
- * request nexthop only. If we find a saved connection, then we know that
- * plaintext was permitted, because we never save a connection after
- * turning on TLS.
+ * If TLS is proxied, lookup the TLS policy now so that we reuse only
+ * matching sessions. Otherwise, request a dummy "TLS disabled" policy
+ * for connection-cache lookup by request nexthop only.
*/
#ifdef USE_TLS
-smtp_tls_policy_dummy(state->tls);
+if (!smtp_tls_policy_cache_query(why, state->tls, iter)) {
+ msg_warn("TLS policy lookup error for %s/%s: %s",
+STR(iter->dest), STR(iter->host), STR(why->reason));
+ return (0); /* XXX */
+}
#endif
--
Viktor.
Re: PATCH: multiple deliveries per TLS-encrypted connection
Wietse Venema: > Viktor Dukhovni: > > > > > > > On Jun 19, 2018, at 11:58 AM, Wietse Venema wrote: > > > > > > No error (btw, posttls-finger -X will chdir() to the queue directory, > > > it just needs root privs). > > > > > > So what was the domain that was failing with the Postfix SMTP client? > > > > The crash (from Ralf's stack trace) was in a code path that is not > > exercised by posttls-finger, it is in the TLS policy setup (for gmail.com): > > > > Program received signal SIGSEGV, Segmentation fault. > > tls_dane_resolve (port=6400, proto=proto@entry=0x558114406c72 > > "tcp", > >hostrr=0x0, forcetlsa=0) at tls_dane.c:1146 > >1146int iscname = strcasecmp(hostrr->rname, > > hostrr->qname); > > (gdb) #0 tls_dane_resolve (port=6400, proto=proto@entry=0x558114406c72 > > "tcp", > >hostrr=0x0, forcetlsa=0) at tls_dane.c:1146 > >#1 0x558114402add in dane_init (iter=0x558114882378, > > tls=0x55811487e9a0) > >at smtp_tls_policy.c:828 > >#2 policy_create (unused_key=,context=0x558114882378) at > > smtp_tls_policy.c:558 > >#3 0x7f0ee95307ec in ctable_locate (cache=0x55811487ea80, > >key=0x55811487f900 "gmail.com::25:") at ctable.c:163 > >#4 0x55811440318a in smtp_tls_policy_cache_query ( > > why=why@entry=0x558114899bd0, tls=tls@entry=0x5581148823c0, > > iter=iter@entry=0x558114882378) at smtp_tls_policy.c:675 > > How would I reproduce this? I can send mail to gmail.com just fine, > with postfix-3.4-20180618 and with "smtp_tls_connection_reuse = yes", > Linux and FreeBSD. > > Or can you provide a stack trace? Argh, the trace ends in the smtp_tls_policy_cache_query which is called from more than one place. Investigating... Wietse
Re: PATCH: multiple deliveries per TLS-encrypted connection
Viktor Dukhovni: > > > > On Jun 19, 2018, at 11:58 AM, Wietse Venema wrote: > > > > No error (btw, posttls-finger -X will chdir() to the queue directory, > > it just needs root privs). > > > > So what was the domain that was failing with the Postfix SMTP client? > > The crash (from Ralf's stack trace) was in a code path that is not > exercised by posttls-finger, it is in the TLS policy setup (for gmail.com): > > Program received signal SIGSEGV, Segmentation fault. > tls_dane_resolve (port=6400, proto=proto@entry=0x558114406c72 > "tcp", >hostrr=0x0, forcetlsa=0) at tls_dane.c:1146 >1146int iscname = strcasecmp(hostrr->rname, > hostrr->qname); > (gdb) #0 tls_dane_resolve (port=6400, proto=proto@entry=0x558114406c72 > "tcp", >hostrr=0x0, forcetlsa=0) at tls_dane.c:1146 >#1 0x558114402add in dane_init (iter=0x558114882378, > tls=0x55811487e9a0) >at smtp_tls_policy.c:828 >#2 policy_create (unused_key=,context=0x558114882378) at > smtp_tls_policy.c:558 >#3 0x7f0ee95307ec in ctable_locate (cache=0x55811487ea80, >key=0x55811487f900 "gmail.com::25:") at ctable.c:163 >#4 0x55811440318a in smtp_tls_policy_cache_query ( > why=why@entry=0x558114899bd0, tls=tls@entry=0x5581148823c0, > iter=iter@entry=0x558114882378) at smtp_tls_policy.c:675 How would I reproduce this? I can send mail to gmail.com just fine, with postfix-3.4-20180618 and with "smtp_tls_connection_reuse = yes", Linux and FreeBSD. Or can you provide a stack trace? Wietse
Re: PATCH: multiple deliveries per TLS-encrypted connection
> On Jun 19, 2018, at 11:58 AM, Wietse Venema wrote: > > No error (btw, posttls-finger -X will chdir() to the queue directory, > it just needs root privs). > > So what was the domain that was failing with the Postfix SMTP client? The crash (from Ralf's stack trace) was in a code path that is not exercised by posttls-finger, it is in the TLS policy setup (for gmail.com): Program received signal SIGSEGV, Segmentation fault. tls_dane_resolve (port=6400, proto=proto@entry=0x558114406c72 "tcp", hostrr=0x0, forcetlsa=0) at tls_dane.c:1146 1146int iscname = strcasecmp(hostrr->rname, hostrr->qname); (gdb) #0 tls_dane_resolve (port=6400, proto=proto@entry=0x558114406c72 "tcp", hostrr=0x0, forcetlsa=0) at tls_dane.c:1146 #1 0x558114402add in dane_init (iter=0x558114882378, tls=0x55811487e9a0) at smtp_tls_policy.c:828 #2 policy_create (unused_key=,context=0x558114882378) at smtp_tls_policy.c:558 #3 0x7f0ee95307ec in ctable_locate (cache=0x55811487ea80, key=0x55811487f900 "gmail.com::25:") at ctable.c:163 #4 0x55811440318a in smtp_tls_policy_cache_query ( why=why@entry=0x558114899bd0, tls=tls@entry=0x5581148823c0, iter=iter@entry=0x558114882378) at smtp_tls_policy.c:675 -- Viktor.
Re: PATCH: multiple deliveries per TLS-encrypted connection
Ralf Hildebrandt: > * Wietse Venema : > > Ralf Hildebrandt: > > > * Ralf Hildebrandt : > > > > > > > Error inducing change was introduced between postfix-3.4-20180603 and > > > > postfix-3.4-20180605-nonprod > > > > > > I also tried postfix-3.4-20180603-nonprod which seems to be working > > > ok! So I guess it must have been between postfix-3.4-20180603-nonprod > > > and postfix-3.4-20180605-nonprod > > > > Does this reproduce with "posttls-finger -X "? > > posttls-finger from the source directory run from /var/spool/postfix gives me: No error (btw, posttls-finger -X will chdir() to the queue directory, it just needs root privs). So what was the domain that was failing with the Postfix SMTP client? Wietse
Re: PATCH: multiple deliveries per TLS-encrypted connection
* Wietse Venema : > Ralf Hildebrandt: > > * Ralf Hildebrandt : > > > > > Error inducing change was introduced between postfix-3.4-20180603 and > > > postfix-3.4-20180605-nonprod > > > > I also tried postfix-3.4-20180603-nonprod which seems to be working > > ok! So I guess it must have been between postfix-3.4-20180603-nonprod > > and postfix-3.4-20180605-nonprod > > Does this reproduce with "posttls-finger -X "? posttls-finger from the source directory run from /var/spool/postfix gives me: root@mail:/var/spool/postfix# /usr/src/postfix-3.4-20180605-nonprod/bin/posttls-finger -X gmail.com posttls-finger: Connected to gmail-smtp-in.l.google.com[2a00:1450:4013:c03::1a]:25 posttls-finger: < 220 mx.google.com ESMTP j7-v6si170503eda.357 - gsmtp posttls-finger: > EHLO mail.python.org posttls-finger: < 250-mx.google.com at your service, [2a03:b0c0:2:d0::71:1] posttls-finger: < 250-SIZE 157286400 posttls-finger: < 250-8BITMIME posttls-finger: < 250-STARTTLS posttls-finger: < 250-ENHANCEDSTATUSCODES posttls-finger: < 250-PIPELINING posttls-finger: < 250-CHUNKING posttls-finger: < 250 SMTPUTF8 posttls-finger: > STARTTLS posttls-finger: < 220 2.0.0 Ready to start TLS posttls-finger: gmail-smtp-in.l.google.com[2a00:1450:4013:c03::1a]:25: subject_CN=gmail-smtp-in.l.google.com, issuer_CN=Google Internet Authority G3,fingerprint=0C:EE:E6:2E:90:68:56:5A:88:BF:C7:81:B7:D1:A1:3C:EB:2C:B0:30,pkey_fingerprint=64:95:0D:F5:04:A2:FF:66:00:4E:06:AE:F6:37:B5:CF:7F:AE:20:99 posttls-finger: Untrusted TLS connection established to gmail-smtp-in.l.google.com[2a00:1450:4013:c03::1a]:25: TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits) posttls-finger: > EHLO mail.python.org posttls-finger: < 250-mx.google.com at your service, [2a03:b0c0:2:d0::71:1] posttls-finger: < 250-SIZE 157286400 posttls-finger: < 250-8BITMIME posttls-finger: < 250-ENHANCEDSTATUSCODES posttls-finger: < 250-PIPELINING posttls-finger: < 250-CHUNKING posttls-finger: < 250 SMTPUTF8 posttls-finger: > QUIT posttls-finger: warning: lost connection while sending QUIT command -- [*] sys4 AG https://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Re: Redirect all email to an external address
Martin LEUSCH: > Hi, > > I'm trying to create postfix master process listening on port 2525 and > redirect all email send trough this port to an external address. > > I create regexp table "/etc/postfix/canonical-redirect-test" to map any > address to the external address I want, test query with postmap return > the expected address. > > /^.+@.+$/ t...@example.com > > I add new process in /etc/postfix/master.cf > > :2525 inet n - n - - smtpd > -o canonical_maps=regexp:/etc/postfix/canonical-redirect-test > -o virtual_alias_maps=regexp:/etc/postfix/canonical-redirect-test The canonical and virtual_alias mappings are implemented in the cleanup daemon, not smtpd. /etc/postfix/master.cf: :2525 inet n - n - - smtpd -o cleanup_service=cleanup_2525 cleanup_2525 unix n - n - 0 cleanup -o canonical_maps=regexp:/etc/postfix/canonical-redirect-test -o virtual_alias_maps=regexp:/etc/postfix/canonical-redirect-test Wietse
Re: PATCH: multiple deliveries per TLS-encrypted connection
Ralf Hildebrandt: > * Ralf Hildebrandt : > > > Error inducing change was introduced between postfix-3.4-20180603 and > > postfix-3.4-20180605-nonprod > > I also tried postfix-3.4-20180603-nonprod which seems to be working > ok! So I guess it must have been between postfix-3.4-20180603-nonprod > and postfix-3.4-20180605-nonprod Does this reproduce with "posttls-finger -X "? Wietse
Redirect all email to an external address
Hi, I'm trying to create postfix master process listening on port 2525 and redirect all email send trough this port to an external address. I create regexp table "/etc/postfix/canonical-redirect-test" to map any address to the external address I want, test query with postmap return the expected address. /^.+@.+$/ t...@example.com I add new process in /etc/postfix/master.cf :2525 inet n - n - - smtpd -o canonical_maps=regexp:/etc/postfix/canonical-redirect-test -o virtual_alias_maps=regexp:/etc/postfix/canonical-redirect-test I also try other settings with "recipient_canonical_maps" or "virtual_maps" but email are always send to the original recipient address and not to t...@example.com. Thanks,
RE: Feedback on Tutorial
Hi, I took a quick look, I did not see which version of Postfix this was based on, you may want to mention it as some settings require a minimum Postfix version to be running. As well as some command examples you gave have minimum versions needed to work. I am no expert but I get lots of legit email from servers with no RDNS so this may be too strong a setting ?? smtpd_client_restrictions = permit_mynetworks permit_sasl_authenticated reject_unknown_reverse_client_hostname The purpose of this section seems to be a bad idea ? Removing Telltale Headers How do you troubleshoot delivery or other problems without logs to show what postfix decided to do about an email ? Finally you mention "Nextcloud" I did not google it so I do not know what it is, if you are talking about Postfix you should stick to generic topics without specifying specific brands of products so a larger audience knows what you are talking about. IMHO. For example your user repository could be a Directory, database, or files. It may not help to mention specific products that implement those repositories ? Thank you. -ANGELO FAZZINA ITS Service Manager: Spam and Virus Prevention Mass Mailing G Suite/Gmail ang...@uconn.edu University of Connecticut, ITS, SSG, Server Systems 860-486-9075 -Original Message- From: owner-postfix-us...@postfix.org On Behalf Of da-postfixusers...@abelonline.de Sent: Friday, June 15, 2018 7:52 PM To: Postfix users Subject: Feedback on Tutorial Hello Postfix users, I made a relatively comprehensive tutorial[1] on how to set up a mail server (Postfix, Dovecot, Rspamd,..) and integrate it with Nextcloud. My goal was to create a all-in-one, step-by-step tutorial from beginning to end. I partly used other tutorials as a basis, but also did a lot of research and e.g. used much stricter smtpd_*_restrictions than I have seen anywhere else. It's a hobby project, I am not a full-time mail admin, so probably not useful for large companies. I would greatly appreciate your feedback and hints on possible errors or oversights. A direct link to the Postfix section: https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2F123qwe.com%2Ftutorial%2F%23postfix=02%7C01%7Cangelo.fazzina%40uconn.edu%7C213ae3bd20fb4b79bb4c08d5d31b3189%7C17f1a87e2a254eaab9df9d439034b080%7C0%7C1%7C636647036128899329=PfVxAft0NrOjsxCaTUhqmw3Y9cB3Nm2GXCmN%2BE0Hv4s%3D=0 Thank You Alexey [1] https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2F123qwe.com=02%7C01%7Cangelo.fazzina%40uconn.edu%7C213ae3bd20fb4b79bb4c08d5d31b3189%7C17f1a87e2a254eaab9df9d439034b080%7C0%7C1%7C636647036128899329=XLi2ndZ3kDr9qP9uwDmyixzKmFdMuEK7%2F5RvUrAzb1E%3D=0
Re: PATCH: multiple deliveries per TLS-encrypted connection
* Ralf Hildebrandt : > Error inducing change was introduced between postfix-3.4-20180603 and > postfix-3.4-20180605-nonprod I also tried postfix-3.4-20180603-nonprod which seems to be working ok! So I guess it must have been between postfix-3.4-20180603-nonprod and postfix-3.4-20180605-nonprod -- [*] sys4 AG https://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Re: PATCH: multiple deliveries per TLS-encrypted connection
* Ralf Hildebrandt : > > Also released as postfix-3.4-20180618. > > postfix-3.4-20180618. Is crashing for me: > > Jun 19 09:39:10 mail postfix/qmgr[12033]: warning: private/smtp socket: > malformed response > Jun 19 09:39:10 mail postfix/qmgr[12033]: warning: transport smtp failure -- > see a previous warning/fatal/panic logfile record for the problem description > Jun 19 09:39:11 mail postfix/master[6142]: warning: process > /usr/lib/postfix/smtp pid 12341 killed by signal 11 > > No matter what the setting for smtp_tls_connection_reuse is. > > smtpd -D yields: > > Loaded symbols for /lib/x86_64-linux-gnu/libnss_files.so.2 > 0x7f0ee8b8995c in __libc_waitpid (pid=25675, > stat_loc=stat_loc@entry=0x7ffd22ba62a0, options=options@entry=0) > at ../sysdeps/unix/sysv/linux/waitpid.c:31 > (gdb) Continuing. > > Program received signal SIGSEGV, Segmentation fault. > tls_dane_resolve (port=6400, proto=proto@entry=0x558114406c72 > "tcp", > hostrr=0x0, forcetlsa=0) at tls_dane.c:1146 > 1146int iscname = strcasecmp(hostrr->rname, > hostrr->qname); Error inducing change was introduced between postfix-3.4-20180603 and postfix-3.4-20180605-nonprod
Re: PATCH: multiple deliveries per TLS-encrypted connection
* Ralf Hildebrandt : > > Also released as postfix-3.4-20180618. > > postfix-3.4-20180618. Is crashing for me: > > Jun 19 09:39:10 mail postfix/qmgr[12033]: warning: private/smtp socket: > malformed response > Jun 19 09:39:10 mail postfix/qmgr[12033]: warning: transport smtp failure -- > see a previous warning/fatal/panic logfile record for the problem description > Jun 19 09:39:11 mail postfix/master[6142]: warning: process > /usr/lib/postfix/smtp pid 12341 killed by signal 11 > > No matter what the setting for smtp_tls_connection_reuse is. > > smtpd -D yields: Typo, of course I debugged smtp, not smtpd!!! -- [*] sys4 AG https://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Re: PATCH: multiple deliveries per TLS-encrypted connection
> Also released as postfix-3.4-20180618. postfix-3.4-20180618. Is crashing for me: Jun 19 09:39:10 mail postfix/qmgr[12033]: warning: private/smtp socket: malformed response Jun 19 09:39:10 mail postfix/qmgr[12033]: warning: transport smtp failure -- see a previous warning/fatal/panic logfile record for the problem description Jun 19 09:39:11 mail postfix/master[6142]: warning: process /usr/lib/postfix/smtp pid 12341 killed by signal 11 No matter what the setting for smtp_tls_connection_reuse is. smtpd -D yields: Loaded symbols for /lib/x86_64-linux-gnu/libnss_files.so.2 0x7f0ee8b8995c in __libc_waitpid (pid=25675, stat_loc=stat_loc@entry=0x7ffd22ba62a0, options=options@entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:31 (gdb) Continuing. Program received signal SIGSEGV, Segmentation fault. tls_dane_resolve (port=6400, proto=proto@entry=0x558114406c72 "tcp", hostrr=0x0, forcetlsa=0) at tls_dane.c:1146 1146int iscname = strcasecmp(hostrr->rname, hostrr->qname); (gdb) #0 tls_dane_resolve (port=6400, proto=proto@entry=0x558114406c72 "tcp", hostrr=0x0, forcetlsa=0) at tls_dane.c:1146 #1 0x558114402add in dane_init (iter=0x558114882378, tls=0x55811487e9a0) at smtp_tls_policy.c:828 #2 policy_create (unused_key=,context=0x558114882378) at smtp_tls_policy.c:558 #3 0x7f0ee95307ec in ctable_locate (cache=0x55811487ea80, key=0x55811487f900 "gmail.com::25:") at ctable.c:163 #4 0x55811440318a in smtp_tls_policy_cache_query ( why=why@entry=0x558114899bd0, tls=tls@entry=0x5581148823c0, iter=iter@entry=0x558114882378) at smtp_tls_policy.c:675 #5 0x5581143fa1e2 in smtp_reuse_session (domain_best_pref=5, addr_list=0x7ffd22ba5cc0, state=0x558114882340) at smtp_connect.c:675 #6 smtp_connect_inet (state=0x558114882340, nexthop=, def_service=0x55811484ee80 "smtp") at smtp_connect.c:920 #7 0x5581143fb280 in smtp_connect (state=0x558114882340) at smtp_connect.c:1172 #8 0x5581143f8f3d in deliver_message (request=0x55811487bcd0, service=0x7ffd22ba7f8e "smtp") at smtp.c:1040 #9 smtp_service (client_stream=0x558114899890, service=0x7ffd22ba7f8e "smtp", argv=) at smtp.c:1072 #10 0x7f0ee9dc478a in single_server_wakeup (fd=fd@entry=17, attr=attr@entry=0x0) at single_server.c:304 #11 0x7f0ee9dc49cf in single_server_accept_local ( unused_event=, context=) at single_server.c:350 #12 0x7f0ee9536b01 in event_loop (delay=) at events.c:1186 #13 0x7f0ee9dc5847 in single_server_main (argc=, argv=, service=0x5581143f8e65 ) at single_server.c:817 #14 0x5581143f9857 in main (argc=5, argv=0x7ffd22ba6688) at smtp.c:1385 (gdb)
