On Mon, Aug 17, 2026 at 01:38:14PM +0000, Rajesh Mishra via Postfix-users wrote:

> We recently migrated a portion of our inbound smtp server from Amazon
> Linux 2 (Postfix 3.5.3, OpenSSL 1.0.2k) to Amazon Linux 2023 (Postfix
> 3.11.1, OpenSSL 3.5.7), same instance type, same postfix config, same
> traffic.  We saw a large, reproducible increase in smtpd concurrency
> needed to handle the same connection volume, and traced it down to the
> STARTTLS handshake itself being meaningfully slower under OpenSSL 3.x.

Well, OpenSSL 3.5 introduces support for X25519MLKEM768 key exchange,
which makes the TLS Client Hello (CH) and ServerHello (SH) messages
noticeably larger.  Just the TCP overhead could account for the observed
latency change.

Also, OpenSSL 3.x comes with TLS 1.3 support, in which not just full
handshakes, but also session resumption performs a fresh asymmetric key
exchange, "exacerbating" the impact of larger key exchange messages.
Open 1.0.2 (long ago deprecated) supported at most TLS 1.2.

> We measured a reproducible ~7-8x slower STARTTLS handshake under
> OpenSSL 3.5.7 vs OpenSSL 1.0.2k on otherwise identical hosts/config
> (numbers in the original message below), with the gap accelerating
> further under concurrent load.

Micro-benchmarks of just the TLS handshake are rarely reflective of real
SMTP workflows, but if your actual traffic is dominated by the TLS
handshake latency, you might consider changing your SMTP client to
do connection caching as well as session caching, and higher concurrency
on modern hardware is quite reasonable.  Already ~20 years ago I was
setting concurrency limits of ~1000 on high-volume public-internet MX
hosts.

> Mainly curious whether this is a known/discussed tradeoff of the 3.x
> provider architecture, or whether it's specific to our setup somehow.

I'm inclined to attribute this mostly to the micro-benchmark design.

> Sharing the numbers in case it's useful data for the list, and to ask
> whether AWS-LC support has come up before as an alternative TLS
> backend for Postfix — HAProxy already supports building against it and
> reports substantial improvements for the same class of workload (many
> short-lived connections doing one handshake each).

The AWS TLS stack is not a viable alternative to OpenSSL in Postfix, it
is missing some required features, for example DANE support.

> Environment
> -----------
> 
> - Old: Amazon Linux 2, Postfix 3.5.3, OpenSSL 1.0.2k-fips (linked as
>   libssl.so.10 / libcrypto.so.10)
> 
> - New: Amazon Linux 2023, Postfix 3.11.1, OpenSSL 3.5.7 (linked as
>   libssl.so.3 / libcrypto.so.3)
> 
> What we measured
> -----------------
> 
> Isolated the STARTTLS stage specifically (raw-socket SMTP client,
> timing each protocol stage individually — connect, EHLO, STARTTLS,
> EHLO#2, MAIL,

What is a "raw socket SMTP client"?  What library was the client
linked with?  Is the client single- or multi-threaded?

> RCPT — against real postfix+postscreen, not a synthetic TLS-only
> benchmark) to rule out confounds from milters/DNS/content-filtering:

The "postscreen" program is not representative of Postfix TLS
performance.  In typical (intended design) deployments it does
not do TLS, exposing only a cleartext 220 greeting banner and
handing off good SMTP connections to the real SMTP server.

Measurements of "postscreen" TLS handshake performance are misleading,
it is a single-threaded program not designed for high TLS loads.
The larger payload sizes in TLS 1.3 may well be the whole story when
it comes to "postscreen" if you've configured it to do "deep protocol"
tests requiring TLS support (via tlsproxy).

>   Serial, one connection at a time (n=15-30 each):
> 
>     OpenSSL 1.0.2k:  ~7ms  mean per handshake
>     OpenSSL 3.5.7:   ~53ms mean per handshake   (~7.8x)

You have not provided sufficient configuration details about
the server or client to make meaningful conclusions.

> Under concurrent load (idle test box, 5 -> 300 simultaneous
> connections):
> 
>     5 concurrent:    60ms mean
>     20 concurrent:   66ms mean
>     50 concurrent:   88ms mean
>     100 concurrent:  89ms mean
>     200 concurrent:  103ms mean
>     300 concurrent:  149ms mean, 459ms max
> 
> The growth rate itself accelerates at higher concurrency (200->300,
> only 1.5x more connections, produced a 45% jump in mean) — looks like
> real lock/resource contention in the provider-fetch path, not simple
> queueing.

Everything in Postfix is single-threaded, there is never any "lock
contension".  A few services (postscreen, tlsmgr, ...)  handle multipe
concurrent connections via an "event loop", most are one connection
per process (smtpd, cleanup, smtp, ...).

> Separately: enabling smtpd_tls_session_cache_database
> -----------------------------------------------------

That could only slow you down, the session cache database is useless
when the client does not do session caching, and if it does, you
should use session tickets instead.  Stateful server-side session
caching with "smtpd_tls_session_cache_database" is essentially
obsolete.

> Since the full-handshake-every-time issue above affects both
> generations, we're working on enabling session caching regardless of
> the AWS-LC question.

Your client should probably have a session cache, and perhaps even reuse
connections, if it is driving a high mail volume via a single relay.

> Has AWS-LC come up as a possible alternative TLS backend for Postfix,
> similar to how HAProxy supports building against it? We're aware this
> isn't a drop-in swap (different build/link target, not a config
> change, and would need real compatibility testing across the
> cipher/protocol matrix actual senders use, not just a perf benchmark)
> — mainly curious whether this has been discussed/attempted before,
> whether there's a known blocker specific to Postfix's TLS integration,
> or whether patches in this direction would be of interest.

No, AWS-LC is neither viable nor necessary.

-- 
    Viktor.  🇺🇦 Слава Україні!
_______________________________________________
Postfix-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]

Reply via email to