On 03/12/2021 21:36, Evgeniy Berdnikov via Exim-users wrote:
Hello.
On Fri, Dec 03, 2021 at 12:00:33PM +0000, Andrew C Aitchison via
Exim-users wrote:
We currently have about 1,000 emails queued for google accounts
and these are failing with a timeout, eg from exim:
2021-12-02 11:48:05 1msize-005feQ-DI
H=gmail-smtp-in.l.google.com [142.250.13.26] TLS error on
connection (send): The TLS connection was non-properly
terminated.
2021-12-02 11:48:05 1msize-005feQ-DI
H=gmail-smtp-in.l.google.com [142.250.13.26]: SMTP timeout after
sending data block (525043 bytes written): Connection timed out
Looks as broken Path MTU Discovery.
Based on past experience, eventually, the emails may will
through. This is also intermittent, as sending to my own gmail
address tends to get through eventually, though they do still
time out once or twice before eventually getting through.
If some mails could be passed, this is a hint that path is changing
over time, so effect is not stable. With MTR utility path can be
monitored in real time (like traceroute, but much better).
This report is rather vague, but we do have further examples,
logs, pcaps.
Use pcaps, study the moments when connections are timed out.
After trying a few things, disabling TCP Fast Open seems to have
worked...
TFO affects connection open stage, but this problem is definitely due
to timeout in the middle of data transfer.
Many internet sites (may be millions) communicate with Google, so
this problem is very likely local for one site and its internet
connection.
Thanks for the reply...
We have a direct interconnect to Google, so there is only a fibre
between our router and Google's router, that part of path is stable and
identical - though there are a number of hops withing Google's AS that
could be changing.
ping/mtr - no packet loss
Tried reducing MTU (can't pmtu discover to gmail as they block too much
ICMP) - no difference
Checked IPv4 vs IPv6 - no difference
Andrew.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
