Jeremy Harris via Exim-users wrote: > On 30/12/2020 08:04, Evgeniy Berdnikov via Exim-users wrote: > > On Wed, Dec 30, 2020 at 02:25:19PM +0700, Victor Sudakov via Exim-users > > wrote: > > > Is this ktrace informative https://termbin.com/zjsv ? > > Yes; thanks. > > > > > 8889 exim CALL socket(PF_INET,0x1<SOCK_STREAM>,IPPROTO_IP) > > 8889 exim RET socket 5 > > 8889 exim CALL setitimer(0,0x7fffffff30d0,0x7fffffff30b0) > > 8889 exim STRU itimerval { .interval = {0, 0}, .value = {5, 0} } > > 8889 exim STRU itimerval { .interval = {0, 0}, .value = {0, 0} } > > 8889 exim RET setitimer 0 > > 8889 exim CALL setsockopt(0x5,IPPROTO_TCP,TCP_FASTOPEN,0x254270,0x4) > > 8889 exim RET setsockopt 0 > > 8889 exim CALL sendto(0x5,0x2322d6,0xa,0,0x7fffffff3150,0x10) > > 8889 exim STRU struct sockaddr { AF_INET, 192.168.153.104:3310 } > > 8889 exim GIO fd 5 wrote 10 bytes > > "zINSTREAM\0" > > 8889 exim RET sendto 10/0xa > > 8889 exim CALL setitimer(0,0x7fffffff30d0,0x7fffffff30b0) > > 8889 exim STRU itimerval { .interval = {0, 0}, .value = {0, 0} } > > 8889 exim STRU itimerval { .interval = {0, 0}, .value = {4, 999970} > > } > > 8889 exim RET setitimer 0 > > 8889 exim CALL close(0xffffffff) > > 8889 exim RET close -1 errno 9 Bad file descriptor > > > > As packet is sent, it may be some problem with TCP_FASTOPEN, probably > > with its handling in hypervisor and/or external firewall. > > Kernel, I think. The packet capture showed the SYNs not carrying > any TFO request, despite that TCP_FASTOPEN setsockopt. Probably the > FreeBSD implementation has changed since I worked on the Exim > implementation, in such a way as to break the combination. > > In case it helps, my notes from then include: > > # FreeBSD: it looks like you have to compile a custom kernel, with > # 'options TCP_RFC7413' in the config. Also set > # 'net.inet.tcp.fastopen.server_enable=1' in /etc/sysctl.conf
Well, the defaults in FreeBSD 12.2 (GENERIC kernel) are: $ sysctl -a | grep fastopen net.inet.tcp.fastopen.server_enable: 0 net.inet.tcp.fastopen.psk_enable: 0 net.inet.tcp.fastopen.path_disable_time: 900 net.inet.tcp.fastopen.numpsks: 0 net.inet.tcp.fastopen.numkeys: 0 net.inet.tcp.fastopen.maxpsks: 2 net.inet.tcp.fastopen.maxkeys: 2 net.inet.tcp.fastopen.keylen: 16 net.inet.tcp.fastopen.client_enable: 1 net.inet.tcp.fastopen.ccache_buckets: 2048 net.inet.tcp.fastopen.ccache_bucket_limit: 16 net.inet.tcp.fastopen.autokey: 120 In my situation, I set net.inet.tcp.fastopen.client_enable=0 on the client (exim host) and it cured the problem. I could probably have set net.inet.tcp.fastopen.server_enable=1 on the server side (clamd host) and it would cure the problem too? > > If there's a sysctl for enabling the client side, try changing > it. If that affects this, we need to know. Please see above. > > > I could try to code up a backstop, retrying the connection without > TFO... sigh. Without some effort it wouldn't be particularly efficient > in service operation, and I call having to do that "ugly". Better > to get it working correctly, or deciding before trying that the > feature is not usable on the platform. > Slightly less ugly would be a config option "no TFO", either global > or just on the av_scanner address. Jeremy, I did not quite understand if the whole problem is a bug in FreeBSD, or a bug in Exim, or both, but if I can provide any help or additional info/testing to clear the situation once and for all, I'd be glad to. > > > Meantime, the Exim debug channels "acl" and "transport" would show > the sequence from a higher-level view. We might be able to guess > what that close(-1) was (yes, that's a bug. Not an important one). > -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
