Re: [exim] 4.95-RC0 - SIGSEGV (maybe attempt to write to immutable memory) & other oddities
On Fri, Jul 23, 2021 at 04:35:57PM +0100, Jeremy Harris via Exim-users wrote: > The best-quality info will be a coredump. > > Arranging one is hard as Exim is setuid. I've not tried > on a BSD, but Linux requires some deliberate relaxation of security > restrictions (setuid programs are carrying sensitive info; a > dump file has that info, leaving dump files with such info > lying around is obviously a major risk...) This: sysctl kern.sugid_coredump=1 seems to be the FreeBSD sysctl for enabling this. > Compiling with debug flag (eg, for gcc / gdb, "-ggdb") > before getting the dump would be good. > Don't worry about debug versions of libraries. I used -glldb and stopped the exim binary being stripped. > Then "bt" in gdb will give us a file and line number. So now I believe I've got the 12.2 (non-jailed) machine in a position to get this (this was just me killing -11 it to test): %sudo lldb /usr/exim/bin/exim-4.95-RC0-2 --core core.exim-4.95-RC0-2.40.55758.core (lldb) target create "/usr/exim/bin/exim-4.95-RC0-2" --core "core.exim-4.95-RC0-2.40.55758.core" Core file '/var/spool/exim/core.exim-4.95-RC0-2.40.55758.core' (x86_64) was loaded. (lldb) bt * thread #1, name = 'exim-4.95-RC0-2', stop reason = signal SIGSEGV * frame #0: 0x000800a418da libc.so.7`__sys_select + 10 frame #1: 0x000800c8fcb2 libthr.so.3`___lldb_unnamed_symbol44$$libthr.so.3 + 66 frame #2: 0x0025fd77 exim-4.95-RC0-2`daemon_go at daemon.c:2443:16 [opt] frame #3: 0x00279d5e exim-4.95-RC0-2`main(argc=, cargv=) at exim.c:4947:3 [opt] frame #4: 0x00254f00 exim-4.95-RC0-2`_start(ap=, cleanup=) at crt1.c:76:7 (lldb) And I'll wait and see what happens (This machine did not SEGV like the jailed one but might get some info from it if it fails again). It's possible what I was seeing here was caused by the Jailed exim malfunctioning… > This might be something special about a FreeBSD jail. Indeed. I've asked the Jail Host, very nicely, if they'll set (temporarily) the sysctls for me there. They might say "no", but there was no harm in asking! Fingers crossed, Matthew -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] 4.95-RC0 - SIGSEGV (maybe attempt to write to immutable memory) & other oddities
On 23/07/2021 16:03, Matthew Frost via Exim-users wrote: 4192 end of ACL "acl_check_connection": ACCEPT 4192 host in pipelining_connect_advertise_hosts? yes (matched "*") 4192 LOG: MAIN PANIC 4192 SIGSEGV (maybe attempt to write to immutable memory) 96610 child 4192 ended: status=0xb 96610 signal exit, signal 11 96610 4 SMTP accept processes now running The best-quality info will be a coredump. Arranging one is hard as Exim is setuid. I've not tried on a BSD, but Linux requires some deliberate relaxation of security restrictions (setuid programs are carrying sensitive info; a dump file has that info, leaving dump files with such info lying around is obviously a major risk...) Compiling with debug flag (eg, for gcc / gdb, "-ggdb") before getting the dump would be good. Don't worry about debug versions of libraries. Then "bt" in gdb will give us a file and line number. This might be something special about a FreeBSD jail. -- Cheers, Jeremy -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
[exim] 4.95-RC0 - SIGSEGV (maybe attempt to write to immutable memory) & other oddities
Hello, odd (perhaps) one coming up… I'm still diagnosing what might be wrong here, but I wanted to bring it to your attention as it's stopped me testing the RC0 (and is hard to debug because it takes hours/days to appear). Jailed FreeBSD 12.1-RELEASE-p1 - I have no control over the host, just the jail. (IPv4 & IPv6). Swapping 4.94.2 for 4.95-RC0 seems fine, but after a number of hours in paniclog: "SIGSEGV (maybe attempt to write to immutable memory)" Brief parts of a debug run at the point of it happening: 4192 end of ACL "acl_check_connection": ACCEPT 4192 host in pipelining_connect_advertise_hosts? yes (matched "*") 4192 LOG: MAIN PANIC 4192 SIGSEGV (maybe attempt to write to immutable memory) 96610 child 4192 ended: status=0xb 96610 signal exit, signal 11 96610 4 SMTP accept processes now running 2912 end of ACL "acl_check_connection": ACCEPT 2912 host in pipelining_connect_advertise_hosts? yes (matched "*") 2912 SMTP>> 220 hub-cloud.mail.frost.net ESMTP Exim 4.95-RC0 Thu, 22 Jul 2021 06:50:31 +0100 96610 child 2912 ended: status=0xa 96610 signal exit, signal 10 96610 2 SMTP accept processes now running 2021-07-22 06:50:30 SMTP connection from [185.82.79.5] I=[178.250.76.2]:25 (TCP/IP connection count = 3) 2021-07-22 06:50:35 SIGSEGV (maybe attempt to write to immutable memory) Also other odd failures seems like connections hanging for 5 minutes: 2021-07-22 07:26:16 SMTP connection from [66.220.155.139] I=[178.250.76.2]:25 (TCP/IP connection count = 5) 2021-07-22 07:31:16 SMTP connection from 66-220-155-139.mail-mail.facebook.com [66.220.155.139] I=[178.250.76.2]:25 lost D=4m59s "SMTP connection lost after final dot": 2021-07-22 05:24:04 SMTP connection from [66.231.95.42] I=[178.250.76.2]:25 (TCP/IP connection count = 1) 2021-07-22 05:29:06 1m6QFh-000L6m-UJ SMTP connection lost after final dot H=mta.news.marksandspencer.com [66.231.95.42] I=[178.250.76.2]:25 P=esmtps I also saw (the first time this happened) hundreds of: "50 accept() failures: No such file or directory" in the paniclog. (When I was killing off RC0 to swap back to 4.94.2). Has happened with both OpenSSL 1.1.1d-freebsd and OpenSSL 1.1.1k from FreeBSD ports - I felt it might be TLS related - or malicious, but you can see above "trustworthy" mailers are encoutering issues. Another host (not jailed this time) FreeBSD 12.2-RELEASE-p7 where the jail sends mail started to fail after about 2 days: 2021-07-19 10:11:32 SMTP connection from malodar.frost.net [2a02:1658:1::113:1] I=[2a02:8010:64d4::148]:25 lost D=2m39s 2021-07-19 10:11:32 SMTP connection from malodar.frost.net [2a02:1658:1::113:1] I=[2a02:8010:64d4::148]:25 lost D=43s 2021-07-19 10:11:32 SMTP connection from malodar.frost.net [2a02:1658:1::113:1] I=[2a02:8010:64d4::148]:25 lost D=3m15s 2021-07-19 10:11:32 SMTP connection from malodar.frost.net [2a02:1658:1::113:1] I=[2a02:8010:64d4::148]:25 lost D=34s (but I didn't see SEGV etc). Exim on the jail: Exim version 4.95-RC0 uid=0 gid=0 pid=96610 D=f7715cfd Support for: crypteq IPv6 use_setclassresources PAM TCPwrappers OpenSSL TLS_resume Content_Scanning DANE DKIM DNSSEC Event OCSP PIPE_CONNECT PRDR Experimental_Queue_Ramp SPF SRS TCP_Fast_Open Experimental_ARC Lookups (built-in): lsearch wildlsearch nwildlsearch iplsearch dbm dbmjz dbmnz dnsdb dsearch Authenticators: cyrus_sasl plaintext Routers: accept dnslookup ipliteral manualroute queryprogram redirect Transports: appendfile/maildir autoreply pipe smtp Malware: f-protd f-prot6d drweb aveserver fsecure kavdaemon sophie clamd mksd avast sock cmdline Configure owner: 0:0 Size of off_t: 8 Compiler: CLang [8.0.1 (tags/RELEASE_801/final 366581)] Probably Berkeley DB version 1.8x (native mode) Library version: OpenSSL: Compile: OpenSSL 1.1.1d 10 Sep 2019 Runtime: OpenSSL 1.1.1k 25 Mar 2021 : built on: Sat Jul 17 12:20:27 2021 UTC Library version: spf2: Compile: 1.2.10 Runtime: 1.2.10 Library version: Cyrus SASL: Compile: 2.1.27 Runtime: 2.1.27 [Cyrus SASL] Library version: PCRE: Compile: 8.44 Runtime: 8.44 2020-02-12 Rolling back to 4.94.2 and everthing that was failing comes flooding in fine. Open to suggestions on how to meaningfully work out what's going on (my gut says something TLS related, but I could be totally wrong - I can see FreeBSD mentioned doing a cursory diff of the sources and mention in the ChangeLog). Matthew. -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] 4.95 RC0 - gnutls outgoing TLS cert verification broken
On 23/07/2021 12:14, Heiko Schlittermann via Exim-users wrote: Andreas Metzler via Exim-users (Fr 23 Jul 2021 07:56:30 CEST): Good morning, thank you, looks good and works for me with GnuTLS 3.7.1. I did not test the fallback though. (Even Debian LTS - Stretch/Debian 9 has GnuTLS 3.5.x). Thanks, as soon as it is on master, I'll prepare RC1. Committed; dbbc1c20b5 -- Cheers, Jeremy -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] 4.95 RC0 - gnutls outgoing TLS cert verification broken
Andreas Metzler via Exim-users (Fr 23 Jul 2021 07:56:30 CEST): > Good morning, > > thank you, looks good and works for me with GnuTLS 3.7.1. I did not test > the fallback though. (Even Debian LTS - Stretch/Debian 9 has GnuTLS > 3.5.x). Thanks, as soon as it is on master, I'll prepare RC1. -- Heiko signature.asc Description: PGP signature -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] 4.95 RC0 - gnutls outgoing TLS cert verification broken
On 2021-07-22 Jeremy Harris via Exim-users wrote: > On 19/07/2021 07:29, Andreas Metzler via Exim-users wrote: [...] >> SUPPORT_SYSDEFAULT_CABUNDLE is #defined in src/tls-gnu.c >> #if GNUTLS_VERSION_NUMBER >= 0x030014 >> # define SUPPORT_SYSDEFAULT_CABUNDLE >> #endif >> but checked for in (in vain) in src/transports/smtp.c and src/globals.c. > Thanks for tracing this. > Proposed fix attached. [...] Good morning, thank you, looks good and works for me with GnuTLS 3.7.1. I did not test the fallback though. (Even Debian LTS - Stretch/Debian 9 has GnuTLS 3.5.x). cu Andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
