[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
This bug was fixed in the package exim4 - 4.95-4ubuntu3
---
exim4 (4.95-4ubuntu3) kinetic; urgency=medium
* d/p/lp1974214-segfault-smtp-delivery-0{1,2}.patch: Fix segfault when
there's an SMTP delivery attempt following a deferral. (LP: #1974214)
-- Sergio Durigan Junior Fri, 03 Jun
2022 17:37:10 -0400
** Changed in: exim4 (Ubuntu Kinetic)
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1974214
Title:
Segfaults on verify callout, in _gnutls_trust_list_get_issuer
To manage notifications about this bug go to:
https://bugs.launchpad.net/exim/+bug/1974214/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
** Description changed: [ Impact ] When the user sends a message to someone, if the server responsible for receiving this message defers it, and if there are other possible servers (i.e., other servers listed as secondary MX) to try, exim4 will segfault while trying to connect to the second server. [ Test Plan ] The test case for this bug is a bit involved. It makes use of the upstream reporter's mail server, which has been configured to defer emails when they come through the primary MX, but accept when they come through the secondary MX. This means that you will need access to port 25 (unfortunately canonistack seems to block it). $ lxc launch ubuntu-daily:jammy exim4-bug1974214 $ lxc shell exim4-bug1974214 # apt update && apt full-upgrade # apt install -y exim4 # dpkg-reconfigure exim4-config ... In the "Mail Server configuration" screen, select "internet site; mail is sent and received directly using SMTP". Leave everything else untouched. # cat > /etc/netplan/99-disable-ipv6.yaml << _EOF_ network: ethernets: eth0: link-local: [ ipv4 ] _EOF_ # netplan apply # reboot $ lxc shell exim4-bug1974214 # cat > 1.msg << _EOF_ Subject: test this is a test _EOF_ # exim4 -odq -f defe...@example.com geda...@gedalya.net < 1.msg # exim4 -bp 0m 321 1nxC3o-Ax-AS geda...@gedalya.net ... You will have to grab the message ID, which is 1nxC3o-Ax-AS in this case. You have to use this ID in the following command. # exim4 -d+all -q 1nxC3o-Ax-AS 2>&1 | tee /tmp/exim.debug ... # grep SEGV /tmp/exim.debug You should be able to see exim4 signalling the segmentation fault that occurred while attempting to connect to the second server. [ Where problems could occur ] The patches, albeit well contained and relatively simple, touch code that deals with TLS and security. There is always the risk of introducing an unwanted vulnerability or normal regression here. If that happens, the very first thing we need to do is revert the patches and work with upstream to develop a fix. + + In the unlikely case that we encounter regressions, they are probably + going to affect TLS connections when sending/receiving messages. Email + servers nowadays generally offer encrypted connections (via TLS or + STARTTLS), and some still offer plaintext as well. If there is a + problem with TLS and exim4 is configured to fallback to plaintext, + things will still work assuming that the other end also talks plaintext. + Otherwise, we might see reports of undelivered emails. + + Finally, the fix is composed of two patches. The first one prevents + exim4 from discarding the cached credentials when the transport + connection with the primary MX closes, and the second resets headers + before trying to connect to the secondary MX. [ Original Description ] We are experiencing segfaults in exim since upgrading from impish (4.94.2-7ubuntu2 with libgnutls30 3.7.1-5ubuntu1) to jammy (4.95-4ubuntu2 with libgnutls30 3.7.3-4ubuntu1), in _gnutls_trust_list_get_issuer, seemingly in the sender/recipient verify callout during message submission. Typically the initial attempt to submit a message crashes an exim child thread, but the same message is accepted when the sender retries. gdb backtrace: Thread 2.1 "exim4" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fe2f844d080 (LWP 29278)] 0x7fe2f8f3eb2b in _gnutls_trust_list_get_issuer (flags=, issuer=, cert=, list=) at x509/../../../lib/x509/verify-high.c:1026 1026x509/../../../lib/x509/verify-high.c: No such file or directory. (gdb) bt #0 0x7fe2f8f3eb2b in _gnutls_trust_list_get_issuer (flags=, issuer=, cert=, list=) at x509/../../../lib/x509/verify-high.c:1026 #1 gnutls_x509_trust_list_get_issuer (list=list@entry=0x55ef6bd9c260, cert=0x55ef6bd9be20, issuer=issuer@entry=0x7ffc82dba510, flags=flags@entry=16) at x509/../../../lib/x509/verify-high.c:1129 #2 0x7fe2f8f3f679 in gnutls_x509_trust_list_verify_crt2 (list=0x55ef6bd9c260, cert_list=0x7ffc82dba5c0, cert_list_size=, data=, elements=, flags=33554432, voutput=0x7ffc82dba888, func=0x0) at x509/../../../lib/x509/verify-high.c:1522 #3 0x7fe2f8ed7516 in _gnutls_x509_cert_verify_peers (status=0x7ffc82dba888, elements=0, data=0x0, session=0x55ef6c0c1150) at ../../lib/cert-session.c:597 #4 gnutls_certificate_verify_peers (session=0x55ef6c0c1150, data=data@entry=0x0, elements=elements@entry=0, status=status@entry=0x7ffc82dba888) at ../../lib/cert-session.c:776 #5 0x7fe2f8ed8000 in gnutls_certificate_verify_peers2 (session=, status=status@entry=0x7ffc82dba888) at ../../lib/cert-session.c:653 #6 0x55ef6b7698ef in verify_certificate (state=, errstr=0x7ffc82dbaa20) at /build/exim4-sMcKLv/exim4-4.95/b-exim4-daemon-light/build-Linux-x86_64/tls-gnu.c:2
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
** Description changed: [ Impact ] When the user sends a message to someone, if the server responsible for receiving this message defers it, and if there are other possible servers (i.e., other servers listed as secondary MX) to try, exim4 will segfault while trying to connect to the second server. - [ Test Case ] + [ Test Plan ] The test case for this bug is a bit involved. It makes use of the upstream reporter's mail server, which has been configured to defer emails when they come through the primary MX, but accept when they come through the secondary MX. This means that you will need access to port 25 (unfortunately canonistack seems to block it). $ lxc launch ubuntu-daily:jammy exim4-bug1974214 $ lxc shell exim4-bug1974214 # apt update && apt full-upgrade # apt install -y exim4 # dpkg-reconfigure exim4-config ... In the "Mail Server configuration" screen, select "internet site; mail is sent and received directly using SMTP". Leave everything else untouched. # cat > /etc/netplan/99-disable-ipv6.yaml << _EOF_ network: - ethernets: - eth0: - link-local: [ ipv4 ] + ethernets: + eth0: + link-local: [ ipv4 ] _EOF_ # netplan apply # reboot $ lxc shell exim4-bug1974214 # cat > 1.msg << _EOF_ Subject: test this is a test _EOF_ # exim4 -odq -f defe...@example.com geda...@gedalya.net < 1.msg # exim4 -bp - 0m 321 1nxC3o-Ax-AS - geda...@gedalya.net + 0m 321 1nxC3o-Ax-AS + geda...@gedalya.net ... You will have to grab the message ID, which is 1nxC3o-Ax-AS in this case. You have to use this ID in the following command. # exim4 -d+all -q 1nxC3o-Ax-AS 2>&1 | tee /tmp/exim.debug ... # grep SEGV /tmp/exim.debug You should be able to see exim4 signalling the segmentation fault that occurred while attempting to connect to the second server. [ Where problems could occur ] The patches, albeit well contained and relatively simple, touch code that deals with TLS and security. There is always the risk of introducing an unwanted vulnerability or normal regression here. If that happens, the very first thing we need to do is revert the patches and work with upstream to develop a fix. [ Original Description ] We are experiencing segfaults in exim since upgrading from impish (4.94.2-7ubuntu2 with libgnutls30 3.7.1-5ubuntu1) to jammy (4.95-4ubuntu2 with libgnutls30 3.7.3-4ubuntu1), in _gnutls_trust_list_get_issuer, seemingly in the sender/recipient verify callout during message submission. Typically the initial attempt to submit a message crashes an exim child thread, but the same message is accepted when the sender retries. gdb backtrace: Thread 2.1 "exim4" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fe2f844d080 (LWP 29278)] 0x7fe2f8f3eb2b in _gnutls_trust_list_get_issuer (flags=, issuer=, cert=, list=) at x509/../../../lib/x509/verify-high.c:1026 1026x509/../../../lib/x509/verify-high.c: No such file or directory. (gdb) bt #0 0x7fe2f8f3eb2b in _gnutls_trust_list_get_issuer (flags=, issuer=, cert=, list=) at x509/../../../lib/x509/verify-high.c:1026 #1 gnutls_x509_trust_list_get_issuer (list=list@entry=0x55ef6bd9c260, cert=0x55ef6bd9be20, issuer=issuer@entry=0x7ffc82dba510, flags=flags@entry=16) at x509/../../../lib/x509/verify-high.c:1129 #2 0x7fe2f8f3f679 in gnutls_x509_trust_list_verify_crt2 (list=0x55ef6bd9c260, cert_list=0x7ffc82dba5c0, cert_list_size=, data=, elements=, flags=33554432, voutput=0x7ffc82dba888, func=0x0) at x509/../../../lib/x509/verify-high.c:1522 #3 0x7fe2f8ed7516 in _gnutls_x509_cert_verify_peers (status=0x7ffc82dba888, elements=0, data=0x0, session=0x55ef6c0c1150) at ../../lib/cert-session.c:597 #4 gnutls_certificate_verify_peers (session=0x55ef6c0c1150, data=data@entry=0x0, elements=elements@entry=0, status=status@entry=0x7ffc82dba888) at ../../lib/cert-session.c:776 #5 0x7fe2f8ed8000 in gnutls_certificate_verify_peers2 (session=, status=status@entry=0x7ffc82dba888) at ../../lib/cert-session.c:653 #6 0x55ef6b7698ef in verify_certificate (state=, errstr=0x7ffc82dbaa20) at /build/exim4-sMcKLv/exim4-4.95/b-exim4-daemon-light/build-Linux-x86_64/tls-gnu.c:2519 #7 0x55ef6b7a5d7b in tls_client_start.constprop.0 (cctx=cctx@entry=0x55ef6be0e688, conn_args=conn_args@entry=0x55ef6bdfe5f8, tlsp=0x55ef6b7f59c0 , errstr=errstr@entry=0x7ffc82dbaa20, cookie=) at /build/exim4-sMcKLv/exim4-4.95/b-exim4-daemon-light/build-Linux-x86_64/tls-gnu.c:3593 #8 0x55ef6b78b0ef in smtp_setup_conn (sx=0x55ef6bdfe5e8, suppress_tls=) at transports/smtp.c:2673 #9 0x55ef6b776350 in do_callout (pm_mailfrom=, se_mailfrom=, options=, callout_connect=, callout_overall=, callout=, tf=0x7ffc82dbbc10, host_list=, addr=0x7ffc82dbbdd0) at
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
** Changed in: exim4 (Debian) Status: Unknown => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1974214 Title: Segfaults on verify callout, in _gnutls_trust_list_get_issuer To manage notifications about this bug go to: https://bugs.launchpad.net/exim/+bug/1974214/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
** Merge proposal linked: https://code.launchpad.net/~sergiodj/ubuntu/+source/exim4/+git/exim4/+merge/423963 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1974214 Title: Segfaults on verify callout, in _gnutls_trust_list_get_issuer To manage notifications about this bug go to: https://bugs.launchpad.net/exim/+bug/1974214/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
Launchpad has imported 48 comments from the remote bug at
http://bugs.exim.org/show_bug.cgi?id=2886.
If you reply to an imported comment from within Launchpad, your comment
will be sent to the remote bug automatically. Read more about
Launchpad's inter-bugtracker facilities at
https://help.launchpad.net/InterBugTracking.
On 2022-05-11T18:41:54+00:00 Gedalya-b wrote:
Created attachment 1414
exim4-daemon-light 4.96~RC0-1
Versions: 4.94 good, 4.9[56] bad.
OS: Debian testing x86_64
TLS: gnutls 3.7.4-2, haven't tried OpenSSL
This doesn't happen in an immediate delivery attempt or in "exim -M",
but in a queue runner, if the first remote server responds with a
deferral, exim crashes some time when talking to the next server. It can
happen in tls_close() (after the message was successfully delivered, if
the remote parry allows, or after getting a deferral): smtp_deliver ->
tls_close -> gnutls_certificate_free_credentials ->
gnutls_x509_trust_list_deinit,
or it can be: smtp_deliver -> smtp_setup_conn -> tls_client_start ->
verify_certificate -> gnutls_certificate_verify_peers2 ->
gnutls_certificate_verify_peers -> _gnutls_x509_cert_verify_peers ->
gnutls_x509_trust_list_verify_crt2 -> gnutls_x509_trust_list_get_issuer
-> _gnutls_trust_list_get_issuer
And on one occasion it crashed in arc_sign() (on an exim thus built)
But long story short, it seems like exim would consistently crash,
SIGFPE or SIGSEGV, during a subsequent delivery attempt after a deferral
response.
The real life circumstances are a gmail account over quota or a
forwarded message graylisted by gmail or such. But I am reproducing this
by simply configuring my own exim servers to defer.
Attached is a log excerpt and backtrace from Debian's exim4-daemon-light
4.96~RC0-1
Reply at:
https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/comments/0
On 2022-05-11T18:55:58+00:00 Gedalya-b wrote:
Created attachment 1415
backtrace for crash during ARC signing
Adding a backtrace for crash in arc_sign()
Reply at:
https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/comments/1
On 2022-05-11T19:02:45+00:00 Gedalya-b wrote:
The deferrals are either in response to RCPT TO (gmail over quota) or
post DATA (suspicious content)
Reply at:
https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/comments/2
On 2022-05-12T12:50:56+00:00 Jgh146exb wrote:
This needs following up; we can't trust that bt
> warning: core file may not match specified executable file.
Reply at:
https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/comments/3
On 2022-05-12T13:02:51+00:00 Gedalya-b wrote:
If I go out of my way to invoke "/usr/sbin/exim4 -q" when causing the crash,
that message is not displayed.
On Debian, /usr/sbin/exim -> exim4. One naturally uses "exim -q", and gdb gives
that message, every single time. The rest of the bt seems unaffected.
Reply at:
https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/comments/4
On 2022-05-12T13:06:48+00:00 Gedalya-b wrote:
Either way I think this should be very easily reproducible, I've tried
several boxes, several custom builds, with or without ARC, DMARC, MySQL
and so on, exim 4.95 and 4.96, the only thing in common was that I was
using Debian's packaging and stuck with gnutls.
Reply at:
https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/comments/5
On 2022-05-12T14:32:56+00:00 Jgh146exb wrote:
I wonder if your build is failing to null-fill not-specifically-initialized
file-scope statics?
#0 0x5620ff359503 in arc_sign (signspec=,
sigheaders=0x5621001fc580, errstr=errstr@entry=0x7ffd081d0980) at
./b-exim4-daemon-custom/build-Linux-x86_64/arc.c:1663
1660 if ((rheaders = arc_sign_scan_headers(&arc_sign_ctx, sigheaders)))
1661 {
1662 hdr_rlist ** rp;
1663 for (rp = &headers_rlist; *rp; ) rp = &(*rp)->prev;
1664 *rp = rheaders;
1665 }
What do "p *rp" and "p headers_rlist" say for that core?
(On that theory, an "=NULL" at line 93 would help. But only for the arcsigning
case).
Reply at:
https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/comments/6
On 2022-05-12T14:38:25+00:00 Gedalya-b wrote:
I didn't keep that exact same core, but I have a new one that seems
equivalent.
#0 0x559e51e3 in arc_sign (signspec=,
sigheaders=0x559e529bb580, errstr=errstr@entry=0x7ffcc0624e20) at
./b-exim4-daemon-custom/build-Linux-x86_64/arc.c:1663
rp = 0x6
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
** Merge proposal linked: https://code.launchpad.net/~sergiodj/ubuntu/+source/exim4/+git/exim4/+merge/423962 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1974214 Title: Segfaults on verify callout, in _gnutls_trust_list_get_issuer To manage notifications about this bug go to: https://bugs.launchpad.net/exim/+bug/1974214/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
** Tags added: server-todo ** Changed in: exim4 (Ubuntu Jammy) Status: Triaged => In Progress ** Changed in: exim4 (Ubuntu Kinetic) Status: Triaged => In Progress ** Bug watch added: Debian Bug tracker #1004740 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1004740 ** Also affects: exim4 (Debian) via https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1004740 Importance: Unknown Status: Unknown ** Also affects: exim via http://bugs.exim.org/show_bug.cgi?id=2886 Importance: Unknown Status: Unknown -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1974214 Title: Segfaults on verify callout, in _gnutls_trust_list_get_issuer To manage notifications about this bug go to: https://bugs.launchpad.net/exim/+bug/1974214/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
** Description changed: + [ Impact ] + + When the user sends a message to someone, if the server responsible for + receiving this message defers it, and if there are other possible + servers (i.e., other servers listed as secondary MX) to try, exim4 will + segfault while trying to connect to the second server. + + [ Test Case ] + + The test case for this bug is a bit involved. It makes use of the + upstream reporter's mail server, which has been configured to defer + emails when they come through the primary MX, but accept when they come + through the secondary MX. This means that you will need access to port + 25 (unfortunately canonistack seems to block it). + + $ lxc launch ubuntu-daily:jammy exim4-bug1974214 + $ lxc shell exim4-bug1974214 + # apt update && apt full-upgrade + # apt install -y exim4 + # dpkg-reconfigure exim4-config + ... In the "Mail Server configuration" screen, select "internet site; mail is sent and received directly using SMTP". Leave everything else untouched. + # cat > /etc/netplan/99-disable-ipv6.yaml << _EOF_ + network: + ethernets: + eth0: + link-local: [ ipv4 ] + _EOF_ + # netplan apply + # reboot + $ lxc shell exim4-bug1974214 + # cat > 1.msg << _EOF_ + Subject: test + + this is a test + _EOF_ + # exim4 -odq -f defe...@example.com geda...@gedalya.net < 1.msg + # exim4 -bp + 0m 321 1nxC3o-Ax-AS + geda...@gedalya.net + + ... You will have to grab the message ID, which is 1nxC3o-Ax-AS in + this case. You have to use this ID in the following command. + + # exim4 -d+all -q 1nxC3o-Ax-AS 2>&1 | tee /tmp/exim.debug + ... + # grep SEGV /tmp/exim.debug + + You should be able to see exim4 signalling the segmentation fault that + occurred while attempting to connect to the second server. + + [ Where problems could occur ] + + The patches, albeit well contained and relatively simple, touch code + that deals with TLS and security. There is always the risk of + introducing an unwanted vulnerability or normal regression here. If + that happens, the very first thing we need to do is revert the patches + and work with upstream to develop a fix. + + [ Original Description ] + We are experiencing segfaults in exim since upgrading from impish (4.94.2-7ubuntu2 with libgnutls30 3.7.1-5ubuntu1) to jammy (4.95-4ubuntu2 with libgnutls30 3.7.3-4ubuntu1), in _gnutls_trust_list_get_issuer, seemingly in the sender/recipient verify callout during message submission. Typically the initial attempt to submit a message crashes an exim child thread, but the same message is accepted when the sender retries. gdb backtrace: Thread 2.1 "exim4" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fe2f844d080 (LWP 29278)] 0x7fe2f8f3eb2b in _gnutls_trust_list_get_issuer (flags=, issuer=, cert=, list=) at x509/../../../lib/x509/verify-high.c:1026 1026x509/../../../lib/x509/verify-high.c: No such file or directory. (gdb) bt #0 0x7fe2f8f3eb2b in _gnutls_trust_list_get_issuer (flags=, issuer=, cert=, list=) at x509/../../../lib/x509/verify-high.c:1026 #1 gnutls_x509_trust_list_get_issuer (list=list@entry=0x55ef6bd9c260, cert=0x55ef6bd9be20, issuer=issuer@entry=0x7ffc82dba510, flags=flags@entry=16) at x509/../../../lib/x509/verify-high.c:1129 #2 0x7fe2f8f3f679 in gnutls_x509_trust_list_verify_crt2 (list=0x55ef6bd9c260, cert_list=0x7ffc82dba5c0, cert_list_size=, data=, elements=, flags=33554432, voutput=0x7ffc82dba888, func=0x0) at x509/../../../lib/x509/verify-high.c:1522 #3 0x7fe2f8ed7516 in _gnutls_x509_cert_verify_peers (status=0x7ffc82dba888, elements=0, data=0x0, session=0x55ef6c0c1150) at ../../lib/cert-session.c:597 #4 gnutls_certificate_verify_peers (session=0x55ef6c0c1150, data=data@entry=0x0, elements=elements@entry=0, status=status@entry=0x7ffc82dba888) at ../../lib/cert-session.c:776 #5 0x7fe2f8ed8000 in gnutls_certificate_verify_peers2 (session=, status=status@entry=0x7ffc82dba888) at ../../lib/cert-session.c:653 #6 0x55ef6b7698ef in verify_certificate (state=, errstr=0x7ffc82dbaa20) at /build/exim4-sMcKLv/exim4-4.95/b-exim4-daemon-light/build-Linux-x86_64/tls-gnu.c:2519 #7 0x55ef6b7a5d7b in tls_client_start.constprop.0 (cctx=cctx@entry=0x55ef6be0e688, conn_args=conn_args@entry=0x55ef6bdfe5f8, tlsp=0x55ef6b7f59c0 , errstr=errstr@entry=0x7ffc82dbaa20, cookie=) at /build/exim4-sMcKLv/exim4-4.95/b-exim4-daemon-light/build-Linux-x86_64/tls-gnu.c:3593 #8 0x55ef6b78b0ef in smtp_setup_conn (sx=0x55ef6bdfe5e8, suppress_tls=) at transports/smtp.c:2673 #9 0x55ef6b776350 in do_callout (pm_mailfrom=, se_mailfrom=, options=, callout_connect=, callout_overall=, callout=, tf=0x7ffc82dbbc10, host_list=, addr=0x7ffc82dbbdd0) at /build/exim4-sMcKLv/exim4-4.95/b-exim4-daemon-light/build-Linux-x86_64/verify.c:677 #10 verify_address (vaddr=, fp=, options=, cal
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
Thank you for testing the package from the PPA and providing feedback, Malcom and Raf. I took some time today to try and come up with a reproducer for the bug, since this is one of the requirements for a successful SRU to Jammy. So far I haven't had any success, but I'll try to continue working on this tomorrow. I found some possible reproducer in the upstream bug but none of them worked OOTB for me. Meanwhile, if any of you already have a simple reproducer that'd be great. Thanks. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1974214 Title: Segfaults on verify callout, in _gnutls_trust_list_get_issuer To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
** Changed in: exim4 (Ubuntu Kinetic) Assignee: (unassigned) => Sergio Durigan Junior (sergiodj) ** Changed in: exim4 (Ubuntu Jammy) Assignee: (unassigned) => Sergio Durigan Junior (sergiodj) ** Changed in: exim4 (Ubuntu Jammy) Importance: Medium => High ** Changed in: exim4 (Ubuntu Kinetic) Importance: Medium => High -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1974214 Title: Segfaults on verify callout, in _gnutls_trust_list_get_issuer To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
Previously exim would get SIGFPE on each mail delivery attempt. The PPA version has been installed for almost 2 days and no more crashes. Thanks! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1974214 Title: Segfaults on verify callout, in _gnutls_trust_list_get_issuer To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
We've been running Sergio's exim 4.95-4ubuntu3~ppa2 for 27 hours so far with no segfaults (previously it was segfaulting every few minutes) -- looks like the bug is fixed with those patches; thanks! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1974214 Title: Segfaults on verify callout, in _gnutls_trust_list_get_issuer To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
On Thursday, May 26 2022, Malcolm Scott wrote: > Hi Sergio, unfortunately your patched package does not fix the problem > -- we still see segfaults. > > From the upstream discussion, I think we may need this commit as well: > > https://git.exim.org/exim.git/commitdiff/8c74b00980bc7e3e479e8dfcd7c0008b2ac3f543 Ah, you're right, I missed this other commit. I've backported it and uploaded the new package to the PPA. Unfortunately LP is experiencing some problems so it may take a while for the build to complete. Cheers, -- Sergio GPG key ID: E92F D0B3 6B14 F1F4 D8E0 EB2F 106D A1C8 C3CB BF14 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1974214 Title: Segfaults on verify callout, in _gnutls_trust_list_get_issuer To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
Hi Sergio, unfortunately your patched package does not fix the problem -- we still see segfaults. From the upstream discussion, I think we may need this commit as well: https://git.exim.org/exim.git/commitdiff/8c74b00980bc7e3e479e8dfcd7c0008b2ac3f543 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1974214 Title: Segfaults on verify callout, in _gnutls_trust_list_get_issuer To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
The corresponding commit that supposedly fixes this issue is: https://git.exim.org/exim.git/commitdiff/5a8015582376ff3cc0c0d034d9237008b10d2164 I haven't tried to reproduce the problem, but if it's affecting Jammy then we will need to SRU the fix, which means that we need a reproducer first. I took the liberty to backport the upstream patch and prepare a PPA build with it. Malcom, would you be able to test the following PPA and let us know if it fixes the problem? https://launchpad.net/~sergiodj/+archive/ubuntu/exim4-bug1974214 Thanks. ** Also affects: exim4 (Ubuntu Jammy) Importance: Undecided Status: New ** Also affects: exim4 (Ubuntu Kinetic) Importance: Medium Status: Triaged ** Changed in: exim4 (Ubuntu Jammy) Status: New => Triaged ** Changed in: exim4 (Ubuntu Jammy) Importance: Undecided => Medium -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1974214 Title: Segfaults on verify callout, in _gnutls_trust_list_get_issuer To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
It looks like this is indeed an exim issue that was fixed in a recent update. exim bug report can be found at: https://bugs.exim.org/show_bug.cgi?id=2886 ** Bug watch added: bugs.exim.org/ #2886 http://bugs.exim.org/show_bug.cgi?id=2886 ** Changed in: exim4 (Ubuntu) Importance: Undecided => Medium ** Changed in: exim4 (Ubuntu) Status: Confirmed => Triaged ** No longer affects: gnutls28 (Ubuntu) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1974214 Title: Segfaults on verify callout, in _gnutls_trust_list_get_issuer To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
I have forwarded this bug to upstream at https://gitlab.com/gnutls/gnutls/-/issues/1374 ** Bug watch added: gitlab.com/gnutls/gnutls/-/issues #1374 https://gitlab.com/gnutls/gnutls/-/issues/1374 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1974214 Title: Segfaults on verify callout, in _gnutls_trust_list_get_issuer To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
You are right, 3.7.3-4 from jammy already contains the fix. I am suspecting that the fix might be the cause of your segfault since this was the last change in this part of the code and it seems to be a regression introduced in jammy. It looks like this bug hasn't been reported upstream yet. so we should probably take the discussion there. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1974214 Title: Segfaults on verify callout, in _gnutls_trust_list_get_issuer To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
@tobhe Thanks for looking into this. However that upstream bug was apparently fixed in GnuTLS 3.7.4; I just tried libgnutls30 3.7.4-2ubuntu1 from kinetic and I still see these crashes in exim4. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1974214 Title: Segfaults on verify callout, in _gnutls_trust_list_get_issuer To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/exim4/+bug/1974214/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1974214] Re: Segfaults on verify callout, in _gnutls_trust_list_get_issuer
** Summary changed: - Segfaults on sender verify callout, in _gnutls_trust_list_get_issuer + Segfaults on verify callout, in _gnutls_trust_list_get_issuer ** Description changed: We are experiencing segfaults in exim since upgrading from impish (4.94.2-7ubuntu2 with libgnutls30 3.7.1-5ubuntu1) to jammy (4.95-4ubuntu2 with libgnutls30 3.7.3-4ubuntu1), in - _gnutls_trust_list_get_issuer, seemingly in the sender verify callout - during message submission. + _gnutls_trust_list_get_issuer, seemingly in the sender/recipient verify + callout during message submission. Typically the initial attempt to submit a message crashes an exim child thread, but the same message is accepted when the sender retries. gdb backtrace: Thread 2.1 "exim4" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fe2f844d080 (LWP 29278)] 0x7fe2f8f3eb2b in _gnutls_trust_list_get_issuer (flags=, issuer=, cert=, list=) at x509/../../../lib/x509/verify-high.c:1026 1026x509/../../../lib/x509/verify-high.c: No such file or directory. (gdb) bt #0 0x7fe2f8f3eb2b in _gnutls_trust_list_get_issuer (flags=, issuer=, cert=, - list=) at x509/../../../lib/x509/verify-high.c:1026 + list=) at x509/../../../lib/x509/verify-high.c:1026 #1 gnutls_x509_trust_list_get_issuer (list=list@entry=0x55ef6bd9c260, cert=0x55ef6bd9be20, issuer=issuer@entry=0x7ffc82dba510, - flags=flags@entry=16) at x509/../../../lib/x509/verify-high.c:1129 + flags=flags@entry=16) at x509/../../../lib/x509/verify-high.c:1129 #2 0x7fe2f8f3f679 in gnutls_x509_trust_list_verify_crt2 (list=0x55ef6bd9c260, cert_list=0x7ffc82dba5c0, - cert_list_size=, data=, elements=, flags=33554432, voutput=0x7ffc82dba888, func=0x0) - at x509/../../../lib/x509/verify-high.c:1522 + cert_list_size=, data=, elements=, flags=33554432, voutput=0x7ffc82dba888, func=0x0) + at x509/../../../lib/x509/verify-high.c:1522 #3 0x7fe2f8ed7516 in _gnutls_x509_cert_verify_peers (status=0x7ffc82dba888, elements=0, data=0x0, session=0x55ef6c0c1150) - at ../../lib/cert-session.c:597 + at ../../lib/cert-session.c:597 #4 gnutls_certificate_verify_peers (session=0x55ef6c0c1150, data=data@entry=0x0, elements=elements@entry=0, - status=status@entry=0x7ffc82dba888) at ../../lib/cert-session.c:776 + status=status@entry=0x7ffc82dba888) at ../../lib/cert-session.c:776 #5 0x7fe2f8ed8000 in gnutls_certificate_verify_peers2 (session=, status=status@entry=0x7ffc82dba888) - at ../../lib/cert-session.c:653 + at ../../lib/cert-session.c:653 #6 0x55ef6b7698ef in verify_certificate (state=, errstr=0x7ffc82dbaa20) - at /build/exim4-sMcKLv/exim4-4.95/b-exim4-daemon-light/build-Linux-x86_64/tls-gnu.c:2519 + at /build/exim4-sMcKLv/exim4-4.95/b-exim4-daemon-light/build-Linux-x86_64/tls-gnu.c:2519 #7 0x55ef6b7a5d7b in tls_client_start.constprop.0 (cctx=cctx@entry=0x55ef6be0e688, conn_args=conn_args@entry=0x55ef6bdfe5f8, - tlsp=0x55ef6b7f59c0 , errstr=errstr@entry=0x7ffc82dbaa20, cookie=) - at /build/exim4-sMcKLv/exim4-4.95/b-exim4-daemon-light/build-Linux-x86_64/tls-gnu.c:3593 + tlsp=0x55ef6b7f59c0 , errstr=errstr@entry=0x7ffc82dbaa20, cookie=) + at /build/exim4-sMcKLv/exim4-4.95/b-exim4-daemon-light/build-Linux-x86_64/tls-gnu.c:3593 #8 0x55ef6b78b0ef in smtp_setup_conn (sx=0x55ef6bdfe5e8, suppress_tls=) at transports/smtp.c:2673 #9 0x55ef6b776350 in do_callout (pm_mailfrom=, se_mailfrom=, options=, - callout_connect=, callout_overall=, callout=, tf=0x7ffc82dbbc10, - host_list=, addr=0x7ffc82dbbdd0) - at /build/exim4-sMcKLv/exim4-4.95/b-exim4-daemon-light/build-Linux-x86_64/verify.c:677 + callout_connect=, callout_overall=, callout=, tf=0x7ffc82dbbc10, + host_list=, addr=0x7ffc82dbbdd0) + at /build/exim4-sMcKLv/exim4-4.95/b-exim4-daemon-light/build-Linux-x86_64/verify.c:677 #10 verify_address (vaddr=, fp=, options=, callout=, - callout_overall=, callout_connect=, se_mailfrom=, pm_mailfrom=, - routed=) at /build/exim4-sMcKLv/exim4-4.95/b-exim4-daemon-light/build-Linux-x86_64/verify.c:1947 + callout_overall=, callout_connect=, se_mailfrom=, pm_mailfrom=, + routed=) at /build/exim4-sMcKLv/exim4-4.95/b-exim4-daemon-light/build-Linux-x86_64/verify.c:1947 #11 0x55ef6b6f1660 in acl_verify (where=where@entry=0, addr=addr@entry=0x7ffc82dbc5e0, - arg=0x55ef6babc2b8 "recipient/defer_ok/callout=30s,defer_ok,use_postmaster", user_msgptr=user_msgptr@entry=0x7ffc82dbca50, - log_msgptr=log_msgptr@entry=0x7ffc82dbca58, basic_errno=basic_errno@entry=0x7ffc82dbc38c) - at /build/exim4-sMcKLv/exim4-4.95/b-exim4-daemon-light/build-Linux-x86_64/acl.c:2168 + arg=0x55ef6babc2b8 "recipient/defer_ok/callout=30s,defer_ok,use_postmaster", user_msgptr=user_msgptr@entry=0x7ffc82dbca50, + log_msgptr=log_msgptr@entry=0x7ffc82dbca58, basic_errno=basic_errno@entry=0x7ffc82dbc38c) + a
