Re: [exim] 4.95 RC0 - gnutls outgoing TLS cert verification broken
On 23/07/2021 12:14, Heiko Schlittermann via Exim-users wrote: Andreas Metzler via Exim-users (Fr 23 Jul 2021 07:56:30 CEST): Good morning, thank you, looks good and works for me with GnuTLS 3.7.1. I did not test the fallback though. (Even Debian LTS - Stretch/Debian 9 has GnuTLS 3.5.x). Thanks, as soon as it is on master, I'll prepare RC1. Committed; dbbc1c20b5 -- Cheers, Jeremy -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] 4.95 RC0 - gnutls outgoing TLS cert verification broken
Andreas Metzler via Exim-users (Fr 23 Jul 2021 07:56:30 CEST): > Good morning, > > thank you, looks good and works for me with GnuTLS 3.7.1. I did not test > the fallback though. (Even Debian LTS - Stretch/Debian 9 has GnuTLS > 3.5.x). Thanks, as soon as it is on master, I'll prepare RC1. -- Heiko signature.asc Description: PGP signature -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] 4.95 RC0 - gnutls outgoing TLS cert verification broken
On 2021-07-22 Jeremy Harris via Exim-users wrote: > On 19/07/2021 07:29, Andreas Metzler via Exim-users wrote: [...] >> SUPPORT_SYSDEFAULT_CABUNDLE is #defined in src/tls-gnu.c >> #if GNUTLS_VERSION_NUMBER >= 0x030014 >> # define SUPPORT_SYSDEFAULT_CABUNDLE >> #endif >> but checked for in (in vain) in src/transports/smtp.c and src/globals.c. > Thanks for tracing this. > Proposed fix attached. [...] Good morning, thank you, looks good and works for me with GnuTLS 3.7.1. I did not test the fallback though. (Even Debian LTS - Stretch/Debian 9 has GnuTLS 3.5.x). cu Andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] 4.95 RC0 - gnutls outgoing TLS cert verification broken
On 19/07/2021 07:29, Andreas Metzler via Exim-users wrote:
On 2021-07-19 Andreas Metzler wrote:
[...]
with 4.92.2
tls_verify_certificates = system
and 4.95 rc0
tls_verify_certificates =
Hello,
SUPPORT_SYSDEFAULT_CABUNDLE is #defined in src/tls-gnu.c
#if GNUTLS_VERSION_NUMBER >= 0x030014
# define SUPPORT_SYSDEFAULT_CABUNDLE
#endif
but checked for in (in vain) in src/transports/smtp.c and src/globals.c.
Thanks for tracing this.
Proposed fix attached.
--
Cheers,
Jeremy
diff --git a/src/src/globals.c b/src/src/globals.c
index 1e12bcb92..c3e8a16cf 100644
--- a/src/src/globals.c
+++ b/src/src/globals.c
@@ -147,11 +147,7 @@ uschar *tls_require_ciphers= NULL;
uschar *tls_resumption_hosts = NULL;
# endif
uschar *tls_try_verify_hosts = NULL;
-#if defined(SUPPORT_SYSDEFAULT_CABUNDLE) || !defined(USE_GNUTLS)
uschar *tls_verify_certificates= US"system";
-#else
-uschar *tls_verify_certificates= NULL;
-#endif
uschar *tls_verify_hosts = NULL;
int tls_watch_fd = -1;
time_t tls_watch_trigger_time = (time_t)0;
diff --git a/src/src/tls-gnu.c b/src/src/tls-gnu.c
index b9f8d2a51..7d434f6af 100644
--- a/src/src/tls-gnu.c
+++ b/src/src/tls-gnu.c
@@ -1526,9 +1526,14 @@ else if ( !tls_certificate && !tls_privatekey
else
DEBUG(D_tls) debug_printf("TLS: not preloading server certs\n");
-/* If tls_verify_certificates is non-empty and has no $, load CAs */
+/* If tls_verify_certificates is non-empty and has no $, load CAs.
+If none was configured and we can't handle "system", treat as empty. */
-if (opt_set_and_noexpand(tls_verify_certificates))
+if ( opt_set_and_noexpand(tls_verify_certificates)
+#ifndef SUPPORT_SYSDEFAULT_CABUNDLE
+ && Ustrcmp(tls_verify_certificates, "system") != 0
+#endif
+ )
{
if (tls_set_watch(tls_verify_certificates, FALSE))
{
@@ -1632,7 +1637,14 @@ else
DEBUG(D_tls)
debug_printf("TLS: not preloading client certs, for transport '%s'\n", t->name);
-if (opt_set_and_noexpand(ob->tls_verify_certificates))
+/* If tls_verify_certificates is non-empty and has no $, load CAs.
+If none was configured and we can't handle "system", treat as empty. */
+
+if ( opt_set_and_noexpand(ob->tls_verify_certificates)
+#ifndef SUPPORT_SYSDEFAULT_CABUNDLE
+ && Ustrcmp(ob->tls_verify_certificates, "system") != 0
+#endif
+ )
{
if (!watch || tls_set_watch(ob->tls_verify_certificates, FALSE))
{
@@ -1848,7 +1860,8 @@ else
provided. Experiment shows that, if the certificate file is empty, an unhelpful
error message is provided. However, if we just refrain from setting anything up
in that case, certificate verification fails, which seems to be the correct
-behaviour. */
+behaviour.
+If none was configured and we can't handle "system", treat as empty. */
if (!state->lib_state.cabundle)
{
diff --git a/src/src/transports/smtp.c b/src/src/transports/smtp.c
index c62de724d..a121e34ae 100644
--- a/src/src/transports/smtp.c
+++ b/src/src/transports/smtp.c
@@ -194,9 +194,7 @@ smtp_transport_options_block smtp_transport_option_defaults = {
.keepalive = TRUE,
.retry_include_ip_address = TRUE,
#ifndef DISABLE_TLS
-# if defined(SUPPORT_SYSDEFAULT_CABUNDLE) || !defined(USE_GNUTLS)
.tls_verify_certificates = US"system",
-# endif
.tls_dh_min_bits = EXIM_CLIENT_DH_DEFAULT_MIN_BITS,
.tls_tempfail_tryclear = TRUE,
.tls_try_verify_hosts = US"*",
diff --git a/test/runtest b/test/runtest
index 6f142c540..9bde9046f 100755
--- a/test/runtest
+++ b/test/runtest
@@ -1096,6 +1096,9 @@ RESET_AFTER_EXTRA_LINE_READ:
next if /^GnuTLS<2>: added \d+ protocols, \d+ ciphersuites, \d+ sig algos and \d+ groups into priority list$/;
next if /^GnuTLS<2>: (Disabling X.509 extensions|signing structure using RSA-SHA256)/;
next if /^GnuTLS.*(wrap_nettle_mpi_print|gnutls_subject_alt_names_get|get_alt_name)/;
+next if /^GnuTLS<[23]>: (p11|ASSERT: pkcs11.c|Initializing needed PKCS #11 modules)/;
+next if /^Added \d{3} certificate authorities/;
+next if /^TLS: not preloading CRL for server/;
# only kevent platforms (FreeBSD, OpenBSD) say this
next if /^watch dir/;
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] 4.95 RC0 - gnutls outgoing TLS cert verification broken
On 2021-07-19 Andreas Metzler wrote: [...] > with 4.92.2 > tls_verify_certificates = system > and 4.95 rc0 > tls_verify_certificates = Hello, SUPPORT_SYSDEFAULT_CABUNDLE is #defined in src/tls-gnu.c #if GNUTLS_VERSION_NUMBER >= 0x030014 # define SUPPORT_SYSDEFAULT_CABUNDLE #endif but checked for in (in vain) in src/transports/smtp.c and src/globals.c. I would suggest simply dropping support for this ancient gnutls, 3.0.20 was released 2012-06-05. cu Andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] 4.95 RC0 - gnutls outgoing TLS cert verification broken
On 2021-07-18 Andreas Metzler wrote: > Hello, > there seems to be some breakage in 4.95 RC0 with outgoing TLS, it fails > to verify the peer certificate: > > initialising GnuTLS as a client on fd 7 > GnuTLS global init required > initialising GnuTLS client session > Expanding various TLS configuration options for session credentials > TLS: basic cred init, client > TLS: no client certificate specified; okay > TLS: tls_verify_certificates not set or empty, ignoring ^ That seems to be the required hint. Given this transport smarthost_smtp: driver = smtp multi_domain hosts_require_tls = * tls_verify_hosts = * we get different output for »exim -bP transport smarthost_smtp | grep tls_verify_certificates« with 4.92.2 tls_verify_certificates = system and 4.95 rc0 tls_verify_certificates = And indeed explicitely setting "tls_verify_certificates = system" on the transport lets cert verification succeed. (This also applies to the main configuration option tls_verify_certificates, not just to the transport option. - It is also empty by default.) cu Andreas -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] 4.95 RC0 - gnutls outgoing TLS cert verification broken
On Sun, Jul 18, 2021 at 06:29:41PM +0200, Andreas Metzler via Exim-users wrote: > I do not think so. Both exim 4.94.2 and gnutls-cli and s_client[1] are > happy with the cert setup. It is a straightforward Let's Encrypt chain. > > 0 s:CN = vsrv21575.customer.vlinux.de >i:C = US, O = Let's Encrypt, CN = R3 > 1 s:C = US, O = Let's Encrypt, CN = R3 >i:C = US, O = Internet Security Research Group, CN = ISRG Root X1 > 2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1 >i:O = Digital Signature Trust Co., CN = DST Root CA X3 The self-signature on the DST Root CA X3 is SHA-1, any chance the new Exim discriminates against SHA-1 self-signed roots? This root CA expires on 2021-09-30... FWIW, OpenSSL will typically ignore the depth 2 certificate by finding the "ISRG X1" root in the local trust store. I don't know what GnuTLS does, or whether the ISRG Root is installed in the GnuTLS trust store on your system. -- Viktor. -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] 4.95 RC0 - gnutls outgoing TLS cert verification broken
On 2021-07-18 15:47, Jeremy Harris wrote: > On 18/07/2021 15:50, Andreas Metzler via Exim-users wrote: >> I am attaching both server and client logs. (Timezones are different, >> UTC vs. CEST). > Looks like it was an EC connection. The server seems to have had a pair > of cert files; one has "rsa" in the name so I'm guessing the other has > an EC cert? Hello Jeremy, yes that is correct. > What is in that file, and what would the full chain of certs from > CA to leaf be? The client is using the "system" CA bundle, > and saying "certificate issuer is unknown" - I'm wondering > if the knowelege of a cert intermediate between CA and leaf > is missing somewhere along the line. I do not think so. Both exim 4.94.2 and gnutls-cli and s_client[1] are happy with the cert setup. It is a straightforward Let's Encrypt chain. 0 s:CN = vsrv21575.customer.vlinux.de i:C = US, O = Let's Encrypt, CN = R3 1 s:C = US, O = Let's Encrypt, CN = R3 i:C = US, O = Internet Security Research Group, CN = ISRG Root X1 2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1 i:O = Digital Signature Trust Co., CN = DST Root CA X3 As it is a public server where one can grab the certs with e.g. gnutls-cli or s_client -showcerts I am not posting more detail for the sake of brevity. I can setup a /dev/null mailbox for testing if you want me to. cu Andreas [1] gnutls-cli --starttls-proto smtp vsrv21575.customer.vlinux.de openssl s_client -connect vsrv21575.customer.vlinux.de:25 -starttls smtp -verify_return_error -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] 4.95 RC0 - gnutls outgoing TLS cert verification broken
On 18/07/2021 15:50, Andreas Metzler via Exim-users wrote: I am attaching both server and client logs. (Timezones are different, UTC vs. CEST). Looks like it was an EC connection. The server seems to have had a pair of cert files; one has "rsa" in the name so I'm guessing the other has an EC cert? What is in that file, and what would the full chain of certs from CA to leaf be? The client is using the "system" CA bundle, and saying "certificate issuer is unknown" - I'm wondering if the knowelege of a cert intermediate between CA and leaf is missing somewhere along the line. -- Cheers, Jeremy -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] 4.95 RC0 - gnutls outgoing TLS cert verification broken
On 2021-07-18 Jeremy Harris via Exim-users wrote: > On 18/07/2021 14:09, Andreas Metzler via Exim-users wrote: >> there seems to be some breakage in 4.95 RC0 with outgoing TLS, it fails >> to verify the peer certificate: > Since we're happily running the test-suite this must be either an > environment difference or a build problem. > Did you give both client and server debug output there? Hello, No, it is only the (breaking) client. > It wasn't annotated... Part of it says "close on process exit" > right after a TLS handshake starts up, and I don't see any > reason given. > Could you manage to get "all" debug output all the way from > daemon startup? I'm hoping we'll see the PIDs, as well as > the credentials cache load. And also "all" debug for the > client? I temporarily switched local exim (4.95) to queueonly mode and injected a message. I changed the target remote port for the respective transport to 587 and started a debugging server remote (4.94.2) -oX on this port. Then I ran exim4 -d+all -M on the client-side. I am attaching both server and client logs. (Timezones are different, UTC vs. CEST). cu Andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' 14:27:32 21827 Exim version 4.94.2 uid=0 gid=0 pid=21827 D=fff9 Support for: crypteq iconv() IPv6 PAM Perl Expand_dlfunc GnuTLS move_frozen_messages Content_Scanning DANE DKIM DNSSEC Event I18N OCSP PIPE_CONNECT PRDR PROXY SOCKS TCP_Fast_Open Lookups (built-in): lsearch wildlsearch nwildlsearch iplsearch cdb dbm dbmjz dbmnz dnsdb dsearch ldap ldapdn ldapm mysql nis nis0 passwd pgsql sqlite Authenticators: cram_md5 cyrus_sasl dovecot plaintext spa tls Routers: accept dnslookup ipliteral iplookup manualroute queryprogram redirect Transports: appendfile/maildir/mailstore/mbx autoreply lmtp pipe smtp Malware: f-protd f-prot6d drweb fsecure sophie clamd avast sock cmdline Fixed never_users: 0 Configure owner: 0:0 Size of off_t: 8 Compiler: GCC [8.3.0] Library version: Glibc: Compile: 2.28 Runtime: 2.28 Library version: BDB: Compile: Berkeley DB 5.3.28: (September 9, 2013) Runtime: Berkeley DB 5.3.28: (September 9, 2013) Library version: GnuTLS: Compile: 3.6.7 Runtime: 3.6.7 Library version: IDN2: Compile: 2.0.5 Runtime: 2.0.5 Library version: Stringprep: Compile: 1.33 Runtime: 1.33 Library version: Cyrus SASL: Compile: 2.1.27 Runtime: 2.1.27 [Cyrus SASL] Library version: PCRE: Compile: 8.39 Runtime: 8.39 2016-06-14 14:27:32 21827 Total 20 lookups Library version: MySQL: Compile: 100329 10.3.29 [mariadb-10.3] Runtime: 100329 10.3.29 Library version: SQLite: Compile: 3.27.2 Runtime: 3.27.2 WHITELIST_D_MACROS: "OUTGOING" TRUSTED_CONFIG_LIST: "/etc/exim4/trusted_configs" 14:27:32 21827 changed uid/gid: forcing real = effective 14:27:32 21827 uid=0 gid=0 pid=21827 14:27:32 21827 auxiliary group list: 14:27:32 21827 seeking password data for user "uucp": cache not available 14:27:32 21827 getpwnam() succeeded uid=10 gid=10 14:27:32 21827 configuration file is /var/lib/exim4/config.autogenerated 14:27:32 21827 log selectors = cffc 19005022 000f 14:27:32 21827 cwd=/root 5 args: exim4 -bd -d+all -oX 587 14:27:32 21827 trusted user 14:27:32 21827 admin user 14:27:32 21827 dropping to exim gid; retaining priv uid 14:27:32 21827 seeking password data for user "mail": cache not available 14:27:32 21827 getpwnam() succeeded uid=8 gid=8 14:27:32 21827 ╭considering: $1 14:27:32 21827 ├──expanding: $1 14:27:32 21827 ╰─result: root 14:27:32 21827 user name "root" extracted from gecos field "root" 14:27:32 21827 originator: uid=0 gid=0 login=root name=root 14:27:32 21827 fresh-exec forking for cipher-validate 14:27:32 21827 fresh-exec forked for cipher-validate: 21829 14:27:32 21829 postfork: cipher-validate 14:27:32 21829 changed uid/gid: calling tls_validate_require_cipher 14:27:32 21829 uid=108 gid=111 pid=21829 14:27:32 21829 auxiliary group list: 14:27:32 21829 Exim pid=21829 (cipher-validate) terminating with rc=0 14:27:32 21827 tls_validate_require_cipher child 21829 ended: status=0x0 14:27:32 21827 daemon_smtp_port overridden by -oX: 14:27:32 21827 <: 587 14:27:32 21827 -oX used without -oP so not creating notifier socket 14:27:32 21827 listening on all interfaces (IPv6) port 587 14:27:32 21827 listening on all interfaces (IPv4) port 587 14:27:32 21827 changed uid/gid: running as a daemon 14:27:32 21827 uid=108 gid=111 pid=21827 14:27:32 21827 auxiliary group list: 111 14:27:32 21827 LOG: MAIN 14:27:32 21827 exim 4.94.2 daemon started: pid=21827, no queue runs, listening for SMTP on port 587 (IPv6 and IPv4) 14:27:32 21827 set_process_info: 21827 daemon(4.94.2): no
Re: [exim] 4.95 RC0 - gnutls outgoing TLS cert verification broken
On 18/07/2021 14:09, Andreas Metzler via Exim-users wrote: there seems to be some breakage in 4.95 RC0 with outgoing TLS, it fails to verify the peer certificate: Since we're happily running the test-suite this must be either an environment difference or a build problem. Did you give both client and server debug output there? It wasn't annotated... Part of it says "close on process exit" right after a TLS handshake starts up, and I don't see any reason given. Could you manage to get "all" debug output all the way from daemon startup? I'm hoping we'll see the PIDs, as well as the credentials cache load. And also "all" debug for the client? -- Cheers, Jeremy -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
[exim] 4.95 RC0 - gnutls outgoing TLS cert verification broken
Hello, there seems to be some breakage in 4.95 RC0 with outgoing TLS, it fails to verify the peer certificate: initialising GnuTLS as a client on fd 7 GnuTLS global init required initialising GnuTLS client session Expanding various TLS configuration options for session credentials TLS: basic cred init, client TLS: no client certificate specified; okay TLS: tls_verify_certificates not set or empty, ignoring GnuTLS using default session cipher/priority "NORMAL" Setting D-H prime minimum acceptable bits to 1024 31.15.64.248 in tls_verify_hosts? yes (matched "*") 31.15.64.248 in tls_verify_cert_hostnames? yes (matched "*") TLS: server cert verification includes hostname: "vsrv21575.customer.vlinux.de" TLS: server certificate verification required TLS: will request OCSP stapling 31.15.64.248 in tls_resumption_hosts? no (option unset) about to gnutls_handshake search_tidyup called SMTP>>(close on process exit) Exim pid=128174 (daemon-accept) terminating with rc=0 child 128174 ended: status=0x0 normal exit, 0 0 SMTP accept processes now running Listening... (TLS1.3)-(ECDHE-SECP256R1)-(ECDSA-SECP384R1-SHA384)-(AES-256-GCM) To get keying info for TLS1.3 is hard: Set environment variable SSLKEYLOGFILE to a filename relative to the spool directory, and make sure it is writable by the Exim runtime user. Add SSLKEYLOGFILE to keep_environment in the exim config. Start Exim as root. If using sudo, add SSLKEYLOGFILE to env_keep in /etc/sudoers (works for TLS1.2 also, and saves cut-paste into file). Trying to use add_environment for this will not work TLS: checking peer certificate The certificate is NOT trusted. The certificate issuer is unknown. TLS certificate verification failed (certificate invalid): peerdn="CN=vsrv21575.customer.vlinux.de" TLS session fail: (certificate verification failed): certificate invalid For reference with 4.94.2 (+fixes) successful debug output looks like this: initialising GnuTLS as a client on fd 7 GnuTLS global init required. initialising GnuTLS client session Expanding various TLS configuration options for session credentials. TLS: no client certificate specified; okay Added 127 certificate authorities. GnuTLS using default session cipher/priority "NORMAL" Setting D-H prime minimum acceptable bits to 1024 31.15.64.248 in tls_verify_hosts? yes (matched "*") 31.15.64.248 in tls_verify_cert_hostnames? yes (matched "*") TLS: server cert verification includes hostname: "vsrv21575.customer.vlinux.de". TLS: server certificate verification required. TLS: will request OCSP stapling about to gnutls_handshake (TLS1.3)-(ECDHE-SECP256R1)-(ECDSA-SECP384R1-SHA384)-(AES-256-GCM) To get keying info for TLS1.3 is hard: Set environment variable SSLKEYLOGFILE to a filename relative to the spool directory, and make sure it is writable by the Exim runtime user. Add SSLKEYLOGFILE to keep_environment in the exim config. Start Exim as root. If using sudo, add SSLKEYLOGFILE to env_keep in /etc/sudoers (works for TLS1.2 also, and saves cut-paste into file). Trying to use add_environment for this will not work TLS: checking peer certificate TLS certificate verified: peerdn="CN=vsrv21575.customer.vlinux.de" cipher: TLS1.3:ECDHE_SECP256R1__ECDSA_SECP384R1_SHA384__AES_256_GCM:256 Have channel bindings cached for possible auth usage cu Andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
