Re: pkg_add fails with error "TLS connect failure: failed to set session"
On Sat, Feb 08, 2020 at 12:51:24PM +0100, Sven Wolf wrote: > Hi, > > the pkg_add problem with an https installurl is solved (tested with snapshot > #639/2020-02-07). Thanks for letting us know. This issue was first resolved by disabling the TLSv1.3 client last Saturday. A workaround was committed in revision 1.12 of ssl_methods.c: https://cvsweb.openbsd.org/src/lib/libssl/ssl_methods.c The client has since been re-enabled.
Re: pkg_add fails with error "TLS connect failure: failed to set session"
Hi, the pkg_add problem with an https installurl is solved (tested with snapshot #639/2020-02-07). Thanks and best regards, Sven On 1/31/20 9:24 PM, Sven Wolf wrote: Hi, I run current. After I run sysupgrade today (GENERIC.MP #626 build Jan 30) it's not possible to run pkd_add. I always get the error TLS connect failure: failed to set session signify: gzheader truncated The error is reproducible on two machines and didn't occur until build #616 (Jan 21). /etc/installurl points to an internal mirror server. This mirror server runs on Debian/Apache and has a letsencrypt certificate. Maybe the letsencrypt certificate is the root cause. When I switch /etc/installurl to an official OpenBSD mirror (e.g. https://artfiles.org/openbsd/) the error doesn't occur. Also when /etc/installurl points to the internal mirror server using the http instead of the https protocol then there is also no error. sysupgrade runs without errors against the internal mirror server via https. Also an wget of a package (e.g atk-2.34.1p1) via the https protocol shows no error. I compared the atk-2.34.1p1 package against an official mirror. There is no difference in the md5sum. Maybe the pkg_add error has something in common with https://marc.info/?t=15799643511&r=1&w=2 If there is something I should test/change, please let me know. Thanks and best regards, Sven
Re: pkg_add fails with error "TLS connect failure: failed to set session"
Hi Marc, here are the protocol details for my internal mirror. Protocol Details DROWN No, server keys and hostname not seen elsewhere with SSLv2 (1) For a better understanding of this test, please read this longer explanation (2) Key usage data kindly provided by the Censys network search engine; original DROWN website here (3) Censys data is only indicative of possible key and certificate reuse; possibly out-of-date and not complete Secure RenegotiationSupported Secure Client-Initiated Renegotiation No Insecure Client-Initiated Renegotiation No BEAST attackNot mitigated server-side (more info) TLS 1.0: 0x2f POODLE (SSLv3) No, SSL 3 not supported (more info) POODLE (TLS)No (more info) Zombie POODLE No (more info) TLS 1.2 : 0x002f GOLDENDOODLENo (more info) TLS 1.2 : 0x002f OpenSSL 0-LengthNo (more info) TLS 1.2 : 0x002f Sleeping POODLE No (more info) TLS 1.2 : 0x002f Downgrade attack prevention Yes, TLS_FALLBACK_SCSV supported (more info) SSL/TLS compression No RC4 No Heartbeat (extension) No Heartbleed (vulnerability) No (more info) Ticketbleed (vulnerability) No (more info) OpenSSL CCS vuln. (CVE-2014-0224) No (more info) OpenSSL Padding Oracle vuln. (CVE-2016-2107) No (more info) ROBOT (vulnerability) No (more info) Forward Secrecy With some browsers (more info) ALPNYes http/1.1 NPN No Session resumption (caching)Yes Session resumption (tickets)Yes OCSP stapling No Strict Transport Security (HSTS)Yes max-age=15768000 HSTS Preloading Not in: Chrome Edge Firefox IE Public Key Pinning (HPKP) No (more info) Public Key Pinning Report-Only No Public Key Pinning (Static) No (more info) Long handshake intolerance No TLS extension intolerance No TLS version intolerance No Incorrect SNI alertsNo Uses common DH primes No DH public server param (Ys) reuse No ECDH public server param reuse No Supported Named Groups secp256r1, secp384r1, secp521r1, x25519, x448 (Server has no preference) SSL 2 handshake compatibility Yes 0-RTT enabled No Here is the diff of the protocol details for my not working internal server and the artfiles openbsd mirror. 1d0 < Protocol Details 9c8 < BEAST attack Not mitigated server-side (more info) TLS 1.0: 0x2f --- > BEAST attack Not mitigated server-side (more info) TLS 1.0: 0xc013 12,15c11,14 < Zombie POODLE No (more info) TLS 1.2 : 0x002f < GOLDENDOODLE No (more info) TLS 1.2 : 0x002f < OpenSSL 0-Length No (more info) TLS 1.2 : 0x002f < Sleeping POODLENo (more info) TLS 1.2 : 0x002f --- > Zombie POODLE No (more info) TLS 1.2 : 0xc027 > GOLDENDOODLE No (more info) TLS 1.2 : 0xc027 > OpenSSL 0-Length No (more info) TLS 1.2 : 0xc027 > Sleeping POODLENo (more info) TLS 1.2 : 0xc027 26c25 < Forward SecrecyWith some browsers (more info) --- > Forward SecrecyYes (with most browsers) ROBUST (more info) 32,33c31 < Strict Transport Security (HSTS) Yes < max-age=15768000 --- > Strict Transport Security (HSTS) No 45c43 < Supported Named Groups secp256r1, secp384r1, secp521r1, x25519, x448 (Server has no preference) --- > Supported Named Groups x25519, secp256r1, x448, secp521r1, secp384r1 (server preferred order) 47d44 < 0-RTT enabled No Best regards, Sven On 2/1/20 1:36 PM, Marc Espie wrote: On Sat, Feb 01, 2020 at 12:48:40PM +0100, Sven Wolf wrote: Hi, I did some debugging on the server side. Even with loglevel trace5 and also different TLS versions (I tested 1.1, 1.2 and 1.3) I didn't find the root cause. In the attachment you'll find the export of the Apache error log with loglevel trace5. Maybe it's helpfull for the libressl developers. On the client side I just did an pkg_add -v bash Best regards, Sven If you can expose that server to the outside world, try https://www.ssllabs.com/ what does the report say, especially wrt session resumption ?
Re: pkg_add fails with error "TLS connect failure: failed to set session"
On 2020/02/01 13:36, Marc Espie wrote: > On Sat, Feb 01, 2020 at 12:48:40PM +0100, Sven Wolf wrote: > > Hi, > > > > I did some debugging on the server side. > > Even with loglevel trace5 and also different TLS versions (I tested 1.1, 1.2 > > and 1.3) I didn't find the root cause. > > > > In the attachment you'll find the export of the Apache error log with > > loglevel trace5. Maybe it's helpfull for the libressl developers. > > > > On the client side I just did an pkg_add -v bash > > > > Best regards, > > Sven > If you can expose that server to the outside world, try > > https://www.ssllabs.com/ > > what does the report say, especially wrt session resumption ? > jsing@ has committed a workaround to src/lib/libssl for now.
Re: pkg_add fails with error "TLS connect failure: failed to set session"
On Sat, Feb 01, 2020 at 12:48:40PM +0100, Sven Wolf wrote: > Hi, > > I did some debugging on the server side. > Even with loglevel trace5 and also different TLS versions (I tested 1.1, 1.2 > and 1.3) I didn't find the root cause. > > In the attachment you'll find the export of the Apache error log with > loglevel trace5. Maybe it's helpfull for the libressl developers. > > On the client side I just did an pkg_add -v bash > > Best regards, > Sven If you can expose that server to the outside world, try https://www.ssllabs.com/ what does the report say, especially wrt session resumption ?
Re: pkg_add fails with error "TLS connect failure: failed to set session"
Hi, I did some debugging on the server side. Even with loglevel trace5 and also different TLS versions (I tested 1.1, 1.2 and 1.3) I didn't find the root cause. In the attachment you'll find the export of the Apache error log with loglevel trace5. Maybe it's helpfull for the libressl developers. On the client side I just did an pkg_add -v bash Best regards, Sven On 1/31/20 9:24 PM, Sven Wolf wrote: Hi, I run current. After I run sysupgrade today (GENERIC.MP #626 build Jan 30) it's not possible to run pkd_add. I always get the error TLS connect failure: failed to set session signify: gzheader truncated The error is reproducible on two machines and didn't occur until build #616 (Jan 21). /etc/installurl points to an internal mirror server. This mirror server runs on Debian/Apache and has a letsencrypt certificate. Maybe the letsencrypt certificate is the root cause. When I switch /etc/installurl to an official OpenBSD mirror (e.g. https://artfiles.org/openbsd/) the error doesn't occur. Also when /etc/installurl points to the internal mirror server using the http instead of the https protocol then there is also no error. sysupgrade runs without errors against the internal mirror server via https. Also an wget of a package (e.g atk-2.34.1p1) via the https protocol shows no error. I compared the atk-2.34.1p1 package against an official mirror. There is no difference in the md5sum. Maybe the pkg_add error has something in common with https://marc.info/?t=15799643511&r=1&w=2 If there is something I should test/change, please let me know. Thanks and best regards, Sven [Sat Feb 01 12:36:57.605934 2020] [ssl:trace3] [pid 4837] ssl_engine_init.c(579): Creating new SSL context (protocols: TLSv1.1) [Sat Feb 01 12:36:57.606388 2020] [ssl:trace1] [pid 4837] ssl_engine_init.c(915): Configuring permitted SSL ciphers [HIGH:!aNULL:!aNULL:!eNULL:!EXP] [Sat Feb 01 12:36:57.606877 2020] [ssl:debug] [pid 4837] ssl_engine_init.c(): AH01904: Configuring server certificate chain (1 CA certificate) [Sat Feb 01 12:36:57.606894 2020] [ssl:debug] [pid 4837] ssl_engine_init.c(479): AH01893: Configuring TLS extension handling [Sat Feb 01 12:36:57.607477 2020] [ssl:trace3] [pid 4837] ssl_util_ssl.c(465): [mirror.somedomain.de:443] modssl_X509_match_name: expecting name 'mirror.somedomain.de', matched by ID 'mirror.somedomain.de' [Sat Feb 01 12:36:57.607569 2020] [ssl:debug] [pid 4837] ssl_util_ssl.c(476): AH02412: [mirror.somedomain.de:443] Cert matches for name 'mirror.somedomain.de' [subject: CN=mirror.somedomain.de / issuer: CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US / serial: 03D026E69E436EA8EF822C5DE8B359B40DE5 / notbefore: Jan 31 01:25:56 2020 GMT / notafter: Apr 30 01:25:56 2020 GMT] [Sat Feb 01 12:36:57.607587 2020] [ssl:info] [pid 4837] AH02568: Certificate and private key mirror.somedomain.de:443:0 configured from /etc/letsencrypt/live/mirror.somedomain.de/cert.pem and /etc/letsencrypt/live/mirror.somedomain.de/privkey.pem [Sat Feb 01 12:36:57.611517 2020] [mpm_prefork:notice] [pid 4837] AH00163: Apache/2.4.38 (Raspbian) OpenSSL/1.1.1d configured -- resuming normal operations [Sat Feb 01 12:36:57.611601 2020] [core:notice] [pid 4837] AH00094: Command line: '/usr/sbin/apache2' [Sat Feb 01 12:37:02.227125 2020] [ssl:trace4] [pid 4924] ssl_engine_io.c(2214): [client 192.168.1.103:34913] OpenSSL: write 3137/3137 bytes to BIO#9e95a8 [mem: af8250] [Sat Feb 01 12:37:02.234666 2020] [ssl:trace4] [pid 4924] ssl_engine_io.c(2214): [client 192.168.1.103:34913] OpenSSL: read 5/5 bytes from BIO#a65fc0 [mem: af23db] [Sat Feb 01 12:37:02.234828 2020] [ssl:trace4] [pid 4924] ssl_engine_io.c(2214): [client 192.168.1.103:34913] OpenSSL: read 69/69 bytes from BIO#a65fc0 [mem: af23e0] [Sat Feb 01 12:37:02.235645 2020] [socache_shmcb:debug] [pid 4924] mod_socache_shmcb.c(495): AH00831: socache_shmcb_store (0xe2 -> subcache 2) [Sat Feb 01 12:37:02.235809 2020] [socache_shmcb:debug] [pid 4924] mod_socache_shmcb.c(849): AH00847: insert happened at idx=0, data=(0:32) [Sat Feb 01 12:37:02.235865 2020] [socache_shmcb:debug] [pid 4924] mod_socache_shmcb.c(854): AH00848: finished insert, subcache: idx_pos/idx_used=0/1, data_pos/data_used=0/214 [Sat Feb 01 12:37:02.235915 2020] [socache_shmcb:debug] [pid 4924] mod_socache_shmcb.c(516): AH00834: leaving socache_shmcb_store successfully [Sat Feb 01 12:37:02.236010 2020] [ssl:trace2] [pid 4924] ssl_engine_kernel.c(2042): Inter-Process Session Cache: request=SET status=OK id=e259e8e5b9da7257ddb9c78cc0f212faa03132eebedb0b9f8b4afe5b2d1a36ac timeout=300s (session caching) [Sat Feb 01 12:37:02.236172 2020] [ssl:trace4] [pid 4924] ssl_engine_io.c(2214): [client 192.168.1.103:34913] OpenSSL: write 303/303 bytes to BIO#9e95a8 [mem: af8250] [Sat Feb 01 12:37:02.236841 2020] [socache_shmcb:debug] [pid 4924] mod_socache_shmcb.c(495): AH00831: socache_shmcb_store (0x5b -> subcache 27) [Sat Feb 01 12:37:02.236992 2020] [socache_shmcb:debug] [pid 4924] mod_socache_shmcb
Re: pkg_add fails with error "TLS connect failure: failed to set session"
On Fri, Jan 31, 2020 at 09:24:08PM +0100, Sven Wolf wrote: > Hi, > > I run current. After I run sysupgrade today (GENERIC.MP #626 build Jan 30) > it's not possible to run pkd_add. I always get the error > TLS connect failure: failed to set session > signify: gzheader truncated > > The error is reproducible on two machines and didn't occur until build #616 > (Jan 21). > > /etc/installurl points to an internal mirror server. This mirror server runs > on Debian/Apache and has a letsencrypt certificate. Maybe the letsencrypt > certificate is the root cause. > When I switch /etc/installurl to an official OpenBSD mirror (e.g. > https://artfiles.org/openbsd/) the error doesn't occur. > Also when /etc/installurl points to the internal mirror server using the > http instead of the https protocol then there is also no error. pkg_add(1) does not deal directly with network connections. Any TLS bug is related to ftp(1). If you sign your packages, https is not event needed.
Re: pkg_add fails with error "TLS connect failure: failed to set session"
On 2020/01/31 23:03, Stuart Henderson wrote: > On 2020/01/31 21:24, Sven Wolf wrote: > > Hi, > > > > I run current. After I run sysupgrade today (GENERIC.MP #626 build Jan 30) > > it's not possible to run pkd_add. I always get the error > > TLS connect failure: failed to set session > > signify: gzheader truncated > > pkg_add runs ftp many times and tries to resume TLS sessions between calls > to reduce setup overhead. The failure is connected with this but is only > seen with some sites. > Generally it is hard to debug these without access to the server (at > least to make an HTTPS connection if not actually fetch files) so it being > an internal server makes that hard. However I have found some other hosts > which also have the same symptom so hopefully this will help libressl > developers track it down. > > https://cloudflare.cdn.openbsd.org/pub/OpenBSD/ > https://mirrors.ucr.ac.cr/pub/OpenBSD/ > https://mirrors.dotsrc.org/pub/OpenBSD/ > https://mirror.one.com/pub/OpenBSD/ > https://openbsd.c3sl.ufpr.br/pub/OpenBSD/ These all run TLS 1.3. The ones I ran through ssllabs checker all support session resumption as well (I didn't do all of them because it's takes forever :). > And there's a bonus "SSL_internal:unknown failure occurred" at > > https://mirror.vdms.com/pub/OpenBSD/ >
Re: pkg_add fails with error "TLS connect failure: failed to set session"
On 2020/01/31 21:24, Sven Wolf wrote: > Hi, > > I run current. After I run sysupgrade today (GENERIC.MP #626 build Jan 30) > it's not possible to run pkd_add. I always get the error > TLS connect failure: failed to set session > signify: gzheader truncated pkg_add runs ftp many times and tries to resume TLS sessions between calls to reduce setup overhead. The failure is connected with this but is only seen with some sites. > The error is reproducible on two machines and didn't occur until build #616 > (Jan 21). > > /etc/installurl points to an internal mirror server. This mirror server runs > on Debian/Apache and has a letsencrypt certificate. Maybe the letsencrypt > certificate is the root cause. > When I switch /etc/installurl to an official OpenBSD mirror (e.g. > https://artfiles.org/openbsd/) the error doesn't occur. There is no general problem with letsencrypt certificates, probably most of the official mirrors use them (artfiles.org certainly does). > Also when /etc/installurl points to the internal mirror server using the > http instead of the https protocol then there is also no error. > > sysupgrade runs without errors against the internal mirror server via https. > Also an wget of a package (e.g atk-2.34.1p1) via the https protocol shows no > error. > > I compared the atk-2.34.1p1 package against an official mirror. There is no > difference in the md5sum. > > Maybe the pkg_add error has something in common with > https://marc.info/?t=15799643511&r=1&w=2 > > If there is something I should test/change, please let me know. > > Thanks and best regards, > Sven > Generally it is hard to debug these without access to the server (at least to make an HTTPS connection if not actually fetch files) so it being an internal server makes that hard. However I have found some other hosts which also have the same symptom so hopefully this will help libressl developers track it down. https://cloudflare.cdn.openbsd.org/pub/OpenBSD/ https://mirrors.ucr.ac.cr/pub/OpenBSD/ https://mirrors.dotsrc.org/pub/OpenBSD/ https://mirror.one.com/pub/OpenBSD/ https://openbsd.c3sl.ufpr.br/pub/OpenBSD/ And there's a bonus "SSL_internal:unknown failure occurred" at https://mirror.vdms.com/pub/OpenBSD/
