Bug#1061660: liblwp-protocol-https-perl: Fail to verify certificates
On Sun, 04 Feb 2024 17:18:18 +0100, Christian Marillat wrote: > On 04 févr. 2024 17:07, gregor herrmann wrote: > > And before I could file the bug upstream, I noticed that there is > > already a new pull request for this issue: > > https://github.com/libwww-perl/LWP-Protocol-https/pull/77 > This also solves this problem for me. Thanks for testing & confirming! Cheers, gregor -- .''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06 `. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe `- signature.asc Description: Digital Signature
Bug#1061660: liblwp-protocol-https-perl: Fail to verify certificates
On 04 févr. 2024 17:07, gregor herrmann wrote: [...] > And before I could file the bug upstream, I noticed that there is > already a new pull request for this issue: > https://github.com/libwww-perl/LWP-Protocol-https/pull/77 This also solves this problem for me. [...] > I'm preparing an upload with this new fix. Thanks. Christian
Bug#1061660: liblwp-protocol-https-perl: Fail to verify certificates
Control: tag -1 + confirmed upstream patch Control: forwarded -1 https://github.com/libwww-perl/LWP-Protocol-https/pull/77 On Sat, 03 Feb 2024 08:40:41 +0100, Christian Marillat wrote: > This bug should be fixed. If Ipv6 isn't available, Ipv4 should be used. > Before 6.12 this package was working perfectly. > Th best is to forward this bug to upstream author. Right. And before doing this I wanted to understand what's actually going on, and finally I could reproduce it. You gave me the keyword earlier: The problem is in the SNI part of the change, and appears when a _proxy_ is used. So after installing squid in the local network I get: % https_proxy=http://new:3128 HEAD https://metacpan.org/release/LWP-Protocol-https 500 SSL upgrade failed: hostname verification failed Content-Type: text/plain Client-Date: Sun, 04 Feb 2024 15:50:11 GMT Client-Warning: Internal response (And the $host variable in line 85 is undef.) % env -u https_proxy HEAD https://metacpan.org/release/LWP-Protocol-https 200 OK Cache-Control: max-age=3600 Connection: close Date: Sun, 04 Feb 2024 15:50:37 GMT Via: 1.1 varnish, 1.1 varnish Accept-Ranges: bytes Age: 0 Server: nginx Vary: Accept-Encoding Content-Length: 49785 Content-Type: text/html; charset=utf-8 Last-Modified: Mon, 22 Jan 2024 17:51:48 GMT Client-Date: Sun, 04 Feb 2024 15:50:37 GMT Client-Peer: 151.101.194.217:443 Client-Response-Num: 1 Client-SSL-Cert-Issuer: /C=BE/O=GlobalSign nv-sa/CN=GlobalSign Atlas R3 DV TLS CA 2023 Q2 Client-SSL-Cert-Subject: /CN=metacpan.org Client-SSL-Cipher: ECDHE-RSA-CHACHA20-POLY1305 Client-SSL-Socket-Class: IO::Socket::SSL Client-SSL-Version: TLSv1_2 Content-Security-Policy: default-src * data: 'unsafe-inline'; frame-ancestors 'self' *.metacpan.org; script-src 'self' 'unsafe-eval' 'unsafe-inline' *.metacpan.org *.google-analytics.com *.google.com www.gstatic.com Strict-Transport-Security: max-age=31557600 X-Cache: MISS, MISS X-Cache-Hits: 0, 0 X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN X-Runtime: 3.174736 X-Served-By: cache-lhr7344-LHR, cache-vie6362-VIE X-Timer: S1707061835.628790,VS0,VE3218 X-XSS-Protection: 1; mode=block And before I could file the bug upstream, I noticed that there is already a new pull request for this issue: https://github.com/libwww-perl/LWP-Protocol-https/pull/77 And at least for me, the little change from https://patch-diff.githubusercontent.com/raw/libwww-perl/LWP-Protocol-https/pull/77.diff works: % https_proxy=http://new:3128 HEAD https://metacpan.org/release/LWP-Protocol-https 200 OK Cache-Control: max-age=3600 Connection: close Date: Sun, 04 Feb 2024 15:54:18 GMT Via: 1.1 varnish, 1.1 varnish Accept-Ranges: bytes Age: 221 Server: nginx Vary: Accept-Encoding Content-Length: 49785 Content-Type: text/html; charset=utf-8 Last-Modified: Mon, 22 Jan 2024 17:51:48 GMT Client-Date: Sun, 04 Feb 2024 15:54:18 GMT Client-Peer: 192.168.0.247:3128 Client-Response-Num: 1 Content-Security-Policy: default-src * data: 'unsafe-inline'; frame-ancestors 'self' *.metacpan.org; script-src 'self' 'unsafe-eval' 'unsafe-inline' *.metacpan.org *.google-analytics.com *.google.com www.gstatic.com Strict-Transport-Security: max-age=31557600 X-Cache: MISS, HIT X-Cache-Hits: 0, 1 X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN X-Runtime: 3.174736 X-Served-By: cache-lhr7344-LHR, cache-vie6320-VIE X-Timer: S1707062059.838034,VS0,VE2 X-XSS-Protection: 1; mode=block I'm preparing an upload with this new fix. Cheers, gregor -- .''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06 `. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe `- signature.asc Description: Digital Signature
Bug#1061660: liblwp-protocol-https-perl: Fail to verify certificates
On 02 févr. 2024 18:37, gregor herrmann wrote:
> On Tue, 30 Jan 2024 18:18:59 +0100, Christian Marillat wrote:
>
>> > @@ -96,9 +96,12 @@
>> > if ( $Net::HTTPS::SSL_SOCKET_CLASS->can('start_SSL')) {
>> > *_upgrade_sock = sub {
>> >my ($self,$sock,$url) = @_;
>> > +# SNI should be passed there only if it is not an IP address.
>> > +# Details:
>> > https://github.com/libwww-perl/libwww-perl/issues/449#issuecomment-1896175509
>>
>> I had the idea to read this github issue.
>
> Thanks for your further investigations!
>
>> In my case I've a proxy and IPv6 isn't configured so this explain this
>> Debian bug and reverting upstream changes in 6.12 is maybe a bad idea.
>
> Ok; so where does this leave us? Do I understand you correctly that
> we should not revert the above change, and that the issue is with
> your local setup? So should we just close the bug or is there
> anything left?
This bug should be fixed. If Ipv6 isn't available, Ipv4 should be used.
Before 6.12 this package was working perfectly.
Th best is to forward this bug to upstream author.
Christian
Bug#1061660: liblwp-protocol-https-perl: Fail to verify certificates
On Tue, 30 Jan 2024 18:18:59 +0100, Christian Marillat wrote:
> > @@ -96,9 +96,12 @@
> > if ( $Net::HTTPS::SSL_SOCKET_CLASS->can('start_SSL')) {
> > *_upgrade_sock = sub {
> > my ($self,$sock,$url) = @_;
> > +# SNI should be passed there only if it is not an IP address.
> > +# Details:
> > https://github.com/libwww-perl/libwww-perl/issues/449#issuecomment-1896175509
>
> I had the idea to read this github issue.
Thanks for your further investigations!
> In my case I've a proxy and IPv6 isn't configured so this explain this
> Debian bug and reverting upstream changes in 6.12 is maybe a bad idea.
Ok; so where does this leave us? Do I understand you correctly that
we should not revert the above change, and that the issue is with
your local setup? So should we just close the bug or is there
anything left?
Cheers,
gregor
--
.''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org
: :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06
`. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
`-
signature.asc
Description: Digital Signature
Bug#1061660: liblwp-protocol-https-perl: Fail to verify certificates
On 28 janv. 2024 19:03, gregor herrmann wrote:
Hi again,
[...]
> @@ -96,9 +96,12 @@
> if ( $Net::HTTPS::SSL_SOCKET_CLASS->can('start_SSL')) {
> *_upgrade_sock = sub {
> my ($self,$sock,$url) = @_;
> +# SNI should be passed there only if it is not an IP address.
> +# Details:
> https://github.com/libwww-perl/libwww-perl/issues/449#issuecomment-1896175509
I had the idea to read this github issue.
In my case I've a proxy and IPv6 isn't configured so this explain this
Debian bug and reverting upstream changes in 6.12 is maybe a bad idea.
Christian
Bug#1061660: liblwp-protocol-https-perl: Fail to verify certificates
On 29 janv. 2024 20:58, gregor herrmann wrote:
> On Mon, 29 Jan 2024 08:53:45 +0100, Christian Marillat wrote:
[...]
>> This diff fix this issue.
>
> Thanks for checking.
> Alright, so we know that
> 1) something is different between your and my environment, and
> 2) one of the two small changes between 6.11 and 6.12 causes errors
>for you
>
> Could you try which of the two hunks is the culprit? My very random
> guess is that it's the first one [0]; for some reason my laptop
> prefers IPv4 although I also have IPv6 …
My DMZ doesn't have IPv6.
The second hunk fix this issue for me.
,
| @@ -96,9 +96,12 @@
| if ( $Net::HTTPS::SSL_SOCKET_CLASS->can('start_SSL')) {
| *_upgrade_sock = sub {
| my ($self,$sock,$url) = @_;
| +# SNI should be passed there only if it is not an IP address.
| +# Details:
https://github.com/libwww-perl/libwww-perl/issues/449#issuecomment-1896175509
| + my $host = $url->host_port() =~ m/:|^[\d.]+$/s ? undef : $url->host();
| $sock = LWP::Protocol::https::Socket->start_SSL( $sock,
| SSL_verifycn_name => $url->host,
| - SSL_hostname => $url->host,
| + SSL_hostname => $host,
| $self->_extra_sock_opts,
| );
| $@ = LWP::Protocol::https::Socket->errstr if ! $sock;
`
Christian
Bug#1061660: liblwp-protocol-https-perl: Fail to verify certificates
On Mon, 29 Jan 2024 08:53:45 +0100, Christian Marillat wrote:
> >> This issue doesn't exist with 6.11-1. I've seen this bug when 6.12 has
> >> been installed.
> > What doesn't help is that I've uploaded libio-socket-ssl-perl 2.085-1
> > as well … (Although with minimal changes as well.)
> Could you try to reprocduce this bug with the svt-av1 package ?
Sure. Running `uscan --report' in the version from dmo gives no
output at all, in the version currently in testing and unstable tells
me about the new upstream version:
% uscan --report
Newest version of svt-av1 on remote site is 1.8.0, local version is 1.7.0
(mangled local version is 1.7.0)
=> Newer package available from:
=>
https://gitlab.com/AOMediaCodec/SVT-AV1/-/archive/v1.8.0/SVT-AV1-v1.8.0.tar.bz2
In both cases no errors or whatever.
> >> The attached patch isn't clean :
> > Sorry, that was a diff of the upstream git tags.
> This diff fix this issue.
Thanks for checking.
Alright, so we know that
1) something is different between your and my environment, and
2) one of the two small changes between 6.11 and 6.12 causes errors
for you
Could you try which of the two hunks is the culprit? My very random
guess is that it's the first one [0]; for some reason my laptop
prefers IPv4 although I also have IPv6 …
Cheers,
gregor
[0]
@@ -56,7 +56,7 @@
}
}
$self->{ssl_opts} = \%ssl_opts;
-return (%ssl_opts, $self->SUPER::_extra_sock_opts);
+return (%ssl_opts, MultiHomed => 1, $self->SUPER::_extra_sock_opts);
}
# This is a subclass of LWP::Protocol::http.
--
.''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org
: :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06
`. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
`-
signature.asc
Description: Digital Signature
Bug#1061660: liblwp-protocol-https-perl: Fail to verify certificates
On 28 janv. 2024 19:03, gregor herrmann wrote: > On Sun, 28 Jan 2024 18:44:02 +0100, Christian Marillat wrote: > >> > Does it work for you if you downgrade liblwp-protocol-https-perl to 6.11-1 >> > from testing? If yes, which of the two hunks from [0] is causing the >> > problem? >> This issue doesn't exist with 6.11-1. I've seen this bug when 6.12 has >> been installed. > > What doesn't help is that I've uploaded libio-socket-ssl-perl 2.085-1 > as well … (Although with minimal changes as well.) Could you try to reprocduce this bug with the svt-av1 package ? >> The attached patch isn't clean : > > Sorry, that was a diff of the upstream git tags. This diff fix this issue. Christian
Bug#1061660: liblwp-protocol-https-perl: Fail to verify certificates
On Sun, 28 Jan 2024 18:44:02 +0100, Christian Marillat wrote:
> > Does it work for you if you downgrade liblwp-protocol-https-perl to 6.11-1
> > from testing? If yes, which of the two hunks from [0] is causing the
> > problem?
> This issue doesn't exist with 6.11-1. I've seen this bug when 6.12 has
> been installed.
What doesn't help is that I've uploaded libio-socket-ssl-perl 2.085-1
as well … (Although with minimal changes as well.)
> The attached patch isn't clean :
Sorry, that was a diff of the upstream git tags.
From the debdiff:
#v+
diff -Nru liblwp-protocol-https-perl-6.11/lib/LWP/Protocol/https.pm
liblwp-protocol-https-perl-6.12/lib/LWP/Protocol/https.pm
--- liblwp-protocol-https-perl-6.11/lib/LWP/Protocol/https.pm 2023-07-09
17:10:32.0 +0200
+++ liblwp-protocol-https-perl-6.12/lib/LWP/Protocol/https.pm 2024-01-22
18:51:33.0 +0100
@@ -56,7 +56,7 @@
}
}
$self->{ssl_opts} = \%ssl_opts;
-return (%ssl_opts, $self->SUPER::_extra_sock_opts);
+return (%ssl_opts, MultiHomed => 1, $self->SUPER::_extra_sock_opts);
}
# This is a subclass of LWP::Protocol::http.
@@ -96,9 +96,12 @@
if ( $Net::HTTPS::SSL_SOCKET_CLASS->can('start_SSL')) {
*_upgrade_sock = sub {
my ($self,$sock,$url) = @_;
+# SNI should be passed there only if it is not an IP address.
+# Details:
https://github.com/libwww-perl/libwww-perl/issues/449#issuecomment-1896175509
+ my $host = $url->host_port() =~ m/:|^[\d.]+$/s ? undef : $url->host();
$sock = LWP::Protocol::https::Socket->start_SSL( $sock,
SSL_verifycn_name => $url->host,
- SSL_hostname => $url->host,
+ SSL_hostname => $host,
$self->_extra_sock_opts,
);
$@ = LWP::Protocol::https::Socket->errstr if ! $sock;
#v-
> > Do the errors from qa.debian.org go away if you run uscan as
> > "PERL_LWP_SSL_VERIFY_HOSTNAME=1 uscan …"?
> Still the same. I also this bug with gitlab.
Hm …
Cheers,
gregor
--
.''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org
: :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06
`. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
`-
signature.asc
Description: Digital Signature
Bug#1061660: liblwp-protocol-https-perl: Fail to verify certificates
On 28 janv. 2024 18:17, gregor herrmann wrote: [...] > Does it work for you if you downgrade liblwp-protocol-https-perl to 6.11-1 > from testing? If yes, which of the two hunks from [0] is causing the > problem? This issue doesn't exist with 6.11-1. I've seen this bug when 6.12 has been installed. The attached patch isn't clean : , | $ cat ~/https.diff |sudo patch -p1 -R --dry-run | checking file usr/share/perl5/LWP/Protocol/https.pm | Hunk #1 succeeded at 40 with fuzz 2 (offset -16 lines). | Hunk #2 FAILED at 96. | 1 out of 2 hunks FAILED ` > Do the errors from qa.debian.org go away if you run uscan as > "PERL_LWP_SSL_VERIFY_HOSTNAME=1 uscan …"? Still the same. I also this bug with gitlab. Christian
Bug#1061660: liblwp-protocol-https-perl: Fail to verify certificates
Control: tag -1 + unreproducible
On Sun, 28 Jan 2024 09:07:00 +0100, Christian Marillat wrote:
> uscan from devscipts package fail to verify certificates afetr upgrading
> to liblwp-protocol-https-perl 6.12-1
Thanks for your bug report.
> ,
> | uscan warn: In watchfile debian/watch, reading webpage
> | https://qa.debian.org/watch/sf.php/pcre/ failed: 500 SSL upgrade failed:
> hostname verification failed
> | uscan warn: In watchfile debian/watch, reading webpage
> | https://qa.debian.org/watch/sf.php/mjpeg/ failed: 500 SSL upgrade failed:
> hostname verification failed
> | uscan warn: In watchfile debian/watch, reading webpage
> | https://gitlab.com/AOMediaCodec/SVT-AV1/-/tags failed: 500 SSL upgrade
> failed: SSL connect attempt failed error:0A000410:SSL routines::sslv3 alert
> handshake failure
> | uscan warn: In watchfile debian/watch, reading webpage
> | https://qa.debian.org/watch/sf.php/synfig/ failed: 500 SSL upgrade
> failed: hostname verification failed
> `
I was a bit skeptical that these issues come from
liblwp-protocol-https-perl, as the changes between 6.11 and 6.12 are
small[0], and the errors sound like different issues:
- "hostname verification failed" might be the change in HTTP::Tiny …
or no, as that validates SSL certs; the error "hostname verification
failed" comes from libio-socket-ssl-perl
- "routines::sslv3 alert handshake failure" sounds like an openssl
configuration thing
Interestingly I can't reproduce the issue which makes diving into the
problem a bit hard:
% cat qa-sf-watch
version=4
https://qa.debian.org/watch/sf.php/pcre/ .*@ANY_VERSION@@ARCHIVE_EXT@
% cat gitlab-watch
version=4
https://gitlab.com/AOMediaCodec/SVT-AV1/-/tags .*@ANY_VERSION@@ARCHIVE_EXT@
% for w in qa-sf-watch gitlab-watch; do uscan --report --watchfile $w --package
abc --upstream-version 123; done
%
Does it work for you if you downgrade liblwp-protocol-https-perl to 6.11-1
from testing? If yes, which of the two hunks from [0] is causing the
problem?
Do the errors from qa.debian.org go away if you run uscan as
"PERL_LWP_SSL_VERIFY_HOSTNAME=1 uscan …"?
Does anyone else reading along have any ideas?
Cheers,
gregor
[0]
diff --git a/lib/LWP/Protocol/https.pm b/lib/LWP/Protocol/https.pm
index 16fce19..01a800b 100644
--- a/lib/LWP/Protocol/https.pm
+++ b/lib/LWP/Protocol/https.pm
@@ -56,7 +56,7 @@ EOT
}
}
$self->{ssl_opts} = \%ssl_opts;
-return (%ssl_opts, $self->SUPER::_extra_sock_opts);
+return (%ssl_opts, MultiHomed => 1, $self->SUPER::_extra_sock_opts);
}
# This is a subclass of LWP::Protocol::http.
@@ -96,9 +96,12 @@ sub _get_sock_info
if ( $Net::HTTPS::SSL_SOCKET_CLASS->can('start_SSL')) {
*_upgrade_sock = sub {
my ($self,$sock,$url) = @_;
+# SNI should be passed there only if it is not an IP address.
+# Details:
https://github.com/libwww-perl/libwww-perl/issues/449#issuecomment-1896175509
+ my $host = $url->host_port() =~ m/:|^[\d.]+$/s ? undef : $url->host();
$sock = LWP::Protocol::https::Socket->start_SSL( $sock,
SSL_verifycn_name => $url->host,
- SSL_hostname => $url->host,
+ SSL_hostname => $host,
$self->_extra_sock_opts,
);
$@ = LWP::Protocol::https::Socket->errstr if ! $sock;
--
.''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org
: :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06
`. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
`-
signature.asc
Description: Digital Signature
Bug#1061660: liblwp-protocol-https-perl: Fail to verify certificates
Package: liblwp-protocol-https-perl Version: 6.12-1 Severity: serious Dear Maintainer, uscan from devscipts package fail to verify certificates afetr upgrading to liblwp-protocol-https-perl 6.12-1 , | uscan warn: In watchfile debian/watch, reading webpage | https://qa.debian.org/watch/sf.php/pcre/ failed: 500 SSL upgrade failed: hostname verification failed | uscan warn: In watchfile debian/watch, reading webpage | https://qa.debian.org/watch/sf.php/mjpeg/ failed: 500 SSL upgrade failed: hostname verification failed | uscan warn: In watchfile debian/watch, reading webpage | https://gitlab.com/AOMediaCodec/SVT-AV1/-/tags failed: 500 SSL upgrade failed: SSL connect attempt failed error:0A000410:SSL routines::sslv3 alert handshake failure | uscan warn: In watchfile debian/watch, reading webpage | https://qa.debian.org/watch/sf.php/synfig/ failed: 500 SSL upgrade failed: hostname verification failed ` Christian -- System Information: Debian Release: trixie/sid APT prefers buildd-unstable APT policy: (500, 'buildd-unstable'), (500, 'unstable') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 6.7.2-1-custom (SMP w/24 CPU threads; PREEMPT) Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) Versions of packages liblwp-protocol-https-perl depends on: ii ca-certificates20230311 ii libio-socket-ssl-perl 2.085-1 ii libnet-http-perl 6.23-1 ii libwww-perl6.76-1 ii perl 5.38.2-3 liblwp-protocol-https-perl recommends no packages. Versions of packages liblwp-protocol-https-perl suggests: ii libcrypt-ssleay-perl 0.73.06-2+b2 -- no debconf information
