Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-05-14 Thread gregor herrmann
On Mon, 13 May 2019 01:02:45 +0200, Guilhem Moulin wrote: > Thanks for your analysis, Steffen. Dropping the Debian-specific patch > is definitely the way to go for libwww/LWP. Thanks for the confirmation, Guilhem, and all your work on this issue, and thanks alot to Steffen for tracking down the

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-05-14 Thread Guilhem Moulin
On Tue, 14 May 2019 at 03:57:46 +0200, Steffen Ullrich wrote: >> Ah I see, thanks for the clarification. I thought you meant it could >> yield a deadlock. Aren't temporary failures also possible on plain >> sockets (though of course the extra SSL layer make it strictly more >> likely to happen)?

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-05-13 Thread Steffen Ullrich
> Ah I see, thanks for the clarification. I thought you meant it could > yield a deadlock. Aren't temporary failures also possible on plain > sockets (though of course the extra SSL layer make it strictly more > likely to happen)? IIRC if the checksum of the incoming packet > mismatches, which

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-05-13 Thread Guilhem Moulin
On Mon, 13 May 2019 at 22:24:55 +0200, Steffen Ullrich wrote: > On Mon, May 13, 2019 at 03:18:14PM +0200, Guilhem Moulin > wrote: >> Uh, what? “Before” meaning with ≤TLSv1.2, or with OpenSSL <1.1.1a's >> default flags? libssl mentions no such thing beside the new default >> mode. And in fact

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-05-13 Thread Steffen Ullrich
On Mon, May 13, 2019 at 03:18:14PM +0200, Guilhem Moulin wrote: > On Mon, 13 May 2019 at 06:31:26 +0200, Steffen Ullrich wrote: > > Applications which relied on blocking I/O in connection with select could > > also hang before, > > Uh, what? “Before” meaning with ≤TLSv1.2, or with OpenSSL

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-05-13 Thread Guilhem Moulin
On Mon, 13 May 2019 at 06:31:26 +0200, Steffen Ullrich wrote: > Additionally switching off SSL_MODE_AUTO_RETRY would actually just add > a different unexpected behavior: that sysread might return with EAGAIN > on a blocking socket. FWIW as shown below that's always been the case, until OpenSSL

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-05-13 Thread Guilhem Moulin
On Mon, 13 May 2019 at 06:31:26 +0200, Steffen Ullrich wrote: > Applications which relied on blocking I/O in connection with select could > also hang before, Uh, what? “Before” meaning with ≤TLSv1.2, or with OpenSSL <1.1.1a's default flags? libssl mentions no such thing beside the new default

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-05-12 Thread Steffen Ullrich
On Mon, May 13, 2019 at 01:02:45AM +0200, Guilhem Moulin wrote: > Thanks for your analysis, Steffen. Dropping the Debian-specific patch > is definitely the way to go for libwww/LWP. However I still believe > IO::Socket::SSL should provide a way to clear SSL_MODE_AUTO_RETRY in > order to fix

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-05-12 Thread Guilhem Moulin
Thanks for your analysis, Steffen. Dropping the Debian-specific patch is definitely the way to go for libwww/LWP. However I still believe IO::Socket::SSL should provide a way to clear SSL_MODE_AUTO_RETRY in order to fix applications relying on the former OpenSSL defaults, as suggested in the

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-05-12 Thread gregor herrmann
Control: reassign -1 libwww-perl 6.36-1 Control: retitle -1 libwww-perl: ancient patch causes TLSv1.3 connection deadlock Control: tag -1 + patch On Sun, 12 May 2019 07:31:57 +0200, Steffen Ullrich wrote: > Actually, the fix is probably even simpler. > > The bug does not happen with an

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-05-11 Thread Steffen Ullrich
Actually, the fix is probably even simpler. The bug does not happen with an original libwww. It only happens with an LWP which contains Debian patches. Specifically the patch drop-non-blocking-socket.patch causes the problem since it keeps the socket blocking while the original libwww

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-05-11 Thread gregor herrmann
On Sat, 11 May 2019 21:56:01 +0200, Steffen Ullrich wrote: > I think the issue is a bit different than what was analyzed so far. […] Thank you for this detailed analysis! > The following small patch for LWP::protocol::http seems to fix the > problem for me: > > ---

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-05-11 Thread Steffen Ullrich
I think the issue is a bit different than what was analyzed so far. As far as I can see it has nothing to do with SSL_MODE_AUTO_RETRY but instead is caused by expectations on the behavior of select which are wrong with TLS 1.3. I've added some more debugging to IO::Socket::SSL and what I saw

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-05-08 Thread Dimitri John Ledkov
On Tue, 7 May 2019 19:39:15 +0200 Guilhem Moulin wrote: > Hi Dimitri, > > On Tue, 07 May 2019 at 15:46:25 +0100, Dimitri John Ledkov wrote: > > On Tue, 7 May 2019 14:16:43 +0100 Dimitri John Ledkov > > wrote: > >> This issue concerns me a lot at the moment. I am currently trying to > >> upgrade

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-05-07 Thread Guilhem Moulin
Hi Dimitri, On Tue, 07 May 2019 at 15:46:25 +0100, Dimitri John Ledkov wrote: > On Tue, 7 May 2019 14:16:43 +0100 Dimitri John Ledkov wrote: >> This issue concerns me a lot at the moment. I am currently trying to >> upgrade OpenSSL from 1.1.0 to 1.1.1 in Ubuntu 18.04 LTS (bionic). And >> as far

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-05-07 Thread Dimitri John Ledkov
On Tue, 7 May 2019 14:16:43 +0100 Dimitri John Ledkov wrote: > Hi, > > On Wed, 10 Apr 2019 15:22:09 +0200 Guilhem Moulin wrote: > > > > Not setting the SSL_MODE_AUTO_RETRY flag back after removing O_NONBLOCK > > (ie commenting out `Net::SSLeay::set_mode($ssl, $mode_auto_retry);` in > > the

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-05-07 Thread Dimitri John Ledkov
Hi, On Wed, 10 Apr 2019 15:22:09 +0200 Guilhem Moulin wrote: > > Not setting the SSL_MODE_AUTO_RETRY flag back after removing O_NONBLOCK > (ie commenting out `Net::SSLeay::set_mode($ssl, $mode_auto_retry);` in > the patch) solves the problem with blocking I/O and select/poll, but > breaks

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-04-10 Thread Guilhem Moulin
On Tue, 09 Apr 2019 at 23:39:31 +0200, Guilhem Moulin wrote: > AFAICT this worked this time because the socket was *only* marked as > ready for writing after the first select() call. Only during the second > call was there some data to be read: > >> select(8, [3], [3], NULL, {tv_sec=180,

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-04-09 Thread Guilhem Moulin
On Tue, 09 Apr 2019 at 17:26:22 +0200, gregor herrmann wrote: > On Tue, 09 Apr 2019 17:14:32 +0200, Guilhem Moulin wrote: >> With TLS 1.3? (You can pass ‘SSL_version => "TLSv1_3"’ to ssl_opts to >> force this.) Doesn't work here, still hangs on read(): > > Yes, also with using TLSv1_3

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-04-09 Thread gregor herrmann
On Tue, 09 Apr 2019 17:14:32 +0200, Guilhem Moulin wrote: > On Tue, 09 Apr 2019 at 16:59:20 +0200, gregor herrmann wrote: > > When I install the package with the patch and run our test case > > again, I don't get any hangs anymore: > > > > % time perl -MLWP::UserAgent -e > >

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-04-09 Thread Guilhem Moulin
On Tue, 09 Apr 2019 at 16:59:20 +0200, gregor herrmann wrote: > When I install the package with the patch and run our test case > again, I don't get any hangs anymore: > > % time perl -MLWP::UserAgent -e > 'LWP::UserAgent->new->post("https://facebook.com;, { data => "foo" }) or die' > perl

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-04-09 Thread gregor herrmann
On Tue, 09 Apr 2019 07:48:45 +0200, Steffen Ullrich wrote: > > You're welcome :-) Does clearing the SSL_MODE_AUTO_RETRY context flag > > (i.e., reverting the default from OpenSSL <1.1.1) solves this for you > > too? If so, what do you think about my proposed paths forwards from Not yet, I first

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-04-09 Thread Guilhem Moulin
On Tue, 09 Apr 2019 at 07:48:45 +0200, Steffen Ullrich wrote: > Simply clearing SSL_MODE_AUTO_RETRY will cause problems with blocking > connections in TLS 1.3. AFAICT not when SSL_read() is used as documented. Also while the issue is triggered more often for TLS 1.3 than for earlier TLS protocol

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-04-08 Thread Steffen Ullrich
> You're welcome :-) Does clearing the SSL_MODE_AUTO_RETRY context flag > (i.e., reverting the default from OpenSSL <1.1.1) solves this for you > too? If so, what do you think about my proposed paths forwards from Simply clearing SSL_MODE_AUTO_RETRY will cause problems with blocking connections

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-04-07 Thread Guilhem Moulin
On Sun, 07 Apr 2019 at 20:56:41 +0200, gregor herrmann wrote: > Alright, after purging libssl1.0.2 (and the outdated packages which > depended on it *cough*) I get the hang as well: > […] > Thanks for the push in the right direction! You're welcome :-) Does clearing the SSL_MODE_AUTO_RETRY

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-04-07 Thread gregor herrmann
On Sun, 07 Apr 2019 18:39:44 +0200, Guilhem Moulin wrote: > > I can't reproduce this problem: > Interesting, are you talking TLS 1.3? Good question :) > $ dpkg-query -l "libssl*" "libnet-ssleay-perl" "liblwp-protocol-https-perl" > "libio-socket-ssl-perl" >

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-04-07 Thread Guilhem Moulin
On Sun, 07 Apr 2019 at 18:12:45 +0200, gregor herrmann wrote: > On Sun, 18 Nov 2018 19:41:05 +0200, Niko Tyni wrote: > >> Reiterating a bit: the underlying issue with TLSv1.3 seems to be related >> to handling of 'non-application_data_records'. >> >> The client tries to POST but gets an 'SSL

Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

2019-04-07 Thread gregor herrmann
On Sun, 18 Nov 2018 19:41:05 +0200, Niko Tyni wrote: > Reiterating a bit: the underlying issue with TLSv1.3 seems to be related > to handling of 'non-application_data_records'. > > The client tries to POST but gets an 'SSL wants a read first' error, > then waits until timeout for the socket to