Re: HTTP/2 Termination vs. Firefox Quantum

On Sun, Dec 31, 2017 at 07:22:26AM +, Lucas Rolff wrote: > I've tested the 1.8.3 build, and I can indeed confirm it works like charm! Great, thank you for confirming. We're making progress :-) > @Willy, thanks for the extensive time you spend on debugging and > investigating this as well!

Re: HTTP/2 Termination vs. Firefox Quantum

I’ve tested the 1.8.3 build, and I can indeed confirm it works like charm! @Willy, thanks for the extensive time you spend on debugging and investigating this as well! Best Regards, Lucas Rolff

AW: HTTP/2 Termination vs. Firefox Quantum

@formilux.org Betreff: Re: HTTP/2 Termination vs. Firefox Quantum Just a quick update for all those following this thread. Thanks to Lucas' traces, we could finally fix the problem in the hpack decoder. I'm now releasing 1.8.3 which should be much more usable with H2. Willy

Re: HTTP/2 Termination vs. Firefox Quantum

Just a quick update for all those following this thread. Thanks to Lucas' traces, we could finally fix the problem in the hpack decoder. I'm now releasing 1.8.3 which should be much more usable with H2. Willy

Re: HTTP/2 Termination vs. Firefox Quantum

<< :authority: dashboard.domain.com user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:59.0) Gecko/20100101 Firefox/59.0 accept: text/css,*/*;q=0.1 accept-language: da,en-US;q=0.8,en;q=0.6,es;q=0.4,tr;q=0.2 accept-encoding: gzip, deflate, br referer:

Re: HTTP/2 Termination vs. Firefox Quantum

On Fri, Dec 29, 2017 at 06:56:36PM +, Lucas Rolff wrote: > h2_make_h1_request:153 > h2_frt_decode_headers:2621 > h2_frt_decode_headers:2643 > > /* this can be any type of header */ > /* RFC7540#8.1.2: upper case not allowed in header field names */ > for (i = 0; i < list[idx].n.len; i++) >

Re: HTTP/2 Termination vs. Firefox Quantum

On Fri, Dec 29, 2017 at 06:56:36PM +, Lucas Rolff wrote: > h2_make_h1_request:153 > h2_frt_decode_headers:2621 > h2_frt_decode_headers:2643 > > /* this can be any type of header */ > /* RFC7540#8.1.2: upper case not allowed in header field names */ > for (i = 0; i < list[idx].n.len; i++) >

Re: HTTP/2 Termination vs. Firefox Quantum

h2_make_h1_request:153 h2_frt_decode_headers:2621 h2_frt_decode_headers:2643 /* this can be any type of header */ /* RFC7540#8.1.2: upper case not allowed in header field names */ for (i = 0; i < list[idx].n.len; i++) if ((uint8_t)(list[idx].n.ptr[i] - 'A') < 'Z' - 'A')

Re: HTTP/2 Termination vs. Firefox Quantum

On Fri, Dec 29, 2017 at 06:18:00PM +, Lucas Rolff wrote: > I think you forgot to attach the patch Grrr common mistake, sorry. > I did try to add the continue patch for the "blacklisted" headers, and same > result (now in this case, it happened on the website request itself: Thanks, I'll

Re: HTTP/2 Termination vs. Firefox Quantum

I think you forgot to attach the patch I did try to add the continue patch for the “blacklisted” headers, and same result (now in this case, it happened on the website request itself: POST Request to site: h2s_frt_make_resp_data:3180 h2s_frt_make_resp_data:3067 GET Request to site:

Re: HTTP/2 Termination vs. Firefox Quantum

On Fri, Dec 29, 2017 at 06:02:15PM +, Lucas Rolff wrote: > POST Request (to website): > h2s_frt_make_resp_data:3180 > h2s_frt_make_resp_data:3067 > - > GET Request (to website): > h2s_frt_make_resp_data:3180 > h2s_frt_make_resp_data:3067 > - > Get Request (app.css) > h2_frt_decode_headers:2621

Re: HTTP/2 Termination vs. Firefox Quantum

Working page load (total of 4 requests), and we see 4x 3180|3067 POST Request (to website): h2s_frt_make_resp_data:3180 h2s_frt_make_resp_data:3067 - GET Request (to website): h2s_frt_make_resp_data:3180 h2s_frt_make_resp_data:3067 - GET Request (app.css) h2s_frt_make_resp_data:3180

Re: HTTP/2 Termination vs. Firefox Quantum

On Fri, Dec 29, 2017 at 04:48:13PM +, Lucas Rolff wrote: > > If you're willing to run another test, I can prepare a debugging patch > > which will try to report every single error path in the H2 and HPACK code > > so that we can try to understand where the code was upset > > I'd love to run

Re: HTTP/2 Termination vs. Firefox Quantum

> If you're willing to run another test, I can prepare a debugging patch which > will try to report every single error path in the H2 and HPACK code so that > we can try to understand where the code was upset I’d love to run another test or 10 – in the end, we’ll all benefit from it

Re: HTTP/2 Termination vs. Firefox Quantum

On Fri, Dec 29, 2017 at 04:10:12PM +, Lucas Rolff wrote: > > Lucas, Maximilian, can you check the situation with this patch? The POST > > issue should definitely be gone, please also verify the GET issue with this > > patch (as I was unable to reproduce it). > > Sadly didn't fix the GET

Re: HTTP/2 Termination vs. Firefox Quantum

Hi Willy, On Fri, Dec 29, 2017 at 3:58 PM, Willy Tarreau wrote: > On Fri, Dec 29, 2017 at 03:42:30PM +0100, Willy Tarreau wrote: >> OK I managed to reproduce it with nghttp using --expect-continue to >> force it to leave a pause before sending the data. And indeed there >> the data

Re: HTTP/2 Termination vs. Firefox Quantum

Both in Firefox and Chrome my POST requests in 1.8.2 with the supplied patch, seem to do the trick (did about 300 post requests in each browser with no fails). Best Regards, On 29/12/2017, 15.58, "Willy Tarreau" wrote: On Fri, Dec 29, 2017 at 03:42:30PM +0100, Willy Tarreau

Re: HTTP/2 Termination vs. Firefox Quantum

On Fri, Dec 29, 2017 at 03:42:30PM +0100, Willy Tarreau wrote: > OK I managed to reproduce it with nghttp using --expect-continue to > force it to leave a pause before sending the data. And indeed there > the data are immediately followed by a shutdown. Getting closer... So here's what I found :

Re: HTTP/2 Termination vs. Firefox Quantum

Hello, On Fri, Dec 29, 2017 at 3:05 PM, Willy Tarreau wrote: >> Haproxy calls shutdown() after the HTTP payload was transmitted, nginx >> in the default configuration or nc for that matter closes the >> connection (we see recvfrom = 0) and then we close(): > > I can't reproduce

Re: HTTP/2 Termination vs. Firefox Quantum

On Fri, Dec 29, 2017 at 02:52:39PM +0100, Lukas Tribus wrote: > > For me it happens only when I have "option httpclose" in the configuration, > > ie we end up in tunnel mode. I can't reproduce it with either keep-alive, > > http-server-close nor forceclose. At least abortonclose is now safe > >

Re: HTTP/2 Termination vs. Firefox Quantum

Hello, On Fri, Dec 29, 2017 at 2:31 PM, Willy Tarreau wrote: > On Fri, Dec 29, 2017 at 11:45:55AM +0100, Lukas Tribus wrote: >> The FIN behavior comes from a48c141f4 ("BUG/MAJOR: connection: refine >> the situations where we don't send shutw()"), which also hit 1.8.2, so >> that

Re: HTTP/2 Termination vs. Firefox Quantum

On Fri, Dec 29, 2017 at 11:45:55AM +0100, Lukas Tribus wrote: > The FIN behavior comes from a48c141f4 ("BUG/MAJOR: connection: refine > the situations where we don't send shutw()"), which also hit 1.8.2, so > that explains the change in behavior between 1.8.1 and 1.8.2. For me it happens only

Re: HTTP/2 Termination vs. Firefox Quantum

Hi Lukas, On Fri, Dec 29, 2017 at 11:45:55AM +0100, Lukas Tribus wrote: > On Fri, Dec 29, 2017 at 11:22 AM, Lukas Tribus wrote: > > It's that: > > - when sending the POST request to the backend server, haproxy sends a > > FIN before the server responds > > - nginx doesn't like

Re: HTTP/2 Termination vs. Firefox Quantum

> Lucas, can you check my previous mail and see if you can enable ignoring > client aborts in your backend, assuming you are using nginx? I can confirm that ignoring client aborts in my backend using fastcgi_ignore_client_abort “resolves” the issue regarding POST requests. Best Regards, Lucas

Re: HTTP/2 Termination vs. Firefox Quantum

Hello, On Fri, Dec 29, 2017 at 11:22 AM, Lukas Tribus wrote: > It's that: > - when sending the POST request to the backend server, haproxy sends a > FIN before the server responds > - nginx doesn't like that and closes the request (you will see nginx > error code 499 in nginx

Re: HTTP/2 Termination vs. Firefox Quantum

> Actually that's not the case and that may explain the situation. The machine > runs OpenSSL 1.0.1 so only NPN is used, ALPN isn't. I'll try with a static > build of openssl 1.0.2 to see if the ratio increases. That might very well be the case, I know for sure that Chrome dropped support NPN

Re: HTTP/2 Termination vs. Firefox Quantum

On Fri, Dec 29, 2017 at 11:22:56AM +0100, Lukas Tribus wrote: > Hello, > > > On Fri, Dec 29, 2017 at 8:13 AM, Willy Tarreau wrote: > > Yep. For what it's worth, it's been enabled for about one month on > > haproxy.org > > and till now we didn't get any bad report, which is pretty

Re: HTTP/2 Termination vs. Firefox Quantum

> Yep. For what it's worth, it's been enabled for about one month on > haproxy.org and till now we didn't get any bad report, which is pretty > encouraging. Can I ask where? The negotiated protocol I get on https://haproxy.org/ is http/1.1 in both Google Chrome and Firefox as an example. If I

Re: HTTP/2 Termination vs. Firefox Quantum

Hi Lucas, On Fri, Dec 29, 2017 at 06:06:49AM +, Lucas Rolff wrote: > As much as I agree about that specs should be followed, I realized that even > if there's people that want to follow the spec 100%, there will always be > implementations used in large scale that won't be following the spec

Re: HTTP/2 Termination vs. Firefox Quantum

Hi Willy, > In fact it's a race between the GOAWAY frame caused by the invalid request, > and the HEADERS frame being sent in response to the stream being closed > I agree that it's quite confusing, but we're talking about responses to > conditions that are explicitly forbidden in the spec, so

Re: HTTP/2 Termination vs. Firefox Quantum

Hi Lukas, On Thu, Dec 28, 2017 at 09:19:24PM +0100, Lukas Tribus wrote: > On Thu, Dec 28, 2017 at 12:29 PM, Lukas Tribus wrote: > > Hello, > > > > > >> But in this example, you're using HTTP/1.1, The "Connection" header is > >> perfectly valid for 1.1. It's HTTP/2 which forbids

Re: HTTP/2 Termination vs. Firefox Quantum

> the output of the http2 golang test and can you please both clarify which OS > you reproduce this on? If I visit http2 golang test, I also don’t see it, and I saw it in developer tools (Because dev tools shouldn’t put headers that isn’t requested/received) – however based on your findings,

Re: HTTP/2 Termination vs. Firefox Quantum

Hello, On Thu, Dec 28, 2017 at 12:29 PM, Lukas Tribus wrote: > Hello, > > >> But in this example, you're using HTTP/1.1, The "Connection" header is >> perfectly valid for 1.1. It's HTTP/2 which forbids it. There is no >> inconsistency here. > > For me a request like this: > $

Re: HTTP/2 Termination vs. Firefox Quantum

Hello, > But in this example, you're using HTTP/1.1, The "Connection" header is > perfectly valid for 1.1. It's HTTP/2 which forbids it. There is no > inconsistency here. For me a request like this: $ curl -kv --http2 https://localhost/111 -H "Connection: keep-alive" -d "bla=bla" Fired

Re: HTTP/2 Termination vs. Firefox Quantum

er 2017 11:27 An: Willy Tarreau <w...@1wt.eu> Cc: haproxy@formilux.org Betreff: Re: HTTP/2 Termination vs. Firefox Quantum > It's normal then, as it's mandated by the HTTP/2 spec to reject > requests containing any connection-specific header fields In th

AW: HTTP/2 Termination vs. Firefox Quantum

ly Tarreau <w...@1wt.eu> Cc: haproxy@formilux.org Betreff: Re: HTTP/2 Termination vs. Firefox Quantum > It's normal then, as it's mandated by the HTTP/2 spec to reject > requests containing any connection-specific header fields In that case, haproxy should be consistent in it’s way

Re: HTTP/2 Termination vs. Firefox Quantum

Sorry regarding my previous curl – I didn’t use --http2 in my curl request, but result is the same (with negotiated http2 protocol), I’ve removed the TLSv1.2 output since it’s useless in this case: === $ curl 'https://dashboard.domain.com/js/app.js?v=1' -H

Re: HTTP/2 Termination vs. Firefox Quantum

Hi Lucas, On Thu, Dec 28, 2017 at 08:38:52AM +, Lucas Rolff wrote: > It worked as it should, so I started adding more and more headers, until I > hit the culprit: -H "Connection: keep-alive" or -H "Connection: close" (or > even "Connection: test") (...) It's normal then, as it's mandated by

Re: HTTP/2 Termination vs. Firefox Quantum

I think I might have found a cause for this to happen, or at least a way to fully replicate the issue I saw that the issue also happens in Firefox whenever doing POST requests (Let’s say, logging into a website) – I got a bunch of “400 Bad Request” whenever trying to actually do the post

Re: HTTP/2 Termination vs. Firefox Quantum

, 2017 10:51:01 PM To: Lucas Rolff Cc: haproxy@formilux.org Subject: Re: HTTP/2 Termination vs. Firefox Quantum Hello Lucas, On Wed, Dec 27, 2017 at 9:24 PM, Lucas Rolff <lu...@lucasrolff.com> wrote: > Can't even compose an email correctly.. > > So: > > I experience the same issue

Re: HTTP/2 Termination vs. Firefox Quantum

Hello Lucas, On Wed, Dec 27, 2017 at 9:24 PM, Lucas Rolff wrote: > Can't even compose an email correctly.. > > So: > > I experience the same issue however with nginx as a backend. > > I tried enabling “option httplog” within my frontend, it's rather easy for > me to

Re: HTTP/2 Termination vs. Firefox Quantum

I tried enabling “option httplog” within my frontend, I do have the same issue wit

AW: HTTP/2 Termination vs. Firefox Quantum

lto:ber...@luffy.cx] Gesendet: Donnerstag, 21. Dezember 2017 11:04 An: Maximilian Böhm <maximilian.bo...@auctores.de> Cc: haproxy@formilux.org Betreff: Re: HTTP/2 Termination vs. Firefox Quantum ❦ 21 décembre 2017 09:00 GMT, Maximilian Böhm <maximilian.bo...@auctores.de> : > We are

Re: HTTP/2 Termination vs. Firefox Quantum

❦ 21 décembre 2017 09:00 GMT, Maximilian Böhm  : > We are using HA-Proxy version 1.8.1-1~bpo8+1 2017/12/04 on Debian 8. On the > backend, jetty 9.3.11.v20160721 with http/1.1 answers requests. > > Since I've enabled http/2 ("alpn h2,http/1.1"), we are facing issues

HTTP/2 Termination vs. Firefox Quantum

Hi there, thanks for releasing support for http/2! Sadly, we are facing issues since enabling it. We are using HA-Proxy version 1.8.1-1~bpo8+1 2017/12/04 on Debian 8. On the backend, jetty 9.3.11.v20160721 with http/1.1 answers requests. Since I've enabled http/2 ("alpn h2,http/1.1"), we are