Hello Jarno, thanks for your suggestions. It was not successful.
However, I managed to make it reproductible. I would be really happy, if
someone more experienced would take a look on this.
Setup
Client (Chrome) -> Haproxy (Docker) -> Jetty (Docker)
The client executes following script, it can be saved on the local disk, we can
ignore the CORS logging. Do not change the '30000', otherwise it will not occur.
<script type="text/javascript">
function loopMe () {
setTimeout(function () {
var xhr = new XMLHttpRequest();
xhr.open('POST', 'https://[DOMAIN]/app/docker-jetty.json');
xhr.send();
loopMe();
}, 30000)
}
loopMe();
</script>
Haproxy.cfg
global
daemon
tune.ssl.default-dh-param 2048
stats socket /var/run/haproxy.stat
defaults
mode http
option httplog
log stdout format raw daemon
timeout connect 5m
timeout client 5m
timeout server 5m
frontend frontend_h2
bind *:443 ssl crt /usr/local/etc/haproxy/ssl/ alpn h2,http/1.1
use_backend backend_jetty
backend backend_jetty
server web850 127.0.0.1:81
Commands for starting the container
1) docker run -p 443:443 -v [LOCAL_DIR_HAPROXY]:/usr/local/etc/haproxy/ -it
haproxy:1.9.4
2) docker run -d --name jetty -v [LOCAL_DIR_JETTY]:/var/lib/jetty/webapps -p
81:8080 jetty
Substitute the following variables:
1) [DOMAIN]: Domain you have a certificate for or generate one
2) [LOCAL_DIR_HAPROXY]: Local directory where you need to put the "haproxy.cfg"
and your certificate (subdirectory "ssl")
3) [LOCAL_DIR_JETTY]: Local directory, create a subdirectory called "app" and
create an empty file named "docker-jetty.json").
Substitute the variables, start the container and open the script in the
browser. After 10-15 requests you should get a SC 400....
At first sight, it looks like jetty is doing something terribly wrong. But, and
that's the problem, it does not occur if I have just http/1.1 enabled between
the client and haproxy. Any ideas?
Thanks,
Max
-----Ursprüngliche Nachricht-----
Von: Jarno Huuskonen <[email protected]>
Gesendet: Mittwoch, 20. März 2019 12:59
An: Maximilian Böhm <[email protected]>
Cc: [email protected]
Betreff: Re: 400 SC on h2 xhr post
Hi Max,
On Wed, Mar 20, Maximilian Böhm wrote:
> >> If the 400 errors happen within 3mins, have you tried changing
> >> client/keep-alive timeouts to see if anything changes ?
> They do most often happen in the first 3 mins. But that's not always the
> case. And if it's really a timeout, shouldn't it be more clearly recurring?
> Like every tenth request fails. But that's also not the case. Sometimes it's
> the 3rd request, sometimes the 20th or even later.
> However, I did increase the previously set timeouts (40min). But this did not
> change anything at all. Is there another timeout which explicitly only
> affects h2 on the client side?
I'm not aware of any more timeouts to test. I think possible timeouts are in
https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#4.1
Have you tested different values for http-reuse (never to always) ?
(https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#4.2-http-reuse)
(Probably doesn't make any difference).
This could be related to
https://www.mail-archive.com/[email protected]/msg32959.html
that test case also returns 400 error with state CH-- with http2.
-Jarno
> -----Ursprüngliche Nachricht-----
> Von: Jarno Huuskonen <[email protected]>
> Gesendet: Dienstag, 19. März 2019 17:34
> An: Maximilian Böhm <[email protected]>
> Cc: [email protected]
> Betreff: Re: 400 SC on h2 xhr post
>
> Hi,
>
> On Tue, Mar 19, Maximilian Böhm wrote:
> > The problem I experience is within a legacy javascript application which
> > periodically checks if the user is still logged in. It does so by sending
> > an xhr request every 30 seconds (I said, it's a legacy app, right? It does
> > so by POST not GET...). As you may guess, this behavior works using http1.1
> > quasi infinitely. But as soon as I activate HTTP/2, I'll get the following
> > output (sooner or later):
> > 172.17.0.1:46372 [19/Mar/2019:12:10:13.465] [fntnd] [bknd] 0/0/0/14/14 200
> > 368 - - ---- 1/1/0/1/0 0/0 "POST [URL] HTTP/1.1"
> > 172.17.0.1:46372 [19/Mar/2019:12:10:43.465] [fntnd] [bknd] 0/0/0/-1/8 400
> > 187 - - CH-- 1/1/0/0/0 0/0 "POST [URL] HTTP/1.1"
> >
> > Which means, the developer toolbar announces a response code "400" and
> > "<html><body><h1>400 Bad request</h1>Your browser sent an invalid
> > request.</body></html>". I was not yet successful reproduce this behavior
> > with OkHttp (java http2-capable library). Jetty - on the backend site -
> > does not report any requests in its ncsa request log.
>
> I've seen some(very few (maybe one-two a day)) 400 bad requests with haproxy
> 1.9.4 (http2) to apache+php (http/1.1) backend. These requests alos have CH..
> state in logs.
> (400 errors have also happened for GET requests).
>
> > It is not directly reproducible (like every second time) but it usually
> > happens with the first 3 minutes. I experienced this behavior in Chrome
> > (73.0.3683.75), Firefox (65.0.2 (32-Bit)) and Edge (44.17763.1.0). I also
> > tried with different networks and different internet connections.
> >
> > Any ideas? Maybe a similar bug is known? What shall/can I do next? Setting
> > up Wireshark with MITM and comparing the requests? Right now, I can't
> > imagine the error is on side of the client nor on the backend (the backend
> > is not changed).
>
> If the 400 errors happen within 3mins, have you tried changing
> client/keep-alive timeouts to see if anything changes ?
>
> > timeout queue 2m
> > timeout client 2m
> > timeout http-keep-alive 2m
smime.p7s
Description: S/MIME cryptographic signature
