Hi Tristan, I actually read your thread a few times last night. It was helpful, but I couldn't isolate anything I was seeing to match what you're seeing (but it's entirely likely there could be overlap!).
> Out of curiosity, what version of HAProxy are you running? We're currently running latest 2.8.3, compiled with quictls 1.1.1. > I don’t have a magic answer but in > https://github.com/haproxy/haproxy/issues/2095#issuecomment-1570547484 we’ve > been looking at performance issues with H3/QUIC over time, and there’s a > couple of workarounds currently relevant (~ from the comment linked), notably > to use a short timeout connect (currently using 500ms here), and to comment > out timeout client-fin. That looks interesting. Our current timeout values are: # connect [to a backend server] timeout connect 12400ms # wait for a full request [from a client] timeout http-request 9300ms # wait for any data from the server (time to first byte/header) timeout server 60s # wait for the client (keep-alive for h1.1, client for h1.1 & h2) timeout http-keep-alive 15m timeout client 15m Which would also apply to H2 & H/1.1, so I'm hesitant to modify them too significantly. > I see you explicitly set per-connection socket ownership, which is definitely > the right thing to do, but you should check that it is actually working « for > real » and not silently disabled due to permissions somewhere inside the > worker processes. > To do that, have a look at the output of ‘show fd’ and check that H3 conns > aren’t all with fd=-1 (instead it should be a different positive fd number > per H3 conn) That's really good advice—I almost tried setcap last night, but assumed it would throw errors if it wasn't working…I'll try that next. https://jsfiddle.net/vqn3sea7/show Here's the JSFiddle. I can consistently trigger it in Safari on iOS by zooming in a few times and then swiping around fairly quickly (ends up queueing a lot of requests and eventually hanging forever. Thanks for the help! I'll see what I can turn up… Best, Luke — Luke Seelenbinder Stadia Maps | Founder & CEO stadiamaps.com > On Sep 22, 2023, at 14:56, Tristan <tris...@mangadex.org> wrote: > > Hi Luke, > >> Under some conditions (especially high request rate), H3 connections simply >> stall. This appears to be more prevalent in some browsers than others >> (Safari on iOS is the worst offender), and appears to coincide with high >> packet_loss in `show quic`. > > > Out of curiosity, what version of HAProxy are you running? > > I don’t have a magic answer but in > https://github.com/haproxy/haproxy/issues/2095#issuecomment-1570547484 we’ve > been looking at performance issues with H3/QUIC over time, and there’s a > couple of workarounds currently relevant (~ from the comment linked), notably > to use a short timeout connect (currently using 500ms here), and to comment > out timeout client-fin. > > I see you explicitly set per-connection socket ownership, which is definitely > the right thing to do, but you should check that it is actually working « for > real » and not silently disabled due to permissions somewhere inside the > worker processes. > To do that, have a look at the output of ‘show fd’ and check that H3 conns > aren’t all with fd=-1 (instead it should be a different positive fd number > per H3 conn) > > I’d definitely like to see the JSFiddle if there’s a trick to the kind of > request affected to see if I can reproduce it on our nodes. > > Regards, > Tristan
