Hi all, One more bug (or configuration hole) from our transition to 1.9.x using end-to-end h2 connections.
After enabling h2 backends (technically `server … alpn h2,http/1.1`), we began seeing a high number of backend /server/ connection resets. A reasonable number of client-side connection resets due to timeouts, etc., is normal, but the server connection resets were new. I believe the root cause is that our backend servers are NGINX servers, which by default have a 1000 request limit per h2 connection (https://nginx.org/en/docs/http/ngx_http_v2_module.html#http2_max_requests). As far as I can tell there's no way to set this to unlimited. That resulted in NGINX resetting the HAProxy backend connections and thus resulted in user requests being dropped or returning 404s (oddly enough; though this may be as a result of the outstanding bug related to header manipulation and HTX mode). This wouldn't be a problem if one of the following were true: - HAProxy could limit the number of times it reused a connection - HAProxy could retry a failed request due to backend server connection reset (possibly coming in 2.0 with L7 retries?) - NGINX could set that limit to unlimited. Our http-reuse is set to aggressive, but that doesn't make much difference, I don't think, since safe would result in the same behavior (the connection is reusable…but only for a limited number of requests). We've worked around this by only using h/1.1 on the backends, which isn't a big problem for us, but I thought I would raise the issue, since I'm sure a lot of folks are using haproxy <-> nginx pairings, and this is a bit of a subtle result of that in full h2 mode. Thanks again for such great software—I've found it pretty fantastic to run in production. :) Best, Luke — Luke Seelenbinder Stadia Maps | Founder stadiamaps.com
publickey - [email protected] - 0xB23C1E8A.asc
Description: application/pgp-keys
signature.asc
Description: OpenPGP digital signature
