Le 1/10/22 à 14:46, Christian Ruppert a écrit :
On 2022-01-10 13:33, Sander Klein wrote:Hi,I've upgraded to HAProxy 2.4.11 and now I seem to have a problem with bigger file uploads (>70MB). When uploading a file I get a 500 back from HAProxy, and if I retry it immediately it most of the time succeeds. Downgrading to 2.4.10 fixes the issue. The log I get is: Jan 10 12:09:45 [redacted] haproxy[21823]: 2001:67c:[redacted] [10/Jan/2022:12:09:20.543] [redacted]~ [redacted]/[redacted] 11198/0/0/-1/25137 500 1991 - - IH-- 957/282/0/0/0 0/0 {[redacted].[redacted].com|Mozilla/5.0 (Mac|80349066|https://[redacted].[redacted].com/upload} {} “POST https://[redacted].[redacted].com/upload/process?projectId=3431&setId=149 HTTP/2.0” The frontend is HTTP/2.0 and the backend is NGINX talking HTTP/1.1 (non-TLS). The config is quite large, but I think it boils down to: --- frontend [redacted] bind [redactes]]:80 transparent bind 2001:67c:[redacted]:80 transparent bind [redacted]:443 transparent ssl crt /etc/haproxy/ssl/[redacted] strict-sni alpn h2,http/1.1 npn h2,http/1.1 bind 2001:67c:[redacted]:443 transparent ssl crt /etc/haproxy/ssl/[redacted] strict-sni alpn h2,http/1.1 npn h2,http/1.1 mode http maxconn 16384 option httplog option dontlog-normal option http-ignore-probes option forwardfor option http-buffer-request capture request header Host len 64 capture request header User-Agent len 16 capture request header Content-Length len 10 capture request header Referer len 256 capture response header Content-Length len 10 acl [some ACLs here] acl [some ACLs here] http-request deny if [an ACL] http-request deny if [another ACL] use_backend [failing-backend] if [ACL] use_backend %[req.hdr(host),lower,regsub(^www\.,,i),map(/etc/haproxy/map.d/file.map,yes-backend)] default_backend another-backend backend failing-backend fullconn 256 mode http balance roundrobin option abortonclose option prefer-last-server option redispatch option httpchk GET /check-thingy HTTP/1.0 http-check expect status 200 default-server weight 100 maxconn 20 check inter 2s rise 3 fall 3 slowstart 5m agent-check agent-port 8081 agent-inter 20s server server1 [redacted]:80 cookie cookie1 server server2 [redacted]:80 cookie cookie2 # Sorry Server server outage 127.0.0.1:80 backup retries 1 --- If any more info is needed, please let me know. Regards, Sander KleinHi Sander, this might be also related to https://github.com/haproxy/haproxy/issues/1510
Thanks Christian. Indeed, it is most probably the same issue. The proposed patch is in attachment. The 2.6 and 2.5 are also affected.
-- Christopher Faulet
From 2984ccfc1f5c4ca1157f7a498dcdc7de2f7ba934 Mon Sep 17 00:00:00 2001 From: Christopher Faulet <[email protected]> Date: Mon, 10 Jan 2022 17:27:51 +0100 Subject: [PATCH] BUG/MAJOR: mux-h1: Don't decrement .curr_len for unsent data A regression was introduced by commit 140f1a585 ("BUG/MEDIUM: mux-h1: Fix splicing by properly detecting end of message"). To detect end of the outgoing message, when the content-length is announced, we count amount of data already sent. But only data really sent must be counted. If the output buffer is full, we can fail to send data (fully or partially). In this case, we must take care to only count sent data. Otherwise we may think too much data were sent and an internal error may be erroneously reported. This patch should solve issue #1511. It must be backported as far as 2.4. --- src/mux_h1.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/mux_h1.c b/src/mux_h1.c index f9a6120fe4..c2dc80834d 100644 --- a/src/mux_h1.c +++ b/src/mux_h1.c @@ -2180,7 +2180,6 @@ static size_t h1_process_output(struct h1c *h1c, struct buffer *buf, size_t coun H1_EV_TX_DATA|H1_EV_STRM_ERR|H1_EV_H1C_ERR|H1_EV_H1S_ERR, h1c->conn, h1s); goto error; } - h1m->curr_len -= vlen; } if ((h1m->flags & H1_MF_RESP) && (h1s->flags & H1S_F_BODYLESS_RESP)) { TRACE_PROTO("Skip data for bodyless response", H1_EV_TX_DATA|H1_EV_TX_BODY, h1c->conn, h1s, chn_htx); @@ -2226,6 +2225,8 @@ static size_t h1_process_output(struct h1c *h1c, struct buffer *buf, size_t coun H1_EV_TX_DATA|H1_EV_TX_BODY, h1c->conn, h1s, 0, (size_t[]){v.len}); skip_data: + if (h1m->state == H1_MSG_DATA && (h1m->flags & H1_MF_CLEN)) + h1m->curr_len -= vlen; if (last_data) goto done; break; -- 2.33.1
