SPOP_CF_DEM_BLOCK_ANY is documented as "aggregate of the demux flags above except DALLOC/DFULL", like H2_CF_DEM_BLOCK_ANY and FCGI_CF_DEM_BLOCK_ANY, but its value 0x7E0 also covers SPOP_CF_DEM_DALLOC (0x20) and SPOP_CF_DEM_DFULL (0x40).
When a single recv() fills the demux buffer, spop_recv() sets SPOP_CF_DEM_DFULL. spop_process() then skips spop_process_demux() because the flag is part of SPOP_CF_DEM_BLOCK_ANY, and the only place clearing SPOP_CF_DEM_DFULL is inside that skipped block. spop_recv_allowed() also refuses to read. The connection stays stuck with a full dbuf forever: the peer sees a zero TCP window, ACK frames are never parsed, every stream attached to the connection times out, and new streams keep being attached to it because it is still considered available. This happens with pipelining when enough ACK frames are queued at once to fill the 16kB demux buffer in one read (e.g. ACKs carrying ~1-3kB variables), so "no option pipelining" in the spoe-agent section hides it. The value must be 0x780 (MROOM|SALLOC|SFULL|TOOMANY). Reproduced on HAProxy 3.4.7 with nbthread 4, pipelining enabled, 50 concurrent HTTP keep-alive clients, and a Go SPOA agent returning ACK frames carrying approximately 3kB of variables. In three 10-second test runs, the unpatched version left 3-5 connections stuck with DEM_DFULL and produced 400-520 SPOE timeouts per run. With the fix, no stuck connections or SPOE timeouts were observed. This must be backported as far as 3.1. Co-authored-by: ITGG <[email protected]> --- include/haproxy/mux_spop-t.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/haproxy/mux_spop-t.h b/include/haproxy/mux_spop-t.h index 3c3fcc9d6..2e778835b 100644 --- a/include/haproxy/mux_spop-t.h +++ b/include/haproxy/mux_spop-t.h @@ -21,7 +21,7 @@ #define SPOP_CF_DEM_SALLOC 0x00000100 /* demux blocked on lack of stream's rx buffer */ #define SPOP_CF_DEM_SFULL 0x00000200 /* demux blocked on stream request buffer full */ #define SPOP_CF_DEM_TOOMANY 0x00000400 /* demux blocked waiting for some stream connectors to leave */ -#define SPOP_CF_DEM_BLOCK_ANY 0x000007E0 /* aggregate of the demux flags above except DALLOC/DFULL */ +#define SPOP_CF_DEM_BLOCK_ANY 0x00000780 /* aggregate of the demux flags above except DALLOC/DFULL */ /* Other flags */ #define SPOP_CF_DISCO_SENT 0x00001000 /* a frame DISCONNECT was successfully sent */ -- 2.43.0
