Hi, HAProxy 1.9.9 was released on 2019/07/23. It added 110 new commits after version 1.9.8. This release is large because over the last month several bugs affected 2.0 and likely 1.9 and it was uncertain whether some of the fixes to be backported to 1.9 were partially responsible for those bugs nor were totally correct until 2.0.2 was released. Now that 2.0.2 was released, these doubts have been cleared.
I'm not going to rehash all the fixes that were enumerated in 2.0.2 last week, many of which are also here but will only mention the most important ones. First we have a critical issue affecting HTX mode with cookies (stickiness or capture). It is possible to make haproxy enter an infinite loop by sending it an improperly formated cookie header. While in 2.0 this results in the watchdog killing the process, in 1.9 there is no such watchdog so the service becomes unresponsive. The simplest workaround is to not enable HTX mode (it is disabled by default, unless you explicitly enabled it with "option http-use-htx"). Those who enable it generally do so to support gRPC using end-to-end HTTP/2. In this case cookies are normally not used so it shouldn't be a problem to have to disable HTX when cookies are in use and conversely. If for any reason you cannot upgrade nor disable HTX nor disable cookies, there is another (ugly) workaround involving TCP request rules which seems to work but that I'm not posting as I'd rather not encourage people to deploy it. Just raise your hand if you really need it. Another major problem concerns the thread safety when dealing with limited listeners : deadlocks and crashes can happen when the frontend's or process's maxconn were reached on multiple threads and a connection is released by another thread. One workaround may simply consist in disabling threads or significantly raising the frontend's maxconn value. Another bug was affecting data forwarding in HTTP/1 in HTX mode. A mistake in the trash management when trying to limit the amount of copies could result in corrupted responses to be returned depending on the usage ratio of the buffers. The last major bug may trigger with threads when a server fails to accept a connection, then a redispatch happens and in the mean time the selected server becomes full and finally cannot accept the connection anymore. In this case we can enter an infinite loop trying to avoid this server. Most of the other bugs concern locking issues (when threads are enabled), connection issues (CLOSE_WAIT mainly and unhandled events causing 100% CPU while still serving the traffic), and risks of truncated payloads in HTX mode, 1xx response handling in HTX, and various issues affecting health checks that were already covered in great lengths in 2.0.x announces. Given the diversity of all the bugs fixed in this version, all users of 1.9 should upgrade, even if not affected by the security issue. Please find the usual URLs below : Site index : http://www.haproxy.org/ Discourse : http://discourse.haproxy.org/ Slack channel : https://slack.haproxy.org/ Issue tracker : https://github.com/haproxy/haproxy/issues Sources : http://www.haproxy.org/download/1.9/src/ Git repository : http://git.haproxy.org/git/haproxy-1.9.git/ Git Web browsing : http://git.haproxy.org/?p=haproxy-1.9.git Changelog : http://www.haproxy.org/download/1.9/src/CHANGELOG Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/ Willy --- Complete changelog : Ben51Degrees (2): MEDIUM: 51d: Enabled multi threaded operation in the 51Degrees module. BUG/MINOR: 51d/htx: The _51d_fetch method, and the methods it calls are now HTX aware. Christopher Faulet (47): BUG/MINOR: http_fetch: Rely on the smp direction for "cookie()" and "hdr()" BUG/MEDIUM: spoe: Don't use the SPOE applet after releasing it BUG/MINOR: mux-h2: Count EOM in bytes sent when a HEADERS frame is formatted BUG/MINOR: mux-h1: Report EOI instead EOS on parsing error or H2 upgrade BUG/MEDIUM: proto-htx: Not forward too much data when 1xx reponses are handled BUG/MINOR: htx: Remove a forgotten while loop in htx_defrag() BUG/MEDIUM: mux-h1: Don't switch the mux in BUSY mode on 1xx messages BUG/MINOR: mux-h1: Wake up the mux if it is busy when a 1xx response is handled BUG/MEDIUM: mux-h1: Don't skip the TCP splicing when there is no more data to read BUG/MINOR: channel/htx: Don't alter channel during forward for empty HTX message BUG/MINOR: mux-h1: errflag must be set on H1S and not H1M during output processing BUG/MINOR: mux-h1: Don't send more data than expected BUG/MEDIUM: compression/htx: Fix the adding of the last data block BUG/MINOR: channel/htx: Call channel_htx_full() from channel_full() BUG/MINOR: http: Use the global value to limit the number of parsed headers BUG/MEDIUM: proto_htx: Introduce the state ENDING during forwarding BUG/MINOR: flt_trace/htx: Only apply the random forwarding on the message body. MINOR: flt_trace: Don't scrash the original offset during the random forwarding BUG/MINOR: fl_trace/htx: Be sure to always forward trailers and EOM BUG/MINOR: mux-h1: Wake busy mux for I/O when message is fully sent BUG/MEDIUM: h2/htx: Update data length of the HTX when the cookie list is built BUG/MINOR: lua/htx: Make txn.req_req_* and txn.res_rep_* HTX aware BUG/MINOR: mux-h1: Add the header connection in lower case in outgoing messages BUG/MEDIUM: mux-h2: Reset padlen when several frames are demux BUG/MEDIUM: mux-h2: Remove the padding length when a DATA frame size is checked BUG/MEDIUM: lb_fwlc: Don't test the server's lb_tree from outside the lock BUG/MAJOR: mux-h1: Don't crush trash chunk area when outgoing message is formatted BUG/MINOR: memory: Set objects size for pools in the per-thread cache BUG/MEDIUM: mux-h1: Use buf_room_for_htx_data() to detect too large messages BUG/MINOR: mux-h1: Make format errors during output formatting fatal BUG/MEDIUM: mux-h1: Always release H1C if a shutdown for writes was reported BUG/MINOR: mux-h1: Skip trailers for non-chunked outgoing messages BUG/MINOR: mux-h1: Don't return the empty chunk on HEAD responses BUG/MEDIUM: lb_fas: Don't test the server's lb_tree from outside the lock MINOR: stream-int: Factorize processing done after sending data in si_cs_send() BUG/MEDIUM: stream-int: Don't rely on CF_WRITE_PARTIAL to unblock opposite si BUG/MINOR: server: Be really able to keep "pool-max-conn" idle connections BUG/MEDIUM: mux-h1: Don't release h1 connection if there is still data to send BUG/MINOR: http_fetch: Fix http_auth/http_auth_group when called from TCP rules BUG/MINOR: cache/htx: Make maxage calculation HTX aware BUG/MINOR: hlua: Make the function txn:done() HTX aware BUG/MINOR: session: Emit an HTTP error if accept fails only for H1 connection BUG/MINOR: session: Send a default HTTP error if accept fails for a H1 socket BUG/MEDIUM: mux-h1: Trim excess server data at the end of a transaction BUG/MINOR: mux-h1: Close server connection if input data remains in h1_detach() BUG/MINOR: http_ana: Be sure to have an allocated buffer to generate an error BUG/MINOR: http_htx: Support empty errorfiles Dave Pirotte (1): BUG/MINOR: mux-h1: Correctly report Ti timer when HTX and keepalives are used David Carlier (1): BUG/MEDIUM: da: cast the chunk to string. Emmanuel Hocdet (2): BUILD: makefile: use USE_OBSOLETE_LINKER for solaris BUILD: makefile: remove -fomit-frame-pointer optimisation (solaris) Frédéric Lécaille (1): BUG/MINOR: peers: Wrong stick-table update message building. Ilya Shipitsin (2): BUG/MINOR: ssl_sock: Fix memory leak when disabling compression BUILD: ssl: fix latest LibreSSL reg-test error Kazuo Yagi (1): MINOR: doc: Remove -Ds option in man page Michael Prokop (1): DOC: fix typos Olivier Houchard (16): BUG/MEDIUM: threads: Fix build for 32bits arch with dwcas. BUG/MEDIUM: streams: Don't switch from SI_ST_CON to SI_ST_DIS on read0. MEDIUM: sessions: Introduce session flags. BUG/MEDIUM: h2: Don't forget to set h2s->cs to NULL after having free'd cs. BUG/MEDIUM: connection: Use the session to get the origin address if needed. BUG/MEDIUM: connections: Don't call shutdown() if we want to disable linger. BUG/MEDIUM: connections: Don't use ALPN to pick mux when in mode TCP. BUG/MEDIUM: connections: Don't try to send early data if we have no mux. BUG/MEDIUM: ssl: Don't attempt to set alpn if we're not using SSL. BUG/MEDIUM: connections: Always call shutdown, with no linger. BUG/MEDIUM: checks: Make sure the tasklet won't run if the connection is closed. BUG/MEDIUM: sessions: Don't keep an extra idle connection in sessions. BUG/MEDIUM: servers: Don't forget to set srv_cs to NULL if we can't reuse it. BUG/MEDIUM: checks: Don't attempt to read if we destroyed the connection. BUG/MEDIUM: checks: Don't attempt to receive data if we already subscribed. BUG/CRITICAL: http_ana: Fix parsing of malformed cookies which start by a delimiter Tim Duesterhus (2): BUG/MEDIUM: compression: Set Vary: Accept-Encoding for compressed responses BUG/MINOR: spoe: Fix memory leak if failing to allocate memory William Lallemand (5): MINOR: doc: add master-worker in the man page MINOR: doc: mention HAPROXY_LOCALPEER in the man MINOR: doc: update the manpage and usage message about -S BUG/MEDIUM: mworker: don't call the thread and fdtab deinit BUG/MINOR: mworker/cli: don't output a \n before the response Willy Tarreau (28): BUILD: ist: turn the lower/upper case tables to literal on obsolete linkers DOC: management: place "show activity" at the right place MINOR: cli/activity: show the dumping thread ID starting at 1 BUG/MEDIUM: dns: make the port numbers unsigned BUG/MAJOR: lb/threads: make sure the avoided server is not full on second pass BUG/MEDIUM: queue: fix the tree walk in pendconn_redistribute. BUG/MEDIUM: threads: fix double-word CAS on non-optimized 32-bit platforms BUG/MEDIUM: http: fix "http-request reject" when not final BUG/MINOR: deinit/threads: make hard-stop-after perform a clean exit BUG/MEDIUM: connection: fix multiple handshake polling issues BUG/MEDIUM: mux-h1: only check input data for the current stream, not next one BUILD: tools: do not use the weak attribute for trace() on obsolete linkers BUG/MEDIUM: vars: make sure the scope is always valid when accessing vars BUG/MEDIUM: vars: make the tcp/http unset-var() action support conditions BUG/MINOR: time: make sure only one thread sets global_now at boot BUG/MEDIUM: mux-h2: make sure the connection timeout is always set BUG/MINOR: http-rules: mention "deny_status" for "deny" in the error message BUILD: makefile: use :space: instead of digits to count commits BUILD: makefile: do not rely on shell substitutions to determine git version BUG/MEDIUM: fd/threads: fix excessive CPU usage on multi-thread accept MINOR: task: introduce work lists BUG/MAJOR: listener: fix thread safety in resume_listener() BUG/MINOR: mux-pt: do not pretend there's more data after a read0 BUG/MEDIUM: tcp-check: unbreak multiple connect rules again BUG/MEDIUM: http/htx: unbreak option http_proxy BUG/MINOR: backend: do not try to install a mux when the connection failed BUG/MINOR: checks: do not exit tcp-checks from the middle of the loop BUG/MEDIUM: tcp-checks: do not dereference inexisting conn_stream mbellomi (1): BUG/MEDIUM: WURFL: segfault in wurfl-get() with missing info. ---
