Hi, HAProxy 2.3.8 was released on 2021/03/25. It added 22 new commits after version 2.3.7.
This version focuses on fixing a few annoying bugs that were reported on 2.3.7 and older, some of which were recent regressions. - Lua: Maciej hit a situation where the watchdog occasionally triggers while one of his threads is in the Lua's memory allocator. While this is particuarly strange and needs to be investigated, it had the bad taste of trying to dump a Lua backtrace which itself required a memory allocation, and used to deadlock inside the libc's allocator! Christopher fixed this by using a different Lua trace function that does not require any memory allocation. - a fix for a locking bug in the FD migration code unveiled an AB/BA situation with the listeners, that is more visible in 2.2: when reloading processes under a high connection rate traffic, it was possible for one thread to try to stop the listeners while another one had the listener lock, and both would wait for each other, until the watchdog reconciliated them the hard way. As this would require very high loads and would only affect the old process, I guess most users didn't notice, but Christopher triggered it on a reg-test and since then we could only see it. In 2.3 and 2.4 it will be harder to trigger it but it's definitely possible while resuming a temporarily paused listener. This was fixed with Olivier's help by deferring the close to the last user of the FD. - when I fixed the freq_ctr issue (multiple rotation and occasional livelock), I addressed both second-based counters and period-based ones. However my fix on the latter only worked for sub-second periods which were the ones I used in my tests :-/ The visible effect is that some rate counters in stick-tables averaged over more than one second will indefinitely grow. That's now fixed and I could verify that any delay now works. And I still have no idea how to reliably test rate counters automatically. On the other hand if we break them once every 10 years, I'm wondering if it's really worth worrying about them :-/ - during the backport to 2.3 of the patch to relieve the contention on the idle_conns_lock had significant contextual changes, and I apparently misplaced an unlock there that Christopher fixed. - when using DEBUG_UAF (to detect use-after-free), we use mmap/munmap to allocate/free memory. Since this can take a lot of time, the thread entering releases there marks itself harmless around the calls. But Christopher found one case where if that's done under thread_isolate() as happens during pool_gc(), the return back to normal operation can deadlock with the thread itself. - we were contacted by BitMEX security team and @melardev who found that haproxy would accept garbage after an IP address that was improperly formatted by their first level proxy resulting in an issue not being detected for a while. For example if the first layer incorrectly uses semi-colons instead of colons to delimit addresses, the real field would read "1.2.3.4; 5.6.7.8" and haproxy would return 1.2.3.4. The reason is that the parser was also used to parse delimited addresses in URLs. So now we comply more strictly to the format in RFC7239, but still accept a colon and a port because at least Azure's application gateway is documented for appending this, but we reject everything else. This will be easier to detect than an apparently valid address that does not correspond to what the first level proxy meant. - the log-forward section would not support forwarding from UNIX dgram sockets just because the protocol definition was missing! Emeric added it. - Rémi addressed an issue affecting the "add ssl crt-list" command on the CLI: if both a ctr-list and a ca-file were fed from the CLI, the openssl functions used to build them could perform FS accesses. That's nasty because in the unlucky case that one entry matches one present during, say, a config check or something like this, it would not necessarily work later after a chroot or a drop of privileges. Now these functions were adjusted to only use the in-memory API and to return errors directly on the CLI. - Ilya secured another case of OpenSSL version comparison that's more reliably handled by testing a macro. - Eric's previously mentionned fix for the unix-bind-prefix that used to mistakenly affect the master socket location was integrated. I would appreciate it if those who experienced issues with latest 2.2 could give this one a try on a single server and confirm it addresses their concerns as well, so that we're more confident in what goes into 2.2.12, probably early next week to still give some time for last issues to be reported. For all versions there's still an issue with ssl_sock_shutw() crashing under certain circumstances but it was already affecting 2.0.13 a year ago. I'm not aware of any other particular pending issue for 2.3. However there's one report of CLOSE_WAIT on H2 with latest 2.2 that's still under investigation. I'd prefer it if we can fix it before thee next release but it will not hold it up in any case. Please find the usual URLs below : Site index : http://www.haproxy.org/ Discourse : http://discourse.haproxy.org/ Slack channel : https://slack.haproxy.org/ Issue tracker : https://github.com/haproxy/haproxy/issues Wiki : https://github.com/haproxy/wiki/wiki Sources : http://www.haproxy.org/download/2.3/src/ Git repository : http://git.haproxy.org/git/haproxy-2.3.git/ Git Web browsing : http://git.haproxy.org/?p=haproxy-2.3.git Changelog : http://www.haproxy.org/download/2.3/src/CHANGELOG Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/ Willy --- Complete changelog : Christopher Faulet (7): BUG/MEDIUM: mux-fcgi: Fix locking of idle_conns lock in the FCGI I/O callback MEDIUM: lua: Use a per-thread counter to track some non-reentrant parts of lua BUG/MEDIUM: debug/lua: Don't dump the lua stack if not dumpable MINOR: lua: Slightly improve function dumping the lua traceback BUG/MEDIUM: debug/lua: Use internal hlua function to dump the lua traceback BUG/MEDIUM: lua: Always init the lua stack before referencing the context BUG/MEDIUM: thread: Fix a deadlock if an isolated thread is marked as harmless Emeric Brun (1): BUG/MINOR: protocol: add missing support of dgram unix socket. Eric Salama (1): MINOR/BUG: mworker/cli: do not use the unix_bind prefix for the master CLI socket Ilya Shipitsin (1): BUILD: ssl: guard ecdh functions with SSL_CTX_set_tmp_ecdh macro Olivier Houchard (1): BUG/MEDIUM: fd: Take the fd_mig_lock when closing if no DWCAS is available. Remi Tricot-Le Breton (1): BUG/MINOR: ssl: Prevent disk access when using "add ssl crt-list" Willy Tarreau (10): MINOR: time: export the global_now variable BUG/MINOR: freq_ctr/threads: make use of the last updated global time MINOR: time: also provide a global, monotonic global_now_ms timer BUG/MEDIUM: freq_ctr/threads: use the global_now_ms variable MINOR: fd: make fd_clr_running() return the remaining running mask MINOR: fd: remove the unneeded running bit from fd_insert() BUG/MEDIUM: fd: do not wait on FD removal in fd_delete() CLEANUP: fd: remove unused fd_set_running_excl() MINOR: tools: make url2ipv4 return the exact number of bytes parsed BUG/MINOR: http_fetch: make hdr_ip() reject trailing characters ---
