Hi, HAProxy 2.4-dev14 was released on 2021/03/27. It added 82 new commits after version 2.4-dev13.
There were a bit more issues fixed in this version, basically all those that went into 2.3.8 a few days ago (watchdog+lua, FD locking issue with unbinding listeners, freq_ctr, thread_isolate vs DEBUG_UAF, hdr_ip() strictness on garbage, unix-dgram addresses, add ssl crt-list). A new one, reported by Stéphane Graber in issue 1197 shows that the h1_shutw_conn() function was not idempotent and that under rare circumstances there could be a second call there causing a crash in the SSL layer. This bug goes back to at least 2.0 and possibly even 1.9. The idle connections management code had two issues when trying to flush excess connections before creating a new one. First, it would keep the lock held while closing the lower layers, resulting in a deadlock since the lower layers eventually also need this lock. Second, under strong FD restrictions, the competition for the lock could be high and I managed to trigger the watchdog with only 16 threads. There was a previous fix for this at another place by using a trylock instead but this one remained unnoticed till now. There were some more cleanups and robustness improvements for the SSL layer so as to more rely on features than advertised versions. The pool management code started to get the first pass of cleanup (a second one is still needed). The server and proxy initialization / freeing code also got small cleanup so that Lua doesn't use statically allocated server structures anymore, and the initialization/freeing code now becomes generic (there are no Lua-specific duplicates anymore). A substantial part of the init functions that iterate over initcalls were finally moved from haproxy.c to a new init.c file, removing ~250 lines from haproxy.c Last thing, a small feature was added. While discussing with Baptiste about the pain it is to use "proc.something" variables in routing rules (e.g blue/green setups), he told me it would be so much easier if we could declare them in the global section. That made me remind that old wish we had to be able to set them from the CLI, that I didn't remember whether that was implemented or not (it was still not). So now there is a new directive "set-var proc.foo <expression>" supported in the global section to preset such process-wide variables, it's possible to retrieve them from the CLI using "get var name", and under experimental mode it's even possible to change them from the CLI. It's still experimental because while it looks OK it was slightly tricky. There's nothing revolutionary here, plenty of users used to rely on maps to achieve the same. But variables do not require an extra file and can now be preset in the global section, so this should nicely simplify some dynamic configs. There's still a reported issue about some CLOSE_WAIT being seen on 2.2 and 2.3 (issue #1195), I don't know yet if 2.4 is affected or not (it likely is since its lower layers are very close to 2.3). It's still under investigation. William proposed that we split the config manual a little bit by extracting all the stuff related to sample expressions, converters etc, which could also be a nice opportunity to better arrange that part and provide more examples. I think it could be beneficial, I just have no idea how far this could go. If done right it should remove a huge amount of copy-paste between sections in the doc! There's a discussion in issue #1198 about having a global option to indicate where relative paths should start. For now they start from the "current" directory, which nobody knows in case of services. There are a few options to set some of them (certs, UNIX sockets etc). There's nothing yet for maps/acls, and seeing the proliferation of all these options doesn't appeal me well. Here the proposed idea would be instead to have a single option to indicate if all such paths are relative to the current directory (current situation), the directory containing the config file, or the config directory's parent. I think this would allow plenty of users to simply deal with clean, relocatable archives containing configs, certs, map files, error files etc without having to know where they're going to be deployed. And instead of having to do it for each attempt to compute a path, we could simply have haproxy chdir to the config file's directory before continuing, then chdir back to the initial directory. There are probably caveats (e.g. post-config checks, though I don't think there are that many involving files). But I think something like this could be a nice improvement over the current situation, so I'm interested in feedback (better share it on the issue I think). Another discussion started around an easier support for some modern platforms. In issue #1194, Ashley Penney was caught running on AWS's ARM instances with the default ARM target optimizations. For having run some tests on these machines, I can't say enough how great they are, but I also know that in order to unlock their power when dealing with 16 cores or more, it's mandatory to use their modern locking extensions. Worse, the default ones do not scale at all and easily maintain deadlocks until the watchdog gets rid of the situation. I can trivially add a new CPU option in the makefile, there are already a few preset, it's easy. But the question is more how this could flow back into distro packages if possible at all, considering that such code will not run on legacy platforms (e.g. RPi). The difficulty here is that it's not about optimization anymore but rather choose from "crashes all the time" and "works amazingly fast". Another option could be for distros to limit the number of threads on such platforms to 4 to cover legacy devices well and prevent the degradation from happening on larger systems. But this will definitely require that users rebuild themselves to use larger platforms. In my opinion it is exactly the same problem we've seen a long time ago with x86 and cmov/mmx/sse/avx in that these are all extensions used at the lowest compiler level, so I'm sure there's a clean way to deal with this but I'm not qualified to say how. All ideas welcome! Please find the usual URLs below : Site index : http://www.haproxy.org/ Discourse : http://discourse.haproxy.org/ Slack channel : https://slack.haproxy.org/ Issue tracker : https://github.com/haproxy/haproxy/issues Wiki : https://github.com/haproxy/wiki/wiki Sources : http://www.haproxy.org/download/2.4/src/ Git repository : http://git.haproxy.org/git/haproxy.git/ Git Web browsing : http://git.haproxy.org/?p=haproxy.git Changelog : http://www.haproxy.org/download/2.4/src/CHANGELOG Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/ Willy --- Complete changelog : Amaury Denoyelle (11): CLEANUP: mark defproxy as const on parse tune.fail-alloc REGTESTS: remove unneeded experimental-mode in cli add server test REGTESTS: wait for proper return of enable server in cli add server test BUG/MEDIUM: release lock on idle conn killing on reached pool high count REORG: global: move initcall register code in a dedicated file REORG: global: move free acl/action in their related source files REORG: split proxy allocation functions MINOR: proxy: implement a free_proxy function MINOR: proxy: define cap PR_CAP_LUA MINOR: lua: properly allocate the lua Socket proxy MINOR: lua: properly allocate the lua Socket servers Christopher Faulet (4): MINOR: lua: Slightly improve function dumping the lua traceback BUG/MEDIUM: debug/lua: Use internal hlua function to dump the lua traceback BUG/MEDIUM: lua: Always init the lua stack before referencing the context BUG/MEDIUM: thread: Fix a deadlock if an isolated thread is marked as harmless Emeric Brun (1): MINOR: tools: introduce new option PA_O_DEFAULT_DGRAM on str2sa_range. Ilya Shipitsin (8): CI: codespell: whitelist "Dragan Dosen" CLEANUP: assorted typo fixes in the code and comments CI: github actions: update LibreSSL to 3.2.5 REGTESTS: revert workaround for a crash with recent libressl on http-reuse sni CLEANUP: ssl: remove unused definitions BUILD: ssl: guard ecdh functions with SSL_CTX_set_tmp_ecdh macro BUILD: ssl: introduce fine guard for ssl random extraction functions BUILD: ssl: use EVP_CIPH_GCM_MODE macro instead of HA_OPENSSL_VERSION Olivier Houchard (3): MEDIUM: quic: Fix build. MEDIUM: quic: Fix build. BUG/MEDIUM: fd: Take the fd_mig_lock when closing if no DWCAS is available. Remi Tricot-Le Breton (3): BUG/MINOR: ssl: Prevent disk access when using "add ssl crt-list" BUG/MINOR: ssl: Fix update of default certificate BUG/MINOR: ssl: Prevent removal of crt-list line if the instance is a default one Willy Tarreau (52): MINOR: compression: use pool_alloc(), not pool_alloc_dirty() MINOR: spoe: use pool_alloc(), not pool_alloc_dirty() MINOR: fcgi-app: use pool_alloc(), not pool_alloc_dirty() MINOR: cache: use pool_alloc(), not pool_alloc_dirty() MINOR: ssl: use pool_alloc(), not pool_alloc_dirty() MINOR: opentracing: use pool_alloc(), not pool_alloc_dirty() MINOR: dynbuf: make b_alloc() always check if the buffer is allocated CLEANUP: compression: do not test for buffer before calling b_alloc() CLEANUP: l7-retries: do not test the buffer before calling b_alloc() MINOR: channel: simplify the channel's buffer allocation MEDIUM: dynbuf: remove last usages of b_alloc_margin() CLEANUP: dynbuf: remove b_alloc_margin() CLEANUP: dynbuf: remove the unused b_alloc_fast() function CLEANUP: pools: remove the unused pool_get_first() function MINOR: pools: make the pool allocator support a few flags MINOR: pools: add pool_zalloc() to return a zeroed area CLEANUP: connection: use pool_zalloc() in conn_alloc_hash_node() CLEANUP: filters: use pool_zalloc() in flt_stream_add_filter() CLEANUP: spoe: use pool_zalloc() instead of pool_alloc+memset CLEANUP: frontend: use pool_zalloc() in frontend_accept() CLEANUP: mailers: use pool_zalloc() in enqueue_one_email_alert() CLEANUP: resolvers: use pool_zalloc() in resolv_link_resolution() CLEANUP: ssl: use pool_zalloc() in ssl_init_keylog() CLEANUP: tcpcheck: use pool_zalloc() instead of pool_alloc+memset CLEANUP: quic: use pool_zalloc() instead of pool_alloc+memset MINOR: time: also provide a global, monotonic global_now_ms timer BUG/MEDIUM: freq_ctr/threads: use the global_now_ms variable BUILD: tools: fix build error with new PA_O_DEFAULT_DGRAM MINOR: fd: make fd_clr_running() return the remaining running mask MINOR: fd: remove the unneeded running bit from fd_insert() BUG/MEDIUM: fd: do not wait on FD removal in fd_delete() CLEANUP: fd: remove unused fd_set_running_excl() CLEANUP: fd: slightly simplify up _fd_delete_orphan() MINOR: tools: make url2ipv4 return the exact number of bytes parsed BUG/MINOR: http_fetch: make hdr_ip() reject trailing characters BUG/MEDIUM: mux-h1: make h1_shutw_conn() idempotent MINOR: vars: make get_vars() allow the session to be null MINOR: vars: make the var() sample fetch keyword depend on nothing CLEANUP: sample: remove duplicate "stopping" sample fetch keyword MINOR: sample: make smp_resolve_args() return an allocate error message MINOR: sample: add a new SMP_SRC_CONST sample capability MINOR: sample: mark the truly constant sample fetch keywords as such MINOR: sample: add a new CFG_PARSER context for samples MINOR: action: add a new ACT_F_CFG_PARSER origin designation MEDIUM: vars: add support for a "set-var" global directive REGTESTS: add a basic reg-test for some "set-var" commands MINOR: sample: add a new CLI_PARSER context for samples MINOR: action: add a new ACT_F_CLI_PARSER origin designation MINOR: vars/cli: add a "get var" CLI command to retrieve global variables MEDIUM: cli: add a new experimental "set var" command MINOR: compat: add short aliases for a few very commonly used types MEDIUM: backend: use a trylock to grab a connection on high FD counts as well ---
