Hi, HAProxy 2.4-dev15 was released on 2021/04/02. It added 69 new commits after version 2.4-dev14.
I feel like we haven't done much this week due to the time spent dealing with the recent regressions in 2.3 and 2.2 :-/ With this said, we could still merge some long-pending stuff and continue the cleanups: - Christopher finally merged his long-term stabilization updates for the "tcp-request content" rule sets. The problem with this ruleset nowadays is that when used with HTTP, the L6 matches (those relying on req.len, req.payload) mean nothing as they just see the internal HTX contents. There is an emulation layer in place to decode HTTP on the fly but for TCP level it is meaningless. But these were sometimes needed in setups where a TCP frontend branches to an HTTP backend, leading to an implicit TCP->HTTP upgrade, in which case the rules would apply to TCP for the first request, or to HTTP for the next ones. And to add to the fun, I don't even remember what happens if a TCP->HTTP upgrade is done during a frontend-to-backend transition and an H2 upgrade is required, since all requests will have to pass in turn through the frontend again. Well, no need to enter into the long list of details, it's become a complete mess. We figured that the root cause of the problem was that users have valid reasons to use tcp-request rules in TCP frontend and to switch to HTTP backends, as that it was not possible to use http-request rules in the frontend. What was done was the addition of a new "switch-mode" action to the tcp-request ruleset, which ends the TCP analysis and switches to HTTP, where HTTP rules can be used. This will result in the ability to write cleaner configs in the future, where TCP is used only for TCP and HTTP is used everywhere else. Of course current working configs continue to work, but we can hope that over the course of a few years the tricky and unreliable ones will fade away (I think most users already noticed that TCP rules don't work exactly the same with H1 and H2 and tried to achieve something better). - Amaury added a long-awaited feature which is a diagnostic mode for the config: certain constructions are valid but suspicious, and we've often been hesitating about adding a warning or not. For me the rule has always been quite simple: there must always be a way to express a valid config without any warning, to encourage users to fix them. But because of this certain mistakes are hard to spot and can cause trouble. This was the goal of the diag-mode: start haproxy with -dD and watch the suggestions. It may report things that are totally valid for you but uncommon, or others that are the cause of your trouble. Since the addition is new, only a few checks were added (servers with weight 0 which sometimes result from a transient bug in a config generator, servers with the same cookie value, nbthread being specified more than once, out-of-order global sections). But the goal is to add more over time now that the infrastructure is in place, and these are things we can easily decide to backport later if they help users troubleshoot their setups. - I cleaned up the tests/ and contrib/ directories. The tests/ directory is now split into conf (test configs), exp (experimental stuff for developers), unit (unit tests for certain code areas). I expect it to become dirty again over time, it's not a big deal. The contrib/ directory however was a bit more challenging. I managed to establish a classification between the following groups: - development tools (code generators, debugging aids, etc). These were moved to dev/. Those depending on any include file are now built from the main makefile with automatic compiler options so that we don't take a shower of warnings anymore. In addition this will ensure that certain flags match what is used elsewhere. - admin tools (halog, systemd unit, selinux configs etc) were moved to admin/. Again those which need some includes are now built from the main makefile (e.g. halog). - optional addons which depend on 3rd-party products or popular tools (device detection, promex, opentracing) were moved to addons/. Some were slightly renamed (51d->51degrees, prometheus-exporter->promex, opentracing->ot) so that they all have a USE_xxx equivalent that matches the same name. Now using USE_PROMEX=1 is enough to build the prometheus exporter, no need for EXTRA_OBJS=... anymore. Some parts of the makefile could be moved there as opentracing does. Note, I think that some of the doc for the device detection addons could be moved to their respective directories, which would further simplify their discovery by users and even possibly their maintenance. If you're maintaining one of them (Ben, David, Paul), feel free to suggest or send patches. - and I figured that the last remaining ones were all SPOA agents (mod_defender, modsecurity, spoa_example, spoa_server). These ones are agnostic to the haproxy version, and we've already had to fix bugs there and backport the fixes everywhere while only the last version should be relevant. Thus for these ones I'm seriously thinking about taking them out of the repository and creating individual repositories on github/haproxy so that their respective maintainers can more easily update them or even share the effort with other participants. We could then just put a link to the wiki with an up-to-date list so that there is nothing to backport anymore. Christopher, Dragan, Thierry, I'm interested in your opinion on this. It's always satisfying to see some old stuff being tidied and cleaned up a little bit, but I also know I'm not the best one when it comes to proposing names. So if you feel uncomfortable with dev/ admin/ addons/ and have better ideas, feel free to suggest (but be aware that there are lots of places to adjust, including CI scripts). Please find the usual URLs below : Site index : http://www.haproxy.org/ Discourse : http://discourse.haproxy.org/ Slack channel : https://slack.haproxy.org/ Issue tracker : https://github.com/haproxy/haproxy/issues Wiki : https://github.com/haproxy/wiki/wiki Sources : http://www.haproxy.org/download/2.4/src/ Git repository : http://git.haproxy.org/git/haproxy.git/ Git Web browsing : http://git.haproxy.org/?p=haproxy.git Changelog : http://www.haproxy.org/download/2.4/src/CHANGELOG Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/ Willy --- Complete changelog : Amaury Denoyelle (6): MINOR: global: define diagnostic mode of execution MINOR: cfgparse: diag for multiple nbthread statements MINOR: server: diag for 0 weight server MINOR: diag: create cfgdiag module MINOR: diag: diag if servers use the same cookie value MINOR: config: diag if global section after non-global Christopher Faulet (30): BUG/MINOR: payload: Wait for more data if buffer is empty in payload/payload_lv BUG/MINOR: mux-h2: Don't emit log twice if an error occurred on the preface MINOR: stream: Don't trigger errors on destructive HTTP upgrades MINOR: frontend: Create HTTP txn for HTX streams MINOR: stream: Be sure to set HTTP analysers when creating an HTX stream BUG/MINOR: stream: Properly handle TCP>H1>H2 upgrades in http_wait_for_request BUG/MINOR: config: Add warning for http-after-response rules in TCP mode MINOR: muxes: Add a flag to notify a mux does not support any upgrade MINOR: mux-h1: Don't perform implicit HTTP/2 upgrade if not supported by mux MINOR: mux-pt: Don't perform implicit HTTP upgrade if not supported by mux MEDIUM: mux-h1: Expose h1 in the list of supported mux protocols MEDIUM: mux-pt: Expose passthrough in the list of supported mux protocols MINOR: muxes: Show muxes flags when the mux list is displayed DOC: config: Improve documentation about proto/check-proto keywords MINOR: stream: Use stream type instead of proxy mode when appropriate MINOR: filters/http-ana: Decide to filter HTTP headers in HTTP analysers MINOR: http-ana: Simplify creation/destruction of HTTP transactions MINOR: stream: Handle stream HTTP upgrade in a dedicated function MEDIUM: Add tcp-request switch-mode action to perform HTTP upgrade MINOR: config/proxy: Don't warn for HTTP rules in TCP if 'switch-mode http' set MINOR: config/proxy: Warn if a TCP proxy without backend is upgradable to HTTP DOC: config: Add documentation about TCP to HTTP upgrades REGTESTS: Add script to tests TCP to HTTP upgrades BUG/MINOR: payload/htx: Ingore L6 sample fetches for HTX streams/checks MINOR: htx: Make internal.strm.is_htx an internal sample fetch MINOR: action: Use a generic function to check validity of an action rule list MINOR: payload/config: Warn if a L6 sample fetch is used from an HTTP proxy MEDIUM: http-rules: Add wait-for-body action on request and response side REGTESTS: Add script to tests the wait-for-body HTTP action BUG/MINOR: http-fetch: Fix test on message state to capture the version Florian Apolloner (1): BUG/MINOR: stats: Apply proper styles in HTML status page. Julien Pivotto (1): DOC: clarify that compression works for HTTP/2 Miroslav Zagorac (1): BUG/MINOR: opentracing: initialization after establishing daemon mode William Lallemand (2): REGTESTS: ssl: "set ssl cert" and multi-certificates bundle REGTESTS: ssl: mark set_ssl_cert_bundle.vtc as broken Willy Tarreau (28): BUG/MEDIUM: time: make sure to always initialize the global tick BUG/MINOR: tcp: fix silent-drop workaround for IPv6 BUILD: tcp: use IPPROTO_IPV6 instead of SOL_IPV6 on FreeBSD/MacOS CLEANUP: socket: replace SOL_IP/IPV6/TCP with IPPROTO_IP/IPV6/TCP BUG/MINOR: http_fetch: make hdr_ip() resistant to empty fields CLEANUP: vars: always pre-initialize smp in vars_parse_cli_get_var() TESTS: slightly reorganize the code in the tests/ directory TESTS: move tests/*.cfg to tests/config CONTRIB: halog: fix issue with array of type char CONTRIB: tcploop: add a shutr command CONTRIB: debug: add the show-fd-to-flags script CONTRIB: debug: split poll from flags CONTRIB: move some dev-specific tools to dev/ BUILD: makefile: always build the flags utility DEV: flags: replace the unneeded makefile with a README BUILD: makefile: integrate the hpack tools CONTRIB: merge ip6range with iprange CONTRIB: move some admin-related sub-projects to admin/ CONTRIB: move halog to admin/ ADMIN: halog: automatically enable USE_MEMCHR on the right glibc version BUILD: makefile: build halog with the correct flags BUILD: makefile: add a "USE_PROMEX" variable to ease building prometheus-exporter CONTRIB: move prometheus-exporter to addons/promex DOC: add a few words about USE_* and the addons directory CONTRIB: move 51Degrees to addons/51degrees CONTRIB: move src/da.c and contrib/deviceatlas to addons/deviceatlas CONTRIB: move src/wurfl.c and contrib/wurfl to addons/wurfl CONTRIB: move contrib/opentracing to addons/ot ---