Hi,
HAProxy 2.4-dev15 was released on 2021/04/02. It added 69 new commits
after version 2.4-dev14.
I feel like we haven't done much this week due to the time spent dealing
with the recent regressions in 2.3 and 2.2 :-/
With this said, we could still merge some long-pending stuff and continue
the cleanups:
- Christopher finally merged his long-term stabilization updates for
the "tcp-request content" rule sets. The problem with this ruleset
nowadays is that when used with HTTP, the L6 matches (those relying
on req.len, req.payload) mean nothing as they just see the internal
HTX contents. There is an emulation layer in place to decode HTTP on
the fly but for TCP level it is meaningless. But these were sometimes
needed in setups where a TCP frontend branches to an HTTP backend,
leading to an implicit TCP->HTTP upgrade, in which case the rules
would apply to TCP for the first request, or to HTTP for the next
ones. And to add to the fun, I don't even remember what happens if
a TCP->HTTP upgrade is done during a frontend-to-backend transition
and an H2 upgrade is required, since all requests will have to pass
in turn through the frontend again. Well, no need to enter into the
long list of details, it's become a complete mess. We figured that
the root cause of the problem was that users have valid reasons to
use tcp-request rules in TCP frontend and to switch to HTTP backends,
as that it was not possible to use http-request rules in the frontend.
What was done was the addition of a new "switch-mode" action to the
tcp-request ruleset, which ends the TCP analysis and switches to HTTP,
where HTTP rules can be used. This will result in the ability to write
cleaner configs in the future, where TCP is used only for TCP and HTTP
is used everywhere else. Of course current working configs continue to
work, but we can hope that over the course of a few years the tricky
and unreliable ones will fade away (I think most users already noticed
that TCP rules don't work exactly the same with H1 and H2 and tried to
achieve something better).
- Amaury added a long-awaited feature which is a diagnostic mode for the
config: certain constructions are valid but suspicious, and we've often
been hesitating about adding a warning or not. For me the rule has
always been quite simple: there must always be a way to express a valid
config without any warning, to encourage users to fix them. But because
of this certain mistakes are hard to spot and can cause trouble. This
was the goal of the diag-mode: start haproxy with -dD and watch the
suggestions. It may report things that are totally valid for you but
uncommon, or others that are the cause of your trouble. Since the
addition is new, only a few checks were added (servers with weight 0
which sometimes result from a transient bug in a config generator,
servers with the same cookie value, nbthread being specified more than
once, out-of-order global sections). But the goal is to add more over
time now that the infrastructure is in place, and these are things we
can easily decide to backport later if they help users troubleshoot
their setups.
- I cleaned up the tests/ and contrib/ directories. The tests/ directory
is now split into conf (test configs), exp (experimental stuff for
developers), unit (unit tests for certain code areas). I expect it to
become dirty again over time, it's not a big deal. The contrib/
directory however was a bit more challenging. I managed to establish a
classification between the following groups:
- development tools (code generators, debugging aids, etc). These
were moved to dev/. Those depending on any include file are now
built from the main makefile with automatic compiler options so
that we don't take a shower of warnings anymore. In addition this
will ensure that certain flags match what is used elsewhere.
- admin tools (halog, systemd unit, selinux configs etc) were moved
to admin/. Again those which need some includes are now built from
the main makefile (e.g. halog).
- optional addons which depend on 3rd-party products or popular tools
(device detection, promex, opentracing) were moved to addons/. Some
were slightly renamed (51d->51degrees, prometheus-exporter->promex,
opentracing->ot) so that they all have a USE_xxx equivalent that
matches the same name. Now using USE_PROMEX=1 is enough to build
the prometheus exporter, no need for EXTRA_OBJS=... anymore. Some
parts of the makefile could be moved there as opentracing does.
Note, I think that some of the doc for the device detection addons
could be moved to their respective directories, which would further
simplify their discovery by users and even possibly their
maintenance. If you're maintaining one of them (Ben, David, Paul),
feel free to suggest or send patches.
- and I figured that the last remaining ones were all SPOA agents
(mod_defender, modsecurity, spoa_example, spoa_server). These ones
are agnostic to the haproxy version, and we've already had to fix
bugs there and backport the fixes everywhere while only the last
version should be relevant. Thus for these ones I'm seriously
thinking about taking them out of the repository and creating
individual repositories on github/haproxy so that their respective
maintainers can more easily update them or even share the effort
with other participants. We could then just put a link to the
wiki with an up-to-date list so that there is nothing to backport
anymore. Christopher, Dragan, Thierry, I'm interested in your
opinion on this.
It's always satisfying to see some old stuff being tidied and cleaned up
a little bit, but I also know I'm not the best one when it comes to
proposing names. So if you feel uncomfortable with dev/ admin/ addons/ and
have better ideas, feel free to suggest (but be aware that there are lots
of places to adjust, including CI scripts).
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse : http://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Wiki : https://github.com/haproxy/wiki/wiki
Sources : http://www.haproxy.org/download/2.4/src/
Git repository : http://git.haproxy.org/git/haproxy.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy.git
Changelog : http://www.haproxy.org/download/2.4/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
Willy
---
Complete changelog :
Amaury Denoyelle (6):
MINOR: global: define diagnostic mode of execution
MINOR: cfgparse: diag for multiple nbthread statements
MINOR: server: diag for 0 weight server
MINOR: diag: create cfgdiag module
MINOR: diag: diag if servers use the same cookie value
MINOR: config: diag if global section after non-global
Christopher Faulet (30):
BUG/MINOR: payload: Wait for more data if buffer is empty in
payload/payload_lv
BUG/MINOR: mux-h2: Don't emit log twice if an error occurred on the
preface
MINOR: stream: Don't trigger errors on destructive HTTP upgrades
MINOR: frontend: Create HTTP txn for HTX streams
MINOR: stream: Be sure to set HTTP analysers when creating an HTX stream
BUG/MINOR: stream: Properly handle TCP>H1>H2 upgrades in
http_wait_for_request
BUG/MINOR: config: Add warning for http-after-response rules in TCP mode
MINOR: muxes: Add a flag to notify a mux does not support any upgrade
MINOR: mux-h1: Don't perform implicit HTTP/2 upgrade if not supported by
mux
MINOR: mux-pt: Don't perform implicit HTTP upgrade if not supported by mux
MEDIUM: mux-h1: Expose h1 in the list of supported mux protocols
MEDIUM: mux-pt: Expose passthrough in the list of supported mux protocols
MINOR: muxes: Show muxes flags when the mux list is displayed
DOC: config: Improve documentation about proto/check-proto keywords
MINOR: stream: Use stream type instead of proxy mode when appropriate
MINOR: filters/http-ana: Decide to filter HTTP headers in HTTP analysers
MINOR: http-ana: Simplify creation/destruction of HTTP transactions
MINOR: stream: Handle stream HTTP upgrade in a dedicated function
MEDIUM: Add tcp-request switch-mode action to perform HTTP upgrade
MINOR: config/proxy: Don't warn for HTTP rules in TCP if 'switch-mode
http' set
MINOR: config/proxy: Warn if a TCP proxy without backend is upgradable to
HTTP
DOC: config: Add documentation about TCP to HTTP upgrades
REGTESTS: Add script to tests TCP to HTTP upgrades
BUG/MINOR: payload/htx: Ingore L6 sample fetches for HTX streams/checks
MINOR: htx: Make internal.strm.is_htx an internal sample fetch
MINOR: action: Use a generic function to check validity of an action rule
list
MINOR: payload/config: Warn if a L6 sample fetch is used from an HTTP
proxy
MEDIUM: http-rules: Add wait-for-body action on request and response side
REGTESTS: Add script to tests the wait-for-body HTTP action
BUG/MINOR: http-fetch: Fix test on message state to capture the version
Florian Apolloner (1):
BUG/MINOR: stats: Apply proper styles in HTML status page.
Julien Pivotto (1):
DOC: clarify that compression works for HTTP/2
Miroslav Zagorac (1):
BUG/MINOR: opentracing: initialization after establishing daemon mode
William Lallemand (2):
REGTESTS: ssl: "set ssl cert" and multi-certificates bundle
REGTESTS: ssl: mark set_ssl_cert_bundle.vtc as broken
Willy Tarreau (28):
BUG/MEDIUM: time: make sure to always initialize the global tick
BUG/MINOR: tcp: fix silent-drop workaround for IPv6
BUILD: tcp: use IPPROTO_IPV6 instead of SOL_IPV6 on FreeBSD/MacOS
CLEANUP: socket: replace SOL_IP/IPV6/TCP with IPPROTO_IP/IPV6/TCP
BUG/MINOR: http_fetch: make hdr_ip() resistant to empty fields
CLEANUP: vars: always pre-initialize smp in vars_parse_cli_get_var()
TESTS: slightly reorganize the code in the tests/ directory
TESTS: move tests/*.cfg to tests/config
CONTRIB: halog: fix issue with array of type char
CONTRIB: tcploop: add a shutr command
CONTRIB: debug: add the show-fd-to-flags script
CONTRIB: debug: split poll from flags
CONTRIB: move some dev-specific tools to dev/
BUILD: makefile: always build the flags utility
DEV: flags: replace the unneeded makefile with a README
BUILD: makefile: integrate the hpack tools
CONTRIB: merge ip6range with iprange
CONTRIB: move some admin-related sub-projects to admin/
CONTRIB: move halog to admin/
ADMIN: halog: automatically enable USE_MEMCHR on the right glibc version
BUILD: makefile: build halog with the correct flags
BUILD: makefile: add a "USE_PROMEX" variable to ease building
prometheus-exporter
CONTRIB: move prometheus-exporter to addons/promex
DOC: add a few words about USE_* and the addons directory
CONTRIB: move 51Degrees to addons/51degrees
CONTRIB: move src/da.c and contrib/deviceatlas to addons/deviceatlas
CONTRIB: move src/wurfl.c and contrib/wurfl to addons/wurfl
CONTRIB: move contrib/opentracing to addons/ot
---
