Hi,
HAProxy 3.0-dev9 was released on 2024/04/27. It added 84 new commits
after version 3.0-dev8.
Less bug fixes this time and more merges. Most bug fixes were related to the
recent applet improvements and in peers.
Here's what's new:
- Following the previous series of fixes about the locking mechanism
of peers, it appeared there were some flaws. First, learning step
was not properly handled for the local peer, during a reload. Then
some operations performed by the peer applets and that had to be
acknowledged by the sync task were not blocking. It also appeared
the code was not obvious and hard to maintain. So a code cleanup /
reorganisation was performed to improve the situation.
- Add initial support for json, cbor(hex), and cbor(bin) encoding for
logformat expressions. This is achieved by using '{+json}', '{+cbor}'
and '{+cbor,+bin}' node options in sess_build_logline(). The series
is already known to cause a slight performance regression in
sess_build_logline(), mainly due to 0d1e99c08, 3a3bdf1c7, 2e4cc517b,
3f2e8d0ed. Tests are being carried out and optimizations should be
expected in a near future to match up with the original performance
for most common uses (ie: when no encoding is used). Note: unless
you're dealing with 100k+ requests per second, you have no chance to
ever detect the small perf regression, at best you may notice a few
percent increase in CPU usage. Since this is very small, we preferred
to expose the feature regardless so as to get more feedback on it.
- the stats-file feature was finally merged. It allows to dump and
preload stats counters across a reload. This has been mostly requested
by those who are using counters differences over a period to estimate
an activity. Not all counters are reloaded yet, but a number of
cumulative counters (requests, sessions, bytes, status codes) can be
transferred for frontends, backends, listeners, and servers. The
principle is that a complete dump of the current stats is produced
from the CLI using "dump stats-file" sent to a file (like with the
state file), and the new process when starting, opens that file,
figures which objects have counters available and presets their stats
counters from those in the file. Rates and ages are not reloaded yet.
Only objects having a guid assigned will be reloaded, since it is the
only criterion that is used to identify one such object (i.e. no more
ambiguity with server templates etc). We are thinking about providing
a way to automatically assign guids to objects to make this more
seamless for the many who manage their configs manually and would still
like to see their stats transferred, but we already anticipate that
different users coming from different use cases will have different
expectations on how to produce a stable guid, so for now we'll wait
for suggestions instead of trying to influence everyone with our first
thoughts ;-) Please look at "stats-file" in the doc for more info.
This feature was also an opportunity for performing a large cleanup
in the huge "stats.c" file and splitting it.
- the "uuid()" sample fetch function, which takes an optional version
in argument now also supports "7" for UUIDv7 (Thanks Tim!). These
UUIDs regroup many properties found in ULID and other mechanisms,
one of the most interesting one being time-based locality that, for
example, eases the archiving of old data, or the grouping of events
on systems where they'll be processed together.
- an interesting debugging feature that I was not aware of was
contributed by David Carlier: it allows to assign a name to some
memory areas on recent Linux versions (>=5.17) so that when reading
the process map (/proc/$pid/maps or using pmap), the area name
appears there. For now it's only applied to the shctx parts (SSL
and HTTP cache), but I'm already seeing how it could also be
generalized to rings, startup-logs, LRU cache, emergency buffers etc.
- as discussed a while ago, the "wait ... srv-unused" CLI command was
finally renamed to "wait ... srv-removable" to better match the
condition.
- and usual cleanups, regtests etc.
For what's remaining, I'm still trying to see if the fix we estimate for
the low-memory condition on buffers will work or if we should just change
the code everywhere to return hard errors when this happens. Hopefully by
the middle of next week I should know if it's worth going further. I'd
also like to re-apply the mt_list update so that we get the cleaner API
for the LTS version and ease backports or surrounding fixes. There are
some updates and cleanups almost ready around the linux capabilities to
permit to also support namespaces. Passing the gRPC RST reasons between
client and server should also be OK (it works in the lab, it's "just" a
matter to limit what's exposed not to get trapped). Some tiny updates
for rhttp are still planned, as well as applying the very basic building
blocks of glitch counters to h3/quic so that over time we can anticipate
the arrival of future protocol attacks. All of this is really small stuff,
but it's just a bunch of fairly different stuff and it's hard to know what
will be completed by the release. At least the intent is here :-)
Please continue to test and report, that's very useful, even if bug fixing
has the nasty effect of postponing integration of the small improvements
above, bugs are always much easier to fix during dev than during stable
(and they don't require a backport).
Please find the usual URLs below :
Site index : https://www.haproxy.org/
Documentation : https://docs.haproxy.org/
Wiki : https://github.com/haproxy/wiki/wiki
Discourse : https://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Sources : https://www.haproxy.org/download/3.0/src/
Git repository : https://git.haproxy.org/git/haproxy.git/
Git Web browsing : https://git.haproxy.org/?p=haproxy.git
Changelog : https://www.haproxy.org/download/3.0/src/CHANGELOG
Dataplane API :
https://github.com/haproxytech/dataplaneapi/releases/latest
Pending bugs : https://www.haproxy.org/l/pending-bugs
Reviewed bugs : https://www.haproxy.org/l/reviewed-bugs
Code reports : https://www.haproxy.org/l/code-reports
Latest builds : https://www.haproxy.org/l/dev-packages
Willy
---
Complete changelog :
Amaury Denoyelle (21):
MINOR: backend: use be_counters for health down accounting
BUG/MINOR: backend: use cum_sess counters instead of cum_conn
BUG/MINOR: stats: fix stot metric for listeners
MINOR: stats: rename proxy stats
MINOR: stats: rename ambiguous stat_l and stat_count
MINOR: stats: rename info stats
MINOR: stats: use stricter naming stats/field/line
MINOR: stats: use STAT_F_* prefix for flags
MINOR: stats: update ambiguous "metrics" naming to "stat_cols"
MINOR: stats: introduce a more expressive stat definition method
MINOR: stats: implement automatic metric generation from stat_col
MINOR: stats: hide some columns in output
MEDIUM: stats: convert counters to new column definition
MINOR: stats: define stats-file output format support
MEDIUM: stats: implement dump stats-file CLI
MINOR: ist: define iststrip() new function
MINOR: guid: define guid_is_valid_fmt()
MINOR: stats: apply stats-file on process startup
MINOR: stats: parse header lines from stats-file
MINOR: stats: parse values from stats-file
MEDIUM: stats: define stats-file keyword
Aurelien DARRAGON (24):
CLEANUP: log: remove unused checks for encode_{chunk,string}
MINOR: log: store lf_expr nodes inside substruct
MINOR: log: global lf_expr node options
CLEANUP: log: simplify complex values usages in sess_build_logline()
MINOR: log: skip custom logformat_node name if empty
MINOR: log: add lf_int() wrapper to print integers
MINOR: log: add lf_rawtext{_len}() functions
MEDIUM: log: pass date strings to lf_rawtext()
MEDIUM: log: write raw strings using lf_rawtext()
MEDIUM: log: use lf_rawtext for lf_ip() and lf_port() hex strings
MINOR: log: explicitly handle %ts and %tsc as text strings
MINOR: log: use LOG_VARTEXT_{START,END} to enclose text strings
MINOR: log: make all lf_* sess build helper static
MINOR: log: merge lf_encode_string() and lf_encode_chunk() logic
MEDIUM: log: lf_* build helpers now take a ctx argument
MINOR: log: expose node typecast in lf_buildctx struct
MINOR: log: postpone conversion for sample expressions in
sess_build_logline()
MINOR: log: add LOG_OPT_NONE flag
MINOR: log: add no_escape_map to bypass escape with _lf_encode_bytes()
MINOR: log: add +bin logformat node option
MINOR: log: add +json encoding option
MINOR: tools: add cbor encode helpers
MINOR: log: add +cbor encoding option
MINOR: log: support true cbor binary encoding
Christopher Faulet (23):
BUG/MEDIUM: applet: Let's applets decide if they have more data to deliver
BUG/MEDIUM: peers: Automatically start to learn on local peer
BUG/MEDIUM: peers: Reprocess peer state after all session shutdowns
MINOR: peers: Remove unused PEERS_F_RESYNC_REQUESTED flag
MINOR: peers: Don't set TEACH flags on a peer from the sync task
MINOR: peers: Use a peer flag to block the applet waiting ack of the sync
task
BUG/MEDIUM: peers: Wait for sync task ack when a resynchro is finished
MINOR: peers: Remove unused PEERS_F_RESYNC_PROCESS flag
MINOR: applet: Add a function to know the side where an applet was created
MEDIUM: peers: Simplify the peer flags dealing with the connection state
MEDIUM: peers: Use true states for the peer applets as seen from outside
MEDIUM: peers: Use true states for the learn state of a peer
MINOR: peers: Start learning for local peer before receiving messages
MINOR: peers: Rename PEERS_F_TEACH_COMPLETE to
PEERS_F_LOCAL_TEACH_COMPLETE
MINOR: peers: Reorder and slightly rename PEER flags
MINOR: peers: Reorder and rename PEERS flags
REORG: peers: Move peer and peers flags in the corresponding header file
DEV: flags/peers: Decode PEER and PEERS flags
MINOR: peers: Add comment on processing functions of the sync task
MINOR: peers: Use a static variable to wait a resync on reload
BUG/MEDIUM: peers: Use atomic operations on peers flags when necessary
REORG: peers: Rename all occurrences to 'ps' variable
BUG/MINOR: peers: Don't wait for a remote resync if there no remote peer
David Carlier (1):
MEDIUM: shctx: Naming shared memory context
Remi Tricot-Le Breton (1):
REGTESTS: ssl: Remove "sleep" calls from ocsp auto update test
Tim Duesterhus (3):
MINOR: tools: Rename `ha_generate_uuid` to `ha_generate_uuid_v4`
MINOR: Add `ha_generate_uuid_v7`
MINOR: Add support for UUIDv7 to the `uuid` sample fetch
William Lallemand (3):
BUILD: ssl: use %zd for sizeof() in ssl_ckch.c
REGTESTS: use -dI for insecure fork by default in the regtest scripts
BUG/MINOR: mworker: reintroduce way to disable seamless reload with -x
/dev/null
Willy Tarreau (8):
BUILD: stick-tables: silence build warnings when threads are disabled
BUG/MINOR: h1: fix detection of upper bytes in the URI
MINOR: intops: add a pair of functions to check multi-byte ranges
TESTS: add a unit test for the multi-byte range checks
CLEANUP: h1: make use of the multi-byte matching functions
CLEANUP: dynbuf: move the reserve and limit parsers to dynbuf.c
MINOR: list: add a macro to detect that a list contains at most one
element
MINOR: cli/wait: rename the condition "srv-unused" to "srv-removable"
---
