Hi,
HAProxy 3.0-dev1 was released on 2024/01/06. It added 136 new commits
after version 3.0-dev0. I figured we're already one month after 2.9 was
released, so it was about time to issue a first -dev release, even if
the period is usually calm due to vacation.
This version mostly contains fixes for early bugs affecting 2.9 (about
35), most of which in the area of end-of-stream detection during fast
forwarding combined with corner cases There were also a few issues
involving incorrect locking (QUIC CID tree), and pattern ordering in
maps due to the recent optimizations.
There are also a few add-ons and improvements:
- support for the fast forwarding mechanism in applets. This will
ultimately result in lower memory and higher performance for some
applets such as the cache by carefully avoiding to queue more data
when the mux buffer is already full. This can still be disabled by
unsetting tune.cache.zero-copy-forwarding.
- new support for virtual and optional files for patterns. Some users
had to create empty files in order to use maps internally. Now by
prefixing a map file name with "virt@", it will just be an internal
name that will not be looked up on the file system. However all
operations remain supported (adding entries etc). In addition, the
"opt@" prefix supports loading files only if they exist, falling
back to an empty map. This can be useful for per-customer routing
or exceptions lists for example, without having to deal with them on
a case by case basis.
- certain warnings about the presence of HTTP rules in TCP frontends
that are going to be upgraded to HTTP when switching to a backend
will now no longer be reported when it is certain that they will work
as expected. Something partially related is that it happens that stats
configured in a TCP frontend with an explicit upgrade rule would
previously cause a segfault and are now fixed.
- a new set of converters, map_*_key, will report the matching part of
the key itself instead of the associated pattern. This was requested
several times to know what address mask an address did match, or what
regex a pattern did match. Till now the only way to do this was to
build a special map with two strictly identical columns, but now this
becomes simpler.
- the HTTP/2 mux now supports limiting the total number of streams per
connection. There are situations where a client periodically performs
a request (crawlers, API clients etc) and where because of this, it's
difficult to cleanly stop the process, or to force some clients to
reconnect to another node just to maintain a better balance between
multiple frontend nodes, etc. One of the issue is directly related to
the current lack of ability to force to close a connection from HTTP
rules, but even without rules, it makes sense to be able to say that
one wants a connection to be renewed after 1000 requests for example.
Nothing is changed by default, of course, but now at least those who
need this will be able to configure it ("tune.h2.fe-max-total-streams").
- QUIC and HTTP/3 added some traces and refined some error reporting.
- ebtree backports that improve performance on non-x86 machines
(~+3% task switching rate and ~2% faster string lookups on ARM)
- some of the remaining server name lookups that were still linear moved
to use the tree instead, speeding up certain operations or config
parsing.
- Prometheus supports exporting a few more per-server metrics.
- The new certificate selection callback for WolfSSL was now enabled
since it's finally available in the upstream project.
- show/set/clear table now supports a "ptr" argument to directly use the
pointer retrieved from a previous "show" command.
As usual, numerous cleanups all over the code and various doc updates
were merged, and I think that's about all. There's a pending patch from
on the list to update ssl_fc_curve()/ssl_bc_curve() for newer openssl
versions but I prefer to let one of the SSL maintaines check it next
week when they're back (it's not lost Mariam :-)).
By the way some of the fixes mentioned above were already backported to
2.9 and we expect to produce another one soon, probably next week, in
order to help those facing issues. If you've faced a regression from
2.8 to 2.9, please try latest maintenance snapshot and voice in if it
persists.
Please find the usual URLs below :
Site index : https://www.haproxy.org/
Documentation : https://docs.haproxy.org/
Wiki : https://github.com/haproxy/wiki/wiki
Discourse : https://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Sources : https://www.haproxy.org/download/3.0/src/
Git repository : https://git.haproxy.org/git/haproxy.git/
Git Web browsing : https://git.haproxy.org/?p=haproxy.git
Changelog : https://www.haproxy.org/download/3.0/src/CHANGELOG
Dataplane API :
https://github.com/haproxytech/dataplaneapi/releases/latest
Pending bugs : https://www.haproxy.org/l/pending-bugs
Reviewed bugs : https://www.haproxy.org/l/reviewed-bugs
Code reports : https://www.haproxy.org/l/code-reports
Latest builds : https://www.haproxy.org/l/dev-packages
Willy
---
Complete changelog :
Amaury Denoyelle (32):
MINOR: hq-interop: add fastfwd support
CLEANUP: mux_quic: rename ffwd function with prefix qmux_strm_
MINOR: mux-quic: add traces for 0-copy/fast-forward
CLEANUP: mux-quic: remove unused prototype
MINOR: mux-quic: clean up qcs Rx buffer allocation API
MINOR: mux-quic: clean up qcs Tx buffer allocation API
CLEANUP: mux-quic: clean up app ops callback definitions
MINOR: mux-quic: factorize QC_SF_UNKNOWN_PL_LENGTH set
MINOR: h3: complete traces for sending
MINOR: h3: adjust zero-copy sending related code
MINOR: hq-interop: use zero-copy to transfer single HTX data block
BUG/MEDIUM: mux-quic: report early error on stream
MINOR: h3: remove quic_conn only reference
MINOR: mux-quic: remove qcc_shutdown() from qcc_release()
MINOR: mux-quic: use qcc_release in case of init failure
MINOR: mux-quic: adjust error code in init failure
MINOR: h3: add traces for connection init stage
BUG/MINOR: h3: properly handle alloc failure on finalize
MINOR: h3: use INTERNAL_ERROR code for init failure
DOC: fix typo for fastfwd QUIC option
BUG/MINOR: mux-quic: always report error to SC on RESET_STREAM emission
MEDIUM: mux-quic: add BUG_ON if sending on locally closed QCS
BUG/MINOR: mux-quic: disable fast-fwd if connection on error
MINOR: h3: check connection error during sending
BUG/MINOR: h3: close connection on header list too big
BUG/MINOR: h3: close connection on sending alloc errors
BUG/MINOR: h3: disable fast-forward on buffer alloc failure
Revert "MINOR: mux-quic: Disable zero-copy forwarding for send by default"
BUG/MINOR: server: fix server_find_by_name() usage during parsing
REGTESTS: check attach-srv out of order declaration
BUG/MEDIUM: h3: fix incorrect snd_buf return value
MINOR: h3: do not consider missing buf room as error on trailers
Aurelien DARRAGON (36):
BUG/MEDIUM: map/acl: pat_ref_{set,delete}_by_id regressions
BUG/MINOR: ext-check: cannot use without preserve-env
MINOR: stats: store the parent proxy in stats ctx (http)
BUG/MEDIUM: stats: unhandled switching rules with TCP frontend
MEDIUM: proxy: set PR_O_HTTP_UPG on implicit upgrades
MINOR: proxy: monitor-uri works with tcp->http upgrades
OPTIM: server: eb lookup for server_find_by_name()
OPTIM: server: ebtree lookups for findserver_unique_* functions
MINOR: server/event_hdl: add server_inetaddr struct to facilitate event
data usage
MINOR: server/event_hdl: update _srv_event_hdl_prepare_inetaddr prototype
BUG/MINOR: server/event_hdl: propagate map port info through inetaddr
event
MINOR: server: ensure connection cleanup on server addr changes
CLEANUP: server/event_hdl: remove purge_conn hint in INETADDR event
MEDIUM: server: merge srv_update_addr() and srv_update_addr_port() logic
CLEANUP: server: remove unused server_parse_addr_change_request() function
CLEANUP: resolvers: remove duplicate func prototype
MINOR: resolvers: add unique numeric id to nameservers
MEDIUM: server: make server_set_inetaddr() updater serializable
MINOR: server/event_hdl: expose updater info through INETADDR event
MINOR: server: add dns hint in server_inetaddr_updater struct
MEDIUM: server/dns: clear RMAINT when addr resolves again
BUG/MINOR: server/dns: use server_set_inetaddr() to unset srv addr from
DNS
BUG/MEDIUM: server/dns: perform svc_port updates atomically from SRV
records
MEDIUM: peers: use server as stream target
CLEANUP: peers: remove unused sock_init_arg struct member
CLEANUP: peers: remove unused "proto" and "xprt" struct members
MINOR: peers: rely on srv->addr and remove peer->addr
DOC: config: add context hint for server keywords
MINOR: stktable: add table_process_entry helper function
MINOR: stktable: use {show,set,clear} table with ptr
MINOR: map: add map_*_key converters to provide the matching key
MINOR: stktable: stktable_data_ptr() cannot fail in table_process_entry()
CLEANUP: server: remove ambiguous check in srv_update_addr_port()
CLEANUP: resolvers: remove unused RSLV_UPD_OBSOLETE_IP flag
CLEANUP: resolvers: remove some more unused RSLV_UDP flags
MEDIUM: server: simplify snr_set_srv_down() to prevent confusions
Christopher Faulet (27):
MINOR: channel: Use dedicated functions to deal with STREAMER flags
MEDIUM: applet: Handle channel's STREAMER flags on applets size
MINOR: applets: Use channel's field to compute amount of data received
MEDIUM: cache: Save body size of cached objects and track it on delivery
MEDIUM: cache: Add support for endp-to-endp fast-forwarding
MINOR: cache: Add global option to enable/disable zero-copy forwarding
MINOR: pattern: Use reference name as filename to read patterns from a
file
MEDIUM: pattern: Add support for virtual and optional files for patterns
DOC: config: Add section about name format for maps and ACLs
DOC: management/lua: Update commands about map and acl
MINOR: promex: Add support for specialized front/back/li/srv metric names
MINOR: promex: Export active/backup metrics per-server
BUG/MEDIUM: stconn: Block zero-copy forwarding if EOS/ERROR on consumer
side
BUG/MEDIUM: mux-h1: Cound data from input buf during zero-copy forwarding
BUG/MEDIUM: mux-h1: Explicitly skip request's C-L header if not set
originally
CLEANUP: mux-h1: Fix a trace message about C-L header addition
BUG/MEDIUM: mux-h2: Report too large HEADERS frame only when rxbuf is
empty
BUG/MEDIUM: mux-h2: Switch pending error to error if demux buffer is empty
BUG/MEDIUM: mux-h2: Only Report H2C error on read error if demux buffer
is empty
BUG/MEDIUM: mux-h2: Don't report error on SE if error is only pending on
H2C
BUG/MEDIUM: mux-h2: Don't report error on SE for closed H2 streams
DOC: config: Update documentation about local haproxy response
BUG/MINOR: server: Use the configured address family for the initial
resolution
BUG/MAJOR: stconn: Disable zero-copy forwarding if consumer is shut or in
error
BUG/MEDIUM: stconn: Forward shutdown on write timeout only if it is
forwardable
BUG/MEDIUM: stconn: Set fsb date if zero-copy forwarding is blocked
during nego
BUG/MEDIUM: spoe: Never create new spoe applet if there is no server up
Dragan Dosen (3):
MINOR: backend: export get_server_*() functions
MINOR: tcpcheck: export proxy_parse_tcpcheck()
MEDIUM: udp: allow to retrieve the frontend destination address
Frédéric Lécaille (10):
BUG/MINOR: ssl: Double free of OCSP Certificate ID
MINOR: ssl/cli: Add ha_(warning|alert) msgs to CLI ckch callback
BUG/MINOR: ssl: Wrong OCSP CID after modifying an SSL certficate
BUG/MINOR: lua: Wrong OCSP CID after modifying an SSL certficate (LUA)
BUG/MEDIUM: quic: Possible buffer overflow when building TLS records
BUG/MEDIUM: quic: QUIC CID removed from tree without locking
BUG/MINOR: quic: Wrong keylog callback setting.
BUG/MINOR: quic: Missing call to TLS message callbacks
CLEANUP: quic: Remaining useless code into server part
BUILD: quic: Missing quic_ssl.h header protection
Ilya Shipitsin (2):
CLEANUP: assorted typo fixes in the code and comments
CI: use semantic version compare for determing "latest" OpenSSL
William Lallemand (7):
DOC: configuration: typo req.ssl_hello_type
BUG/MINOR: mworker/cli: fix set severity-output support
CLEANUP: mworker/cli: add comments about pcli_find_and_exec_kw()
BUILD: ssl: update types in wolfssl cert selection callback
MINOR: ssl: activate the certificate selection callback for WolfSSL
CI: github: switch to wolfssl git-c4b77ad for new PR
BUG/MINOR: resolvers: default resolvers fails when network not configured
Willy Tarreau (19):
DOC: config: add arguments to sample fetch methods in the table
DOC: config: also add arguments to the converters in the table
SCRIPTS: mk-patch-list: produce a list of patches
DEV: patchbot: add the AI-based bot to pre-select candidate patches to
backport
DEV: patchbot: use checked buttons as reference instead of internal table
DEV: patchbot: allow to show/hide backported patches
MINOR: global: export a way to list build options
MINOR: debug: add features and build options to "show dev"
MINOR: mux-h2: support limiting the total number of H2 streams per
connection
CLEANUP: mux-h2: remove the printfs from previous commit on h2 streams
limit.
DEV: h2: add the ability to emit literals in mkhdr
DEV: h2: add the preface as well in supported output types
DEV: h2: support passing raw data for a frame
IMPORT: ebtree: implement and use flsnz_long() to count bits
IMPORT: ebtree: switch the sizes and offsets to size_t and ssize_t
IMPORT: ebtree: rework the fls macros to better deal with arch-specific
ones
IMPORT: ebtree: make string_equal_bits turn back to unsigned char
IMPORT: ebtree: use unsigned ints for flznz()
IMPORT: ebtree: make string_equal_bits() return an unsigned
---
