Hi,
HAProxy 2.9-dev2 was released on 2023/07/21. It added 91 new commits
after version 2.9-dev1. This remains pretty calm for 3 weeks, because
everyone seems to be busy making progress on complex stuff thanks to
2.8 appearing to be pretty good for now.
There are now a number of low-importance fixes in queue (deadlock on
listeners, a few issues with QUIC, etc). To give an idea of the extent
of fixes for now, one of them removes an unneeded '\n' in some error
essages... We've seen more dramatic changes :-)
Now regarding the new stuff:
- small rework of the sample expression parser to unify the converter
and sample-fetch parser in order to remove some code duplication. As
usual, no regression expected but please report any you'd notice. A
second more sensitive change concerns the introduction of a sample
type "same" that is not exposed but only used in declarations. This
solves a problem with a few converters such as "debug()" that return
the same type as the input but could only be declared with output type
"any", which prevents parsing error from being detected if an
incompatible converter is placed last. At this point only the "debug"
converter was changed. Again, no visible change is expected unless the
config is already broken. Finally, to conclude the sample conversion,
sample fetch functions that used to return both IPv4 and IPv6 such as
"src" etc now return a type "addr". The rationale behind this is that
we used to cheat by declaring them as returning only one type and
letting the converters cast on the fly. The "addr" type was only used
to resolve the chain of casts in the expression. Now these ones can
just return a type addr and the converter behind automatically handles
it. Same, theoretically there should be no visible effect.
- A new pair of SSL sample fetch functions were added: ssl_fc_curve and
ssl_bc_curve. These ones are only available with OpenSSL 3.0 and above,
and they are used to return the name of the curve used during the key
exchange.
- In Lua, it's now possible to read "proc" variables from the core API,
it was found to be convenient for various things such as global
thresholds or dynamic modes that can be adjusted on the fly using the
CLI. Also the Lua-based mailers now handle the "timeout mail" directive
from the mailers section.
- several QUIC fixes, updates and traces, as usual.
- SSL: after the discussion about whether or not to integrate the QUIC
compatibility layer to allow QUIC to be tested on top of OpenSSL, it
appeared that both here on this list and on the QUIC workgroup, the
general sentiment was rather against having this for basically 3
reasons that align with my initial concerns:
- would make users imagine that what they're testing is really
QUIC while it can be of lower quality
- would make the stubborn openssl team win the QUIC battle,
confirming they were right to piss off the whole world
- better aim the resources on supporting alternate libs
I generally agree with the first two. I don't agree with the 3rd
one because that's already what we've been doing for quite a while
and are still continuing, but despite our past and ongoing efforts
and commitments on this, it only very slowly improves the situation
and doesn't solely depend on us, but also library implementers, their
available resources, the ability for distros to adopt new components,
etc. We haven't reduce our efforts on this, we do sincerely hope that
LTS distros shipping next year will support wolfSSL out of the box,
and maybe AWS-LC, who knows so we expect to have more choice. But
this doesn't solve the issue users are facing right now. And I've
received quite a few comments over the last year from people saying
"do you imagine we can build quictls in field?". Of course I know
it's not possible at plenty of places. I also understand that such
people are not actively looking for QUIC deployments right now, but
rather to validate the compability with their infrastructure and
start to run tests. Some are worried that fragmentation may happen
over VPN (we can say "don't worry" but they'll believe it when seeing
it). Others about their firewall's timeouts impacts on bidirectional
UDP streams, etc. Finally, the QUIC maintainers themselves would
welcome bug reports caused by whatever bug may be left. So this tends
to indicate that there is a demand for having something very close to
QUIC with much less deployment trouble, and without fooling the users
into thinking that it's a real QUIC. For now we've thought that
enabling this mechanism only at build time via USE_QUIC_OPENSSL_COMPAT=1
and in the config as well with a directive "limited-quic" in the global
section could be reasonable fit. Maybe we could rename the option to
make it sound even more limited (we thought about "reduced", "degraded",
"unsafe", "unsupported" etc, it's hard to find a name). The idea here
is to convey the idea that if you don't like it it's rather because
of the limited support than the protocol itself, while allowing users
to start to discover it.
I don't know how this will evolve for 2.9, maybe it will be refined,
reworked, reverted or improved, I don't know. Also, the NGINX team
said they're currently working on trying to get 0-RTT to work, so
the long-term question becomes even more justified.
Last point, in case they'd be any doubt about the intent to find an
alternate SSL library, let me remind everyone that OpenSSL 3.x's
disastrous performance is a much bigger problem than its lack of QUIC
support, and that it's not just a QUIC patchset that will fix it, so
the work on an alternative is not going to fade away.
And that's roughly all for this version.
I'm aware of some progress being made on various topics at the moment,
hopefully we'll get some of them in dev3 or dev4:
- a number of log-format tags are receiving an equivalent sample-fetch
so that it will finally be possible to exploit such metrics differently
if needed. Many are still missing, that's a long task due to some
specific formats.
- the mux-to-mux forwarding that takes care of congestion window to
avoid filling buffers with blocked data is progressing.
- the mechanism to automatically bind threads and create thread-groups
based on CPU topology is making progress as well, hopefully by next
-dev we'll be able to see optimal binding on multi-socket machines,
segmented caches and hybrid CPUs.
- the patch series to reduce the locking contention on stick-tables
should be merged soon
- some work was finally started to reduce the locking cost in the
shctx blocks used by the cache.
I don't have the rest in mind at the moment.
Please find the usual URLs below :
Site index : https://www.haproxy.org/
Documentation : https://docs.haproxy.org/
Wiki : https://github.com/haproxy/wiki/wiki
Discourse : https://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Sources : https://www.haproxy.org/download/2.9/src/
Git repository : https://git.haproxy.org/git/haproxy.git/
Git Web browsing : https://git.haproxy.org/?p=haproxy.git
Changelog : https://www.haproxy.org/download/2.9/src/CHANGELOG
Dataplane API :
https://github.com/haproxytech/dataplaneapi/releases/latest
Pending bugs : https://www.haproxy.org/l/pending-bugs
Reviewed bugs : https://www.haproxy.org/l/reviewed-bugs
Code reports : https://www.haproxy.org/l/code-reports
Latest builds : https://www.haproxy.org/l/dev-packages
Willy
---
Complete changelog :
Aurelien DARRAGON (27):
BUG/MINOR: tcp_sample: bc_{dst,src} return IP not INT
MEDIUM: acl/sample: unify sample conv parsing in a single function
MINOR: sample: introduce c_pseudo() conv function
MEDIUM: sample: add missing ADDR=>? compatibility matrix entries
MINOR: sample: fix ipmask sample definition
MEDIUM: tree-wide: fetches that may return IPV4+IPV6 now return ADDR
MEDIUM: sample: introduce 'same' output type
BUG/MEDIUM: sink: invalid server list in sink_new_from_logsrv()
BUG/MINOR: http_ext: unhandled ERR_ABORT in proxy_http_parse_7239()
BUG/MINOR: sink: missing sft free in sink_deinit()
BUG/MINOR: ring: size warning incorrectly reported as fatal error
BUG/MINOR: ring: maxlen warning reported as alert
BUG/MINOR: log: LF upsets maxlen for UDP targets
MINOR: sink/api: pass explicit maxlen parameter to sink_write()
BUG/MEDIUM: log: improper use of logsrv->maxlen for buffer targets
BUG/MINOR: log: fix missing name error message in cfg_parse_log_forward()
BUG/MINOR: log: fix multiple error paths in cfg_parse_log_forward()
BUG/MINOR: log: free errmsg on error in cfg_parse_log_forward()
BUG/MINOR: sink: invalid sft free in sink_deinit()
BUG/MINOR: sink: fix errors handling in cfg_post_parse_ring()
BUG/MINOR: server: set rid default value in new_server()
MINOR: hlua_fcn/mailers: handle timeout mail from mailers section
BUG/MINOR: sink/log: properly deinit srv in sink_new_from_logsrv()
EXAMPLES: maintain haproxy 2.8 retrocompatibility for lua mailers script
BUG/MINOR: hlua_fcn/queue: use atomic load to fetch queue size
BUG/MINOR: hlua: hlua_yieldk ctx argument should support pointers
BUG/MEDIUM: hlua_fcn/queue: bad pop_wait sequencing
Christopher Faulet (6):
DOC: config: Fix fc_src description to state the source address is
returned
BUG/MINOR: sample: Fix wrong overflow detection in add/sub conveters
BUG/MINOR: http: Return the right reason for 302
BUG/MINOR: h1-htx: Return the right reason for 302 FCGI responses
BUG/MINOR: server: Don't warn on server resolution failure with init-addr
none
BUG/MEDIUM: listener: Acquire proxy's lock in relax_listener() if
necessary
Daan van Gorkum (1):
MINOR: lua: Allow reading "proc." scoped vars from LUA core.
Emeric Brun (6):
BUG/MEDIUM: quic: token IV was not computed using a strong secret
BUG/MINOR: quic: retry token remove one useless intermediate expand
BUG/MEDIUM: quic: missing check of dcid for init pkt including a token
BUG/MEDIUM: quic: timestamp shared in token was using internal time clock
CLEANUP: quic: remove useless parameter 'key' from quic_packet_encrypt
BUILD: quic: fix warning during compilation using gcc-6.5
Frédéric Lécaille (33):
BUG/MINOR: quic: Possible leak when allocating an encryption level
BUG/MINOR: quic: Missing QUIC connection path member initialization
BUILD: quic: Compilation fixes for some gcc warnings with -O1
BUG/MINOR: quic: Possible crash in "show quic" dumping packet number
spaces
BUG/MINOR: quic: Unckecked encryption levels availability
MINOR: quic: Stop storing the TX encoded transport parameters
MINOR: quic: Dynamic allocation for negotiated Initial TLS cipher context.
MINOR: quic: Release asap the negotiated Initial TLS context.
MINOR: quic: Add traces to qc_may_build_pkt()
MEDIUM: quic: Packet building rework.
CLEANUP: quic: Remove a useless TLS related variable from
quic_conn_io_cb().
MEDIUM: quic: Handshake I/O handler rework.
MINOR: quic: Add traces for qc_frm_free()
MINOR: quic: add trace about pktns packet/frames releasing
BUG/MINOR: quic: Missing parentheses around PTO probe variable.
MINOR: quic: Ping from Initial pktns before reaching anti-amplification
limit
MINOR: quic: QUIC openssl wrapper implementation
MINOR: quic: Include QUIC opensssl wrapper header from TLS stacks
compatibility header
MINOR: quic: Do not enable O-RTT with USE_QUIC_OPENSSL_COMPAT
MINOR: quic: Set the QUIC connection as extra data before calling
SSL_set_quic_method()
MINOR: quic: Do not enable 0RTT with SSL_set_quic_early_data_enabled()
MINOR: quic: Add a compilation option for the QUIC OpenSSL wrapper
MINOR: quic: Export some KDF functions (QUIC-TLS)
MINOR: quic: Make ->set_encryption_secrets() be callable two times
MINOR: quic: Initialize TLS contexts for QUIC openssl wrapper
MINOR: quic: Call the keylog callback for QUIC openssl wrapper from
SSL_CTX_keylog()
MINOR: quic: Add a quic_openssl_compat struct to quic_conn struct
MINOR: quic: Useless call to SSL_CTX_set_quic_method()
MINOR: quic: SSL context initialization with QUIC OpenSSL wrapper.
MINOR: quic: Missing encoded transport parameters for QUIC OpenSSL wrapper
MINOR: quic: Add "limited-quic" new tuning setting
DOC: quic: Add "limited-quic" new tuning setting
DOC: install: Document how to build a limited support for QUIC
Ilya Shipitsin (2):
CI: add naming convention documentation
CI: explicitely highlight VTest result section if there's something
Marcos de Oliveira (2):
BUG/MINOR: server-state: Ignore empty files
BUG/MINOR: server-state: Avoid warning on 'file not found'
Mariam John (1):
MEDIUM: ssl: new sample fetch method to get curve name
Patrick Hemmer (1):
MINOR: peers: add peers keyword registration
Remi Tricot-Le Breton (3):
DOC: ssl: Fix typo in 'ocsp-update' option
DOC: ssl: Add ocsp-update troubleshooting clues and emphasize on crt-list
only aspect
BUG/MINOR: cache: A 'max-age=0' cache-control directive can be overriden
by a s-maxage
Thierry Fournier (2):
BUG/MINOR: config: Remove final '\n' in error messages
BUG/MINOR: config: Lenient port configuration parsing
Willy Tarreau (6):
MINOR: cpuset: add cpu_map_configured() to know if a cpu-map was found
BUG/MINOR: config: do not detect NUMA topology when cpu-map is configured
BUG/MINOR: cpuset: remove the bogus "proc" from the cpu_map struct
BUG/MINOR: init: set process' affinity even in foreground
CLEANUP: cpuset: remove the unused proc_t1 field in cpu_map
CLEANUP: config: make parse_cpu_set() return documented values
firexinghe (1):
BUG/MINOR: hlua: add check for lua_newstate
---
