Hi,
HAProxy 2.8-dev7 was released on 2023/04/08. It added 147 new commits
after version 2.8-dev6.
Freezing features has good virtues. This version attracted a number of
pending commits that will need a bit of testing before the release. Among
the changes:
- Christopher could finally merge a significant part of his rework of
the status flags that moved from the channel to the stream connector.
This was an important change that we've been hoping to do since 2.5
or so, that will greatly help avoid some of the obscure bugs that are
occasionally reported after a minor change that causes never-ending
streams or ignored aborts. It may possibly bring a bug or two caused
by mistakes during the conversion, and it may possibly wake up some
such older bugs, though the risk is quite low and the patches were
carefully reviewed. But testing and reports are welcome. There's a
second part to this that we're still wondering whether a part can
still be merged or not, we'll see soon.
- Olivier implemented support for compressing requests. While it can
sound surprising at first, it's very useful when the front LB nodes
are selected by DNS and need to route large requests to a remote
server that is reached over a limited link or when outgoing traffic
is expensive.
- Aurélien merged his event handler API. This is publish/subscribe
mechanism used to broadcast notifications about state changes to
internal subsystems. For now, only server state change is supported,
and such events may be subscribed to from Lua. This means that some
Lua code can be notified about added/removed/up/down servers. A mid-
term goal will be to reimplement the mailers entirely in Lua, allowing
unlimited customizing.
- Aurélien addressed a CPU usage issue when stopping; pool_gc() was
called too often, resulting in stick-tables possibly consuming a lot
of CPU during such periods, which was even worse when malloc_trim()
was called to free memory as this call involves locking inside the
libc, hence severe contention.
- Thierry's changes to make Lua support dynamic servers were finally
merged. In the past, Lua could take a long time to start when dealing
with very large configs as it was learning the servers, and wouldn't
be aware of servers added on the fly. This is no more a limitation.
- Fred and Amaury improved the QUIC stack in various areas: less memory
consumption by removing one tree lookup, fixed some RTT calculations
and corner cases in congestion control algorithms, added more info to
"show quic", added a dummy congestion control algorithm with a fixed
window that may help us diagnose some issues in field.
- William managed to improve acme.sh to directly manage haproxy at
runtime via the CLI, and to start a config without certificates.
This means that once his pull request will be merged, Let's Encrypt
support will be totally seamless. In case it wouldn't be merged (there
are many PRs pending there), we'd find a different way to host this
change, either by integrating it as a "plugin" with haproxy, or by
forking the repository and providing it under the haproxy github
organization. More info in the PR here:
https://github.com/acmesh-official/acme.sh/pull/4581
- Martin, our intern for one week, implemented the support for case
insensitive argument match in urlp/url_param (issue 1633). It was a
one year-old feature request that we hoped for 2.6. Not bad at all
for a first contribution ;-)
- the rest is essentially bug fixes, doc and cleanups.
Regarding the pending stuff I'm aware of at the moment:
- Christopher still has changes aiming at making applets much more
reliable by no more stepping over the stream's feet. It's really
desired but possibly risky at the same time. We'll check this
together next week, and possibly take some of it, or maybe insert
just a smaller step towards it in order to ease backports of post-2.9
fixes to 2.8 later.
- Amaury managed to make the QUIC connections behave exactly like
the TCP connections at the lower layers, in that they are migrable
to other threads, giving back the control to listener_accept() like
for TCP. It's only a matter of writing commit messages now.
- My changes to make bind lines support multiple thread groups are
now working, but I was blocked on the QUIC part, which is no more
a problem after Amaury's change above. I've rebased my code on top
of his and will merge it once it's in.
- Aurélien managed to add a working timeout for Lua instructions
that are not yieldable (i.e. busy loops called from sample fetch
functions). The patches were reviewed yesterday afternoon and
will be merged early next week after some cosmetic updates.
And with all this done we'll have to focus more on testing, debugging,
documentation, cleanups and small updates.
I must confess that yesterday I felt like we reached an intersting
milestone with several long-term tasks finally seeing the light at the
end of the tunnel!
As usual, test it, beat it and report any problem!
Have a nice week-end; it's sunny here so I should be silent now ;-)
Willy
Please find the usual URLs below :
Site index : https://www.haproxy.org/
Documentation : https://docs.haproxy.org/
Wiki : https://github.com/haproxy/wiki/wiki
Discourse : https://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Sources : https://www.haproxy.org/download/2.8/src/
Git repository : https://git.haproxy.org/git/haproxy.git/
Git Web browsing : https://git.haproxy.org/?p=haproxy.git
Changelog : https://www.haproxy.org/download/2.8/src/CHANGELOG
Dataplane API :
https://github.com/haproxytech/dataplaneapi/releases/latest
Pending bugs : https://www.haproxy.org/l/pending-bugs
Reviewed bugs : https://www.haproxy.org/l/reviewed-bugs
Code reports : https://www.haproxy.org/l/code-reports
Latest builds : https://www.haproxy.org/l/dev-packages
Willy
---
Complete changelog :
Amaury Denoyelle (3):
MINOR: quic: derive first DCID from client ODCID
MINOR: quic: remove ODCID dedicated tree
MINOR: quic: remove address concatenation to ODCID
Aurelien DARRAGON (44):
MINOR: proxy/pool: prevent unnecessary calls to pool_gc()
CLEANUP: proxy: remove stop_time related dead code
BUG/MEDIUM: proxy/sktable: prevent watchdog trigger on soft-stop
BUG/MINOR: backend: make be_usable_srv() consistent when stopping
MINOR: server: add SRV_F_DELETED flag
BUG/MINOR: server/del: fix srv->next pointer consistency
BUG/MINOR: stats: properly handle server stats dumping resumption
BUG/MINOR: sink: free forward_px on deinit()
BUG/MINOR: log: free log forward proxies on deinit()
MINOR: server: always call ssl->destroy_srv when available
MINOR: server: correctly free servers on deinit()
BUG/MINOR: hlua: hook yield does not behave as expected
MINOR: hlua: properly handle hlua_process_task HLUA_E_ETMOUT
BUG/MINOR: hlua: enforce proper running context for register_x functions
MEDIUM: hlua_fcn/api: remove some old server and proxy attributes
CLEANUP: hlua: fix conflicting comment in hlua_ctx_destroy()
MINOR: hlua: add simple hlua reference handling API
MINOR: hlua: fix return type for hlua_checkfunction() and
hlua_checktable()
BUG/MINOR: hlua: fix reference leak in core.register_task()
BUG/MINOR: hlua: fix reference leak in hlua_post_init_state()
BUG/MINOR: hlua: prevent function and table reference leaks on errors
CLEANUP: hlua: use hlua_ref() instead of luaL_ref()
CLEANUP: hlua: use hlua_pushref() instead of lua_rawgeti()
CLEANUP: hlua: use hlua_unref() instead of luaL_unref()
MINOR: hlua: simplify lua locking
BUG/MEDIUM: hlua: prevent deadlocks with main lua lock
MINOR: hlua_fcn: add server->get_rid() method
MINOR: hlua: support for optional arguments to core.register_task()
DOC: lua: silence "literal block ends without a blank line" Sphinx
warnings
DOC: lua: silence "Unexpected indentation" Sphinx warnings
BUG/MINOR: event_hdl: fix rid storage type
BUG/MINOR: event_hdl: make event_hdl_subscribe thread-safe
MINOR: event_hdl: global sublist management clarification
BUG/MEDIUM: event_hdl: clean soft-stop handling
BUG/MEDIUM: event_hdl: fix async data refcount issue
MINOR: event_hdl: normal tasks support for advanced async mode
MINOR: event_hdl: add event_hdl_async_equeue_isempty() function
MINOR: event_hdl: add event_hdl_async_equeue_size() function
MINOR: event_hdl: pause/resume for subscriptions
MINOR: proxy: add findserver_unique_id() and findserver_unique_name()
MEDIUM: hlua/event_hdl: initial support for event handlers
MINOR: hlua/event_hdl: per-server event subscription
EXAMPLES: add basic event_hdl lua example script
BUG/MINOR: errors: invalid use of memprintf in startup_logs_init()
Christopher Faulet (44):
BUG/MINOR: stats: Don't replace sc_shutr() by SE_FL_EOS flag yet
BUG/MEDIUM: mux-h2: Be able to detect connection error during handshake
BUG/MINOR: tcpcheck: Be able to expect an empty response
BUG/MEDIUM: stconn: Add a missing return statement in sc_app_shutr()
BUG/MINOR: stream: Fix test on channels flags to set clientfin/serverfin
touts
MINOR: applet: Uninline appctx_free()
MEDIUM: applet/trace: Register a new trace source with its events
CLEANUP: stconn: Remove remaining debug messages
BUG/MEDIUM: channel: Improve reports for shut in co_getblk()
BUG/MEDIUM: dns: Properly handle error when a response consumed
MINOR: stconn: Remove unecessary test on SE_FL_EOS before receiving data
MINOR: stconn/channel: Move CF_READ_DONTWAIT into the SC and rename it
MINOR: stconn/channel: Move CF_SEND_DONTWAIT into the SC and rename it
MINOR: stconn/channel: Move CF_NEVER_WAIT into the SC and rename it
MINOR: stconn/channel: Move CF_EXPECT_MORE into the SC and rename it
MINOR: mux-pt: Report end-of-input with the end-of-stream after a read
BUG/MINOR: mux-h1: Properly report EOI/ERROR on read0 in h1_rcv_pipe()
CLEANUP: mux-h1/mux-pt: Remove useless test on SE_FL_SHR/SE_FL_SHW flags
MINOR: mux-h1: Report an error to the SE descriptor on truncated message
MINOR: stconn: Always ack EOS at the end of sc_conn_recv()
MINOR: stconn/applet: Handle EOI in the applet .wake callback function
MINOR: applet: No longer set EOI on the SC
MINOR: stconn/applet: Handle EOS in the applet .wake callback function
MEDIUM: cache: Use the sedesc to report and detect end of processing
MEDIUM: cli: Use the sedesc to report and detect end of processing
MINOR: dns: Remove the test on the opposite SC state to send requests
MEDIUM: dns: Use the sedesc to report and detect end of processing
MEDIUM: spoe: Use the sedesc to report and detect end of processing
MEDIUM: hlua/applet: Use the sedesc to report and detect end of processing
MEDIUM: log: Use the sedesc to report and detect end of processing
MEDIUM: peers: Use the sedesc to report and detect end of processing
MINOR: sink: Remove the tests on the opposite SC state to process messages
MEDIUM: sink: Use the sedesc to report and detect end of processing
MEDIUM: stats: Use the sedesc to report and detect end of processing
MEDIUM: promex: Use the sedesc to report and detect end of processing
MEDIUM: http_client: Use the sedesc to report and detect end of processing
MINOR: stconn/channel: Move CF_EOI into the SC and rename it
MEDIUM: tree-wide: Move flags about shut from the channel to the SC
MINOR: tree-wide: Simplifiy some tests on SHUT flags by accessing SCs
directly
MINOR: stconn/applet: Add BUG_ON_HOT() to be sure SE_FL_EOS is never set
alone
MINOR: http-ana: Add a HTTP_MSGF flag to state the Expect header was
checked
BUG/MINOR: http-ana: Don't switch message to DATA when waiting for payload
MINOR: applet: Use unsafe version to get stream from SC in the trace
function
BUG/MUNOR: http-ana: Use an unsigned integer for http_msg flags
Frédéric Lécaille (22):
BUG/MINOR: quic: Missing padding in very short probe packets
BUG/MINOR: quic: Wrong use of now_ms timestamps (cubic algo)
MINOR: quic: Add recovery related information to "show quic"
BUG/MINOR: quic: Wrong use of now_ms timestamps (newreno algo)
BUG/MINOR: quic: Missing max_idle_timeout initialization for the
connection
MINOR: quic: Implement cubic state trace callback
MINOR: quic: Adjustments for generic control congestion traces
MINOR: quic: Traces adjustments at proto level.
MEDIUM: quic: Ack delay implementation
BUG/MINOR: quic: Wrong rtt variance computing
MINOR: quic: Add a fake congestion control algorithm named "nocc"
BUG/MINOR: quic: Remaining useless statements in cubic slow start callback
BUG/MINOR: quic: Cubic congestion control window may wrap
MINOR: quic: Add missing traces in cubic algorithm implementation
BUG/MAJOR: quic: Congestion algorithms states shared between the
connection
BUG/MINOR: quic: Remove useless BUG_ON() in newreno and cubic algo
implementation
MINOR: quic: Add trace to debug idle timer task issues
BUG/MINOR: quic: Unexpected connection closures upon idle timer task
execution
BUG/MINOR: quic: Wrong idle timer expiration (during 20s)
BUILD: quic: 32bits compilation issue in cli_io_handler_dump_quic()
BUG/MINOR: quic: Possible wrong PTO computing
BUG/MINOR: quic: Possible crashes in qc_idle_timer_task()
Ilya Shipitsin (3):
CI: run smoke tests on config syntax to check memory related issues
CLEANUP: assorted typo fixes in the code and comments
CI: exclude doc/{design-thoughts,internals} from spell check
Marcos de Oliveira (1):
DOC/MINOR: reformat configuration.txt's "quoting and escaping" table
Martin DOLEZ (4):
MINOR: http_fetch: Add support for empty delim in url_param
MINOR: http_fetch: add case insensitive support for smp_fetch_url_param
MINOR: http_fetch: Add case-insensitive argument for url_param/urlp_val
REGTESTS : Add test support for case insentitive for url_param
Olivier Houchard (6):
MINOR: compression: Make compression offload a flag
MINOR: compression: Prepare compression code for request compression
MINOR: compression: Store algo and type for both request and response
MINOR: compression: Count separately request and response compression
MEDIUM: compression: Make it so we can compress requests as well.
BUG/MEDIUM: listeners: Use the right parameters for strlcpy2().
Remi Tricot-Le Breton (4):
BUG/MINOR: ssl: Remove dead code in cli_parse_update_ocsp_response
BUG/MINOR: ssl: Fix potential leak in cli_parse_update_ocsp_response
BUG/MINOR: ssl: ssl-(min|max)-ver parameter not duplicated for bundles in
crt-list
BUG/MINOR: ssl: Undefined reference when building with
OPENSSL_NO_DEPRECATED
Thierry Fournier (4):
MINOR: hlua: Fix two functions that return nothing useful
MEDIUM: hlua: Dynamic list of frontend/backend in Lua
MINOR: hlua_fcn: alternative to old proxy and server attributes
MEDIUM: hlua_fcn: dynamic server iteration and indexing
William Lallemand (2):
DOC: config: strict-sni allows to start without certificate
BUG/MINOR: mworker: unset more internal variables from program section
Willy Tarreau (10):
MINOR: cli: support filtering on FD types in "show fd"
MINOR: http-act: emit a warning when a header field name contains
forbidden chars
BUG/MINOR: lua: remove incorrect usage of strncat()
CLEANUP: tcpcheck: remove the only occurrence of sprintf() in the code
CLEANUP: ocsp: do no use strpcy() to copy a path!
CLEANUP: tree-wide: remove strpcy() from constant strings
CLEANUP: opentracing: remove the last two occurrences of strncat()
BUILD: compiler: fix __equals_1() on older compilers
MINOR: compiler: define a __attribute__warning() macro
BUILD: bug.h: add a warning in the base API when unsafe functions are used
---
