Hi,
HAProxy 2.4-dev7 was released on 2021/02/05. It added 153 new commits
after version 2.4-dev6.
This version is quite large for two reasons, first one being that I wanted
to issue it last week but got stuck on an issue that I didn't imagine would
take my week-end and week, and second one because some changes were upheld
by other ones that had to be reworked several times. But eventually all of
this got sorted out and could be merged.
No less than 29 bugs were addressed this time, showing that some significant
activity remains on stabilization, and comforts me in the decision of closing
the merge window at the end of next week so that all those which remain have
the time to be sorted out for the release.
The main updates this time are in the following areas:
- TCP to HTTP upgrades: these ones were reportedly broken on the last few
-dev versions (since dev3 or dev4, I don't remember) and have now been
addressed.
- HTX: end of the migration to remove the special EOM (end of message)
block that was causing some difficulties, particularly when CONNECT
was used in H2, but was also maintaining a certain level of complexity
in various parts of the muxes. There is a non-null risk of corner case
issues here, though it has run through extensive testing and even been
deployed in production and it looks like everything's fine by now. If
you observe anything strange such as hanging requests or responses, or
breakage with tunnels (you shouldn't), please report them.
- Websocket: the WebSocket translation from H2 to H1 is now fully supported
(RFC8441) as well as H1 to H2. The Extended CONNECT support is also
advertised so that compatible clients will not need to set up a second
H1 connection to use websocket anymore.
- SSL: server-side certificates can now be updated at run time from the
CLI.
- Some significant lifting was done to the Prometheus exporter, including
new fields, better descriptions and some filtering. I've seen quite a
bunch pass in front of me but do not well understand what it does, all
that interests me is that some users are happy with these changes so I
guess they were long awaited :-)
- health-check/agent-check: some old code that used to be there to force
the address/port from the server's address while parsing the config had
some long-tail effects on various parts of the code, making it a nightmare
to update them dynamically or even from state files. And of course, this
was just an antique relic not needed anymore some cleaning was done there
(mostly unused code removal), and will help further consolidation in this
area in the near future.
- HTTP: make sure to reject non-compliant status-101 upgrades. We used not
to check for the upgrade header for example but better stay safe and closer
to the standard here as we're playing with tunnels. The 101 status code is
now also rejected internally and externally on any H2 response.
- The place L7 retries are dealt with changed slightly from the connection
error handler to an analyser. This must not cause any difference in practice
except make the code more maintainable and robust. Similarly if you're using
them and notice a change of behavior, please raise your hand.
- now we make sure never to emit any payload for bodyless responses (204,
304, HEAD). In the past such contents could have been produced from inside
haproxy (Lua for example, maybe http-request return), now the contents will
be discarded if ever present.
- debug: "show fd" now reports a bit more information such as the number of
calls to the registered tasks (useful to detect never-ending loops), local
and report ports (useful to match against netstat), a few more info about
the internal handlers and protocol, and an indicator of suspicious state
(e.g. too many calls, or inconsistent state). This will help bug reporters
to quickly isolate a few candidates when something looks odd.
- debug/monitoring: "show profiling" will now report the CPU calls, %usage,
and latency of each running task since the last time profiling was
turned on. "show tasks" will enumerate the currently running tasks and
their counts. Do not abuse by bots, this is expensive. Finally, crashes
provoked by BUG_ON() statements in DEBUG_STRICT mode will now dump a call
trace in addition to the error message, hopefully helping developers spot
certains issues easier.
- HTTP: chunk size used to be limited to 2 GB because we used to rely on
31-bit integers at a time where nobody else would support larger values
either. This was extended to 4 PB to satisfy some extremely rare but
existing use cases.
And a usual, new regtests (123 running on my machine before any push here),
significant cleanups all over the code, doc and CI improvements. I certainly
have not done justice to those having worked hard on certain parts that
were summed up as a single sentence, but it's also a proof that things are
advancing fast if it's becoming increasingly difficult so say long words
about each of them. Fortunately, contrary to Linus, I can still always
append the short log at the end of my announces for those who want the
details :-)
I'm going to issue 2.3 and 2.2 very soon as well (ideally this evening) to
flush the pipe of pending fixes. Tim reminded me that Debian's last call
for updates before the next release is next week, so I find it important
to have a clean 2.2.9 that users can start with. Also I know that I have
to issue a last 1.6 and close it. I think that in the future I should plan
this for after the feature freeze, it will be easier. This one will wait
two other weeks at least I guess.
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse : http://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Wiki : https://github.com/haproxy/wiki/wiki
Sources : http://www.haproxy.org/download/2.4/src/
Git repository : http://git.haproxy.org/git/haproxy.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy.git
Changelog : http://www.haproxy.org/download/2.4/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
Willy
---
Complete changelog :
Amaury Denoyelle (20):
BUG/MINOR: config: fix leak on proxy.conn_src.bind_hdr_name
MINOR: reg-tests: add http-reuse test
CLEANUP: srv: fix comment for pool-max-conn
CLEANUP: backend: remove an obsolete comment on conn_backend_get
REORG: backend: simplify conn_backend_get
BUG/MEDIUM: session: only retrieve ready idle conn from session
BUG/MEDIUM: backend: never reuse a connection for tcp mode
MINOR: h1: reject websocket handshake if missing key
MEDIUM: h1: generate WebSocket key on response if needed
MINOR: mux_h2: define H2_SF_EXT_CONNECT_SENT stream flag
MEDIUM: h2: parse Extended CONNECT reponse to htx
MEDIUM: mux_h2: generate Extended CONNECT from htx upgrade
MEDIUM: h1: add a WebSocket key on handshake if needed
MEDIUM: mux_h2: generate Extended CONNECT response
MEDIUM: h2: parse Extended CONNECT request to htx
MEDIUM: h2: send connect protocol h2 settings
MINOR: vtc: add test for h1/h2 protocol upgrade translation
MINOR: vtc: add websocket test
BUG/MINOR: backend: check available list allocation for reuse
BUG/MINOR: mux_h2: fix incorrect stat titles
Christopher Faulet (56):
BUG/MINOR: stats: Continue to fill frontend stats on unimplemented metric
BUG/MINOR: stats: Init the metric variable when frontend stats are filled
BUG/MEDIUM: filters/htx: Fix data forwarding when payload length is
unknown
BUG/MINOR: stats: Remove a break preventing ST_F_QCUR to be set for
servers
BUG/MINOR: stats: Add a break after filling ST_F_MODE field for servers
MEDIUM: stream-int: Take care of EOS if the SI wake callback function
MINOR: mux-h1: Try to wake up data layer first before calling its wake
callback
MINOR: mux-h1: Wake up H1C after its creation if input buffer is not empty
MEDIUM: mux-h1: Add ST_READY state for the H1 connections
MINOR: stream: Add a function to validate TCP to H1 upgrades
MEDIUM: http-ana: Do nothing in wait-for-request analyzer if not htx
BUG/MEDIUM: stream: Don't immediatly ack the TCP to H1 upgrades
BUG/MAJOR: mux-h1: Properly handle TCP to H1 upgrades
MINOR: htx/http-ana: Save info about Upgrade option in the Connection
header
MEDIUM: http-ana: Refuse invalid 101-switching-protocols responses
BUG/MINOR: h2/mux-h2: Reject 101 responses with a PROTOCOL_ERROR h2s error
MINOR: mux-h1/mux-fcgi: Don't set TUNNEL mode if payload length is unknown
MINOR: mux-h1: Split H1C_F_WAIT_OPPOSITE flag to separate input/output
sides
MINOR: mux-h2: Add 2 flags to help to properly handle tunnel mode
MEDIUM: mux-h2: Block client data on server side waiting tunnel
establishment
MEDIUM: mux-h2: Close streams when processing data for an aborted tunnel
MEDIUM: mux-h1: Properly handle tunnel establishments and aborts
BUG/MAJOR: mux-h1/mux-h2/htx: Fix HTTP tunnel management at the mux level
MINOR: htx: Rename HTX_FL_EOI flag into HTX_FL_EOM
REGTESTS: Don't run http_msg_full_on_eom script on the 2.4 anymore
MINOR: htx: Add a function to know if a block is the only one in a message
MAJOR: htx: Remove the EOM block type and use HTX_FL_EOM instead
MINOR: mux-h1: Add a flag on H1 streams with a response known to be
bodyless
MEDIUM: mux-h1: Don't emit any payload for bodyless responses
MINOR: mux-h1: Don't emit C-L and T-E headers for 204 and 1xx responses
MINOR: mux-h1: Don't add Connection close/keep-alive header for 1xx
messages
MINOR: h2/mux-h2: Add flags to notify the response is known to have no
body
MEDIUM: mux-h2: Don't emit DATA frame for bodyless responses
MEDIUM: http-ana: Deal with L7 retries in HTTP analysers
REGTESTS: Fix required versions for several scripts
REGTEST: Don't use the websocket to validate http-check
MINOR: mux-h1/trace: add traces at level ERROR for all kind of errors
MINOR: mux-fcgi/trace: add traces at level ERROR for all kind of errors
MINOR: h1: Raise the chunk size limit up to (2^52 - 1)
MINOR: mux-h1: Remove first useless test on count in h1_process_output()
BUG/MINOR: stick-table: Always call smp_fetch_src() with a valid arg list
MINOR: http-fetch: Don't check if argument list is set in sample fetches
MINOR: http-conv: Don't check if argument list is set in sample converters
MINOR: sample: Don't check if argument list is set in sample fetches
MINOR: ssl-sample: Don't check if argument list is set in sample fetches
MINOR: mux-h2: Don't tests the start-line when sending HEADERS frame
MINOR: mux-h2: Slightly improve request HEADERS frames sending
MEDIUM: contrib/prometheus-exporter: Use dynamic labels instead of static
ones
MINOR: checks: Add function to get the result code corresponding to a
status
DOC: contrib/prometheus-exporter: Add missing metrics in README
BUG/MINOR: contrib/prometheus-exporter: Add missing label for
ST_F_HRSP_1XX
BUG/MINOR: contrib/prometheus-exporter: Restart labels dump at the right
pos
MINOR: server: Don't set the check port during the update from a state
file
MINOR: dns: Don't set the check port during a server dns resolution
DOC: server: Add missing params in comment of the server state line
parsing
CLEANUP: http-htx: Set buffer area to NULL instead of malloc(0)
Ilya Shipitsin (2):
BUILD: ssl: guard Client Hello callbacks with HAVE_SSL_CLIENT_HELLO_CB
macro instead of openssl version
CLEANUP: assorted typo fixes in the code and comments
Remi Tricot-Le Breton (6):
MINOR: ssl: Server ssl context prepare function refactoring
MINOR: ssl: Certificate chain loading refactorization
MEDIUM: ssl: Load client certificates in a ckch for backend servers
MEDIUM: ssl: Enable backend certificate hot update
MINOR: ssl: Remove client_crt member of the server's ssl context
BUG/MINOR: sock: Unclosed fd in case of connection allocation failure
Tim Duesterhus (5):
DOC: Improve documentation of the various hdr() fetches
MINOR: abort() on my_unreachable() when DEBUG_USE_ABORT is set.
BUILD: Include stdlib.h in compiler.h if DEBUG_USE_ABORT is set
CI: Fix DEBUG_STRICT definition for Coverity
CI: Fix the coverity builds
William Dauchy (23):
MINOR: contrib/prometheus-exporter: better output of Not-a-Number
CLEANUP: stats: improve field selection for frontend http fields
MEDIUM: stats: allow to select one field in `stats_fill_be_stats`
MINOR: contrib/prometheus-exporter: use fill_be_stats for backend dump
MEDIUM: stats: allow to select one field in `stats_fill_sv_stats`
MINOR: contrib/prometheus-exporter: use fill_sv_stats for server dump
MINOR: contrib/prometheus-exporter: declare states for objects
MAJOR: contrib/prometheus-exporter: move ftd/bkd/srv states to labels
MAJOR: contrib/prometheus-exporter: move health check status to labels
MINOR: contrib/prometheus-exporter: improve service status description
field
MINOR: stats: improve pending connections description
MINOR: stats: improve max stats descriptions
MINOR: contrib/prometheus-exporter: use stats desc when possible
MINOR: contrib/prometheus-exporter: add uweight field
MINOR: contrib/prometheus-exporter: add recv logs_logs_total field
CLEANUP: contrib/prometheus-exporter: remove unused includes
CLEANUP: contrib/prometheus-exporter: align and reorder fields
CLEANUP: contrib/prometheus-exporter: remove description in README
BUG/MINOR: cli: fix set server addr/port coherency with health checks
MEDIUM: check: remove checkport checkaddr flag
MEDIUM: server: adding support for check_port in server state
BUG/MINOR: check: consitent way to set agentaddr
MEDIUM: check: align agentaddr and agentport behaviour
William Lallemand (14):
CLEANUP: ssl/cli: rework free in cli_io_handler_commit_cert()
CLEANUP: ssl: remove SSL_CTX function parameter
CLEANUP: ssl: make load_srv_{ckchs,cert} match their bind counterpart
CLEANUP: ssl: remove dead code in ckch_inst_new_load_srv_store()
BUG/MINOR: ssl: init tmp chunk correctly in ssl_sock_load_sctl_from_file()
REGTESTS: set_ssl_server_cert.vtc: remove the abort command
REGTESTS: set_ssl_server_cert.vtc: check the Sha1 Fingerprint
REGTESTS: set_ssl_server_cert.vtc: check the sha1 from the server
REGTESTS: set_ssl_server_cert.vtc: set as broken
REGTESTS: set_ssl_server_cert.vtc: remove SSL caching and set as working
REGTESTS: set_ssl_server_cert: cleanup the SSL caching option
BUG/MEDIUM: ssl/cli: abort ssl cert is freeing the old store
MINOR: ssl/cli: flush the server session cache upon 'commit ssl cert'
BUILD: Makefile: move REGTESTST_TYPE default setting
Willy Tarreau (27):
BUG/MEDIUM: listener: do not accept connections faster than we can
process them
Revert "BUG/MEDIUM: listener: do not accept connections faster than we
can process them"
DOC: management: fix "show resolvers" alphabetical ordering
MINOR: tools: add print_time_short() to print a condensed duration value
MINOR: activity: make profiling more manageable
MINOR: activity: declare a new structure to collect per-function activity
MEDIUM: tasks/activity: collect per-task statistics when profiling is
enabled
MINOR: activity: also report collected tasks stats in "show profiling"
MINOR: activity: flush scheduler stats on "set profiling tasks on"
MINOR: activity: add a new "show tasks" command to list currently active
tasks
MINOR: listener: export accept_queue_process
MINOR: session: export session_expire_embryonic()
MINOR: muxes: export the timeout and shutr task handlers
MINOR: checks: export a few functions that appear often in trace dumps
MINOR: peers: export process_peer_sync() to improve traces
MINOR: stick-tables: export process_table_expire()
MINOR: listener: export manage_global_listener_queue()
BUG/MINOR: activity: take care of late wakeups in "show tasks"
BUG/MEDIUM: ssl: check a connection's status before computing a handshake
BUG/MINOR: xxhash: make sure armv6 uses memcpy()
REGTESTS: mark http-check-send.vtc as 2.4-only
REGTESTS: mark sample_fetches/hashes.vtc as 2.4-only
BUG/MINOR: ssl: do not try to use early data if not configured
REGTESTS: unbreak http-check-send.vtc
MINOR: cli/show_fd: report local and report ports when known
BUG/MEDIUM: mux-h2: handle remaining read0 cases
BUG/MEDIUM: mux-h2: do not quit the demux loop before setting END_REACHED
---
