Hi,
HAProxy 2.0.28 was released on 2022/03/14. It added 37 new commits
after version 2.0.27.
It was announced few week ago. Now, it is released ! Sorry for the delay. The
main issues fixed in this version are:
* A tiny race condition in the scheduler affecting the rare multi-
threaded tasks. In some cases, a task could be finishing to run on one
thread and expiring on another one, just in the process of being
requeued to the position being in the process of being calculated by the
thread finishing with it. The most likely case was the peers task
disabling the expiration while waiting for other peers to be locked,
causing such a non-expirable task to be queued and to block all other
timers from expiring (typically health checks, peers and resolvers, but
others were affected). This could only happen at high peers traffic rate
but it definitely did. When built with the suitable options such as
DEBUG_STRICT it would immediately crash (which is how it was
detected). This bug was present since 2.0.
* A bug in the Set-Cookie2 response parser may result in an infinite loop
triggering the watchdog if a server sends this while it belongs to a
backend configured with cookie persistence. Usually cookie-based
persistence is not used with untrusted servers, but if that was the
case, the following rule would be usable as a workaround for the time it
takes to upgrade:
http-response del-header Set-Cookie2
It reminded us that 2.5 years ago we were discussing about completely
dropping Set-Cookie2 which never succeeded in field, Tim has opened an
issue so that we don't forget to remove it after 2.6. This issue was
diagnosed, reported and fixed by Andrew McDermott and Grant Spence.
This bug was there since 1.9.
* A bug in the H2 multiplexer. An error during the response processing,
after the HEADERS frame parsing, led to a wakeup loop consuming all the
CPU because the error was not properly reported to the upper layer. For
instance, this happened if an invalid header value, an invalid status
code or a forbidden header was found in the response. Note that only
HAProxy >= 2.4 are affected by this issue.
* A bug in the SPOE error handling. When a connection to an agent dies,
there may still be requests pending that are tied to this connection.
The list of such requests is scanned so that they can be aborted, except
that the condition to scan the list was incorrect, and when these
requests were finally aborted upon processing timeout, they were
updating the memory area they used to point to, which could have been
reused for anything, causing random crashes very commonly seen in libc's
malloc/free va openssl, or haproxy pools with corrupted pointers. In
short, anyone using SPOE must absolutely update to apply the fix
otherwise any bug they face cannot be trusted as we know there's a rare
but real case of memory corruption there. This bug was present since
1.8.
* An issue in the pass-through multiplexer leading to a connection leak on
the server side when timeout occurred during the connection
establishment. In this case, the server connection was detached from the
application stream but not closed. At this stage the connection could
only be closed by the server, if it was finally accepted, or by the
kernel, after all SYN retries. All versions as far as 2.3 are affected
by this bug.
* A FD leak on reload failures. When the master process is reloaded on a
new config, it will try to connect to the previous process' socket to
retrieve all known listening FDs to be reused by the new listeners. If
listeners were removed, their unused FDs are simply closed. However
there's a catch. In case a socket fails to bind, the master will cancel
its startup and switch to wait mode for a new operation to happen. In
this case it didn't close the possibly remaining FDs that were left
unused.
* A FD leak of a sockpair upon a failed reload. When starting HAProxy in
master-worker, the master pre-allocate a struct mworker_proc and do a
socketpair() before the configuration parsing. If the configuration
loading failed, the FD was never closed because they aren't part of
listener, they are not even in the fdtab.
* It was possible to temporarily lose the stats sockets upon reloads in
master-worker mode in case of early error (e.g. missing config file), in
which case the socket transfer from the older process couldn't happen.
* Some issues about errors on buffers allocation. First, in the H1
multiplexer. If we failed to send data because we failed to allocate the
H1 output buffer, the H1 stream was erroneously woken up. This led to a
wakeup loop to send more data while it is not possible because there is
no output buffer. Then, in process_stream(), if we failed to allocate
the channel response buffer while a connect or an analysis timeout
occurred, the stream was woken up in loop because its task was requeued
with an expired date. Now an error is reported when this happens and the
stream processing is interrupted.
Note there is a mechanism to deal with errors on buffers allocation.
Unfortunately, since the 1.7, this mechanism is broken. And it is even
worse now with the multiplexers. All this part must be refactored. But
for now, HAProxy may be partially frozen if too many entities are
waiting for a buffer.
* An issue with multi-line ESMTP response in the mailer code.
* An issue in the resolvers code with domain names with a trailing
dot. The trailing dot was not ignored as expected and a junk character
was added at the end of the encoded part of the domain name.
* An issue in the master CLI. When a command was sent to a worker, the
errors, especially write errors, during the response processing were not
properly handled. The session could remain stuck if a client quickly
closed the connection before the response was fully sent. The maxconn
value of the master CLI is set 10. Thus, it could quickly be
unresponsive if this happened several times.
* An issue with all HTX applets. The end of a message was only reported at
the HTX level. The channel's flags were not updated accordingly. The
only known visible effect of this bug was some server aborts erroneously
reported in the stats counters.
* Proxy mode (tcp, http, cli...) is not properly reported when
displayed. Missing "syslog" and "peers" mode can now be reported.
* The anti-loop protection in process_stream() was improved to only count
the no-progress calls.
Some other issues of lower level of importance were also fixed, such as the
CLI being extremely slow to parse pipelined requests because it was looking
for the line feed first, hence the larger the buffer, the slower it was with
batch updates like ACL/map updates; a possibly truncated pidfile in master
mode; an inconsistency with the parsing of IPv4 addresses.
Thanks everyone for your help and your contributions!
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse : http://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Wiki : https://github.com/haproxy/wiki/wiki
Sources : http://www.haproxy.org/download/2.0/src/
Git repository : http://git.haproxy.org/git/haproxy-2.0.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy-2.0.git
Changelog : http://www.haproxy.org/download/2.0/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
---
Complete changelog :
Alex (1):
DOC: use the req.ssl_sni in examples
Andrew McDermott (1):
BUG/MAJOR: http/htx: prevent unbounded loop in
http_manage_server_side_cookies
Christian Ruppert (1):
DOC: Fix usage/examples of deprecated ACLs
Christopher Faulet (11):
BUG/MEDIUM: resolvers: Really ignore trailing dot in domain names
BUG/MEDIUM: mux-h1: Don't wake h1s if mux is blocked on lack of output
buffer
BUG/MAJOR: mux-h2: Be sure to always report HTX parsing error to the app
layer
BUG/MEDIUM: stream: Abort processing if response buffer allocation fails
BUG/MINOR: hlua: Set conn-stream/channel EOI flags at the end of request
BUG/MINOR: stats: Set conn-stream/channel EOI flags at the end of request
BUG/MINOR: cache: Set conn-stream/channel EOI flags at the end of request
BUG/MINOR: promex: Set conn-stream/channel EOI flags at the end of request
DEBUG: cache: Update underlying buffer when loading HTX message in cache
applet
BUG/MEDIUM: mcli: Properly handle errors and timeouts during reponse
processing
BUG/MAJOR: mux-pt: Always destroy the backend connection on detach
Ilya Shipitsin (1):
CI: github actions: use cache for SSL libs
Lukas Tribus (2):
BUG/MINOR: mailers: negotiate SMTP, not ESMTP
DOC: ssl: req_ssl_sni needs implicit TLS
William Lallemand (4):
BUG/MINOR: mworker: does not erase the pidfile upon reload
BUG/MINOR: mworker: fix a FD leak of a sockpair upon a failed reload
BUG/MINOR: tools: url2sa reads ipv4 too far
BUG/MINOR: cli: shows correct mode in "show sess"
Willy Tarreau (16):
MEDIUM: cli: yield between each pipelined command
MINOR: channel: add new function co_getdelim() to support multiple
delimiters
BUG/MINOR: cli: avoid O(bufsize) parsing cost on pipelined commands
BUG/MEDIUM: mcli: do not try to parse empty buffers
BUG/MEDIUM: mcli: always realign wrapping buffers before parsing them
BUG/MEDIUM: mworker: don't lose the stats socket on failed reload
BUG/MAJOR: spoe: properly detach all agents when releasing the applet
MINOR: sock: move the unused socket cleaning code into its own function
BUG/MEDIUM: mworker: close unused transferred FDs on load failure
CI: ssl: enable parallel builds for OpenSSL on Linux
CI: ssl: do not needlessly build the OpenSSL docs
CI: ssl: keep the old method for ancient OpenSSL versions
CI: github actions: add the output of $CC -dM -E-
CLEANUP: atomic: add a fetch-and-xxx variant for common operations
BUG/MAJOR: sched: prevent rare concurrent wakeup of multi-threaded tasks
BUG/MINOR: stream: make the call_rate only count the no-progress calls
--
Christopher Faulet
