Hi,
HAProxy 3.0-dev8 was released on 2024/04/19. It added 115 new commits
after version 3.0-dev7.
3.0-dev7 showed some recent issues related to stick-tables and peers,
which mechanically inflated the number of fixes in this version (32).
Thanks to Christian Ruppert, Felipe Damasio and Ricardo Sanchez for their
quick and helpful reports BTW, because 3.0-dev7 was still warm out of the
oven that the reports started to come about problems!
Here are the main changes:
- the stick-tables issues and peers problems were addressed. However, this
also led to other bugs being discovered on peers, related to recent
changes and that were fixed as well. Next week we'll have an in-depth
review there to make sure there is no other one. At this point we're not
aware of any remaining one in this area.
- another crash was caused by the recent ring changes and affected the
startup logs. Technically speaking, upon reload the buffer was
reallocated of the previous usable size, so 192 bytes were lost upon
every process switchover, which didn't cope well with ring buffers that
were full of logs...
- the evports polling system on Solaris uses a strange API and didn't cope
well with signals and could lose some event reports resulting in an
inconsistency between the internal FD polling state and the one in the
system, which was visible as spinning loops when using external checks.
In addition, it was found that there was a leftover from a debugging
session since day 1 there, by which it would only handle a single event
per loop, making it very inefficient!
- the other issues fixed in various areas are a bit more technical and out
of scope here, and will be more detailed when backported to their
respective stable branches.
- as I mentioned last week, the makefile was updated so that it is easier
to pass CFLAGS/LDFLAGS etc and no longer necessary to hack into
DEBUG_CFLAGS and such cryptic variables. The few that were dropped will
cause the emission of warning and an advice when set so that packagers
will quickly figure what to change. William even suggested that we bail
on error when ERR=1, which I agree with, but it was not done yet. Also,
now if an unknown USE_foo= option is passed on the make command line, a
warning will mention that it's ignored. This should avoid common typos
like inverted words in long names.
- some of the internal "shutdown" API was cleaned up (one function instead
of one per direction), in the hope that it will at least pave the way to
more easily forward errors verbatim between sides (e.g. gRPC events).
- a new "crt-store" configuration section is supported, it allows to
declare certificates by specifying the path for each element. The aim
is essentially to decorellate the storage from the instantiation, both
of which are currently correlated in crt-lists, and to allow easier
specification of individual components. For example:
crt-store
load crt "/crt/site1.crt" key "/keys/site1.key" ocsp "/ocsp/site1.ocsp"
load crt "/crt/site2.crt" key "/keys/site2.key"
In addition, it's possible to set a "crt-base" and a "key-base" there so
that the path doesn't have to be repeated on each line. The certificates
also support aliases so that they can be referenced from a bind line with
a more convenient names, e.g.:
crt-store web
crt-base /etc/ssl/certs/
key-base /etc/ssl/private/
load crt "site3.crt" alias "site3"
load crt "site4.crt" key "site4.key"
frontend in2
bind *:443 ssl crt "@web/site3" crt "@web/site4.crt"
- the backend equivalent of the frontend keylog mechanism was implemented,
so that it is now possible to decipher TLS captures on the backend side.
The log-format to be used becomes a bit large, please refer to the
example in the doc.
- some cleanup was performed on low level QUIC sending functions. Most
notably, duplicated code is removed. The main objective is to facilitate
the development of new features on top of these functions. As
complement, small optimizations to avoid unnecessary function calls are
also introduced which could improve performance slightly.
- the way the memory limitation specified by "-m" on the command line was
handled on Linux using RLIMIT_AS got completely useless over time due to
much more fragmented memory spaces on 64-bit platforms, ASLR, and the
fact that it had been chosen exclusively to avoid underestimating the
allocated buffers' cost, which originally were allocated all the time
even when empty. But nowadays this is no longer relevant since they're
only allocated when used. This had the nasty effect of causing OOMs way
below the configured limit, rendering it pretty useless. Now we've
dropped this specific case and went back to RLIMIT_DATA like on other
OSes.
- some build warnings were addressed with older compilers.
- various other cleanups and code reorganization to help with pending
changes and long term maintenance.
- CI updates (macos version, attempt to dump ASAN output, revert of
workaround for aslr changes).
- And doc updates and cleanups.
Among the stuff in process of being finished soon that I'm aware of, are
the stats dump/handover that are currently delayed a bit by the need to
seriously clean that old code base (not many changes but they have to be
applied all over a massive file, that will be split), finalization of the
shutdown/error transfer changes, some log encoding stuff, an attempt at
finally fixing the buffer_wait stuff (emergency allocation on low memory)
or to make it fail cleanly, then all the small things which will depend on
the amount of time left by bug reports.
>From now on I'll try to switch again to weekly releases so that testers
can more easily follow the small changes and/or the effectiveness of the
fixes for issues they'll have reported.
Please find the usual URLs below :
Site index : https://www.haproxy.org/
Documentation : https://docs.haproxy.org/
Wiki : https://github.com/haproxy/wiki/wiki
Discourse : https://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Sources : https://www.haproxy.org/download/3.0/src/
Git repository : https://git.haproxy.org/git/haproxy.git/
Git Web browsing : https://git.haproxy.org/?p=haproxy.git
Changelog : https://www.haproxy.org/download/3.0/src/CHANGELOG
Dataplane API :
https://github.com/haproxytech/dataplaneapi/releases/latest
Pending bugs : https://www.haproxy.org/l/pending-bugs
Reviewed bugs : https://www.haproxy.org/l/reviewed-bugs
Code reports : https://www.haproxy.org/l/code-reports
Latest builds : https://www.haproxy.org/l/dev-packages
Willy
---
Complete changelog :
Amaury Denoyelle (11):
MINOR: quic: simplify qc_send_hdshk_pkts() return
MINOR: quic: uniformize sending methods for handshake
MINOR: quic: improve sending API on retransmit
MINOR: quic: use qc_send_hdshk_pkts() in handshake IO cb
MEDIUM: quic: remove duplicate hdshk/app send functions
OPTIM: quic: do not call qc_send() if nothing to emit
OPTIM: quic: do not call qc_prep_pkts() if everything sent
BUG/MINOR: guid: fix crash on invalid guid name
MINOR: stats: remove implicit static trash_chunk usage
REORG: stats: extract HTML related functions
REORG: stats: extract JSON related functions
Andrey Lebedev (1):
DOC: management: fix typos
Aurelien DARRAGON (6):
BUG/MINOR: log: fix lf_text_len() truncate inconsistency
BUG/MINOR: tools/log: invalid encode_{chunk,string} usage
BUG/MINOR: log: invalid snprintf() usage in sess_build_logline()
CLEANUP: log: lf_text_len() returns a pointer not an integer
BUG/MEDIUM: peers: fix localpeer regression with 'bind+server' config
style
MINOR: peers: stop relying on srv->addr to find peer port
Christopher Faulet (28):
BUG/MINOR: cli: Don't warn about a too big command for incomplete commands
BUG/MEDIUM: http-ana: Deliver 502 on keep-alive for fressh server
connection
BUG/MINOR: http-ana: Fix TX_L7_RETRY and TX_D_L7_RETRY values
BUG/MEDIUM: cache/stats: Handle inbuf allocation failure in the I/O
handler
MINOR: peers: Split resync process function to separate running/stopping
states
MINOR: peers: Add 2 peer flags about the peer learn status
MINOR: peers: Add flags to report the peer state to the resync task
MINOR: peers: sligthly adapt part processing the stopping signal
MINOR: peers: Add functions to commit peer changes from the resync task
BUG/MINOR: peers: Report a resync was explicitly requested from a
thread-safe manner
BUG/MAJOR: peers: Update peers section state from a thread-safe manner
MEDIUM: peers: Only lock one peer at a time in the sync process function
MINOR: peer: Restore previous peer flags value to ease debugging
BUG/MEDIUM: stconn: Don't forward channel data if input data must be
filtered
BUG/MEDIUM: applet: Fix applet API to put input data in a buffer
BUG/MEDIUM: spoe: Always retry when an applet fails to send a frame
BUG/MEDIUM: peers: Fix exit condition when max-updates-at-once is reached
BUILD: linuxcap: Properly declare prepare_caps_from_permitted_set()
BUG/MINOR: stconn: Fix sc_mux_strm() return value
MINOR: mux-pt: Test conn flags instead of sedesc ones to perform a full
close
MINOR: stconn/connection: Move shut modes at the SE descriptor level
MINOR: stconn: Rewrite shutdown functions to simplify the switch
statements
MEDIUM: stconn: Use only one SC function to shut connection endpoints
MEDIUM: stconn: Explicitly pass shut modes to shut applet endpoints
MEDIUM: stconn: Use one function to shut connection and applet endpoints
MEDIUM: muxes: Use one callback function to shut a mux stream
BUG/MEDIUM: peers: Don't set PEERS_F_RESYNC_PROCESS flag on a peer
BUG/MEDIUM: peers: Fix state transitions of a peer
Damien Claisse (1):
BUG/MINOR: server: fix slowstart behavior
Frederic Lecaille (2):
MINOR: net_helper: Add support for floats/doubles.
BUG/MEDIUM: grpc: Fix several unaligned 32/64 bits accesses
Ilya Shipitsin (4):
CI: revert kernel addr randomization introduced in 3a0fc864
CI: reduce ASAN log redirection umbrella size
CLEANUP: assorted typo fixes in the code and comments
CI: modernize macos matrix
Olivier Houchard (1):
MINOR: stats: Get the right prototype for stats_dump_html_end().
Valentine Krasnobaeva (3):
MINOR: listener/protocol: add proto name in alerts
MINOR: proto_quic: add proto name in alert
MINOR: init: use RLIMIT_DATA instead of RLIMIT_AS
William Lallemand (15):
MINOR: ssl: add the section parser for 'crt-store'
DOC: configuration: Add 3.12 Certificate Storage
REGTESTS: ssl: test simple case of crt-store
MINOR: ssl: rename ckchs_load_cert_file to new_ckch_store_load_files_path
MINOR: ssl/crtlist: alloc ssl_conf only when a valid keyword is found
CLEANUP: ssl: remove dead code in cfg_parse_crtstore()
MINOR: ssl: supports crt-base in crt-store
MINOR: ssl: 'key-base' allows to load a 'key' from a specific path
MEDIUM: ssl: support aliases in crt-store
BUG/MINOR: ssl: check on forbidden character on wrong value
BUG/MINOR: ssl: fix crt-store load parsing
MEDIUM: ssl: support a named crt-store section
MEDIUM: ssl: crt-base and key-base local keywords for crt-store
MAJOR: ssl: use the msg callback mecanism for backend connections
MINOR: ssl: implement keylog fetches for backend connections
Willy Tarreau (43):
BUG/MINOR: listener: always assign distinct IDs to shards
BUILD: makefile: warn about unknown USE_* variables
BUILD: makefile: support USE_xxx=0 as well
BUILD: atomic: fix peers build regression on gcc < 4.7 after recent
changes
BUG/MINOR: debug: make sure DEBUG_STRICT=0 does work as documented
BUILD: cache: fix non-inline vs inline declaration mismatch to silence a
warning
BUILD: debug: make DEBUG_STRICT=1 the default
BUILD: pools: make DEBUG_MEMORY_POOLS=1 the default option
CI: update the build options to get rid of unneeded DEBUG options
BUILD: makefile: get rid of the config CFLAGS variable
BUILD: makefile: allow to use CFLAGS to append build options
BUILD: makefile: drop the SMALL_OPTS settings
BUILD: makefile: move -O2 from CPU_CFLAGS to OPT_CFLAGS
BUILD: makefile: get rid of the CPU variable
BUILD: makefile: drop the ARCH variable and better document ARCH_FLAGS
BUILD: makefile: extract ARCH_FLAGS out of LDFLAGS
BUILD: makefile: move the fwrapv option to STD_CFLAGS
BUILD: makefile: make the ERR variable also support 0
BUILD: makefile: add FAILFAST to select the -Wfatal-errors behavior
BUILD: makefile: extract -Werror/-Wfatal-errors from automatic CFLAGS
BUILD: makefile: split WARN_CFLAGS from SPEC_CFLAGS
BUILD: makefile: rename SPEC_CFLAGS to NOWARN_CFLAGS
BUILD: makefile: do not pass warnings to VERBOSE_CFLAGS
BUILD: makefile: also drop DEBUG_CFLAGS
CLEANUP: makefile: make the output of the "opts" target more readable
DOC: install: clarify the build process by splitting it into subsections
BUG/MEDIUM: stick-tables: fix the task's next expiration date
CLEANUP: stick-tables: always respect the to_batch limit when trashing
BUG/MEDIUM: peers/trace: fix crash when listing event types
BUG/MAJOR: stick-tables: fix race with peers in entry expiration
DEBUG: pool: improve decoding of corrupted pools
REORG: pool: move the area dump with symbol resolution to tools.c
DEBUG: pools: report the data around the offending area in case of
mismatch
BUG/MINOR: lru: fix the standalone test case for invalid revision
MINOR: ring: clarify the usage of ring_size() and add
ring_allocated_size()
BUG/MAJOR: ring: use the correct size to reallocate startup_logs
MINOR: ring: always check that the old ring fits in the new one in
ring_dup()
BUILD: cache: fix a build warning with gcc < 7
BUILD: xxhash: silence a build warning on Solaris + gcc-5.5
BUG/MEDIUM: evports: do not clear returned events list on signal
MEDIUM: evports: permit to report multiple events at once
BUG/MINOR: sock: handle a weird condition with connect()
BUG/MINOR: fd: my_closefrom() on Linux could skip contiguous series of
sockets
---
