HAProxy 2.4-dev1 was released on 2020/11/21. It added 107 new commits
after version 2.4-dev0.

Let me say that despite having spent quite some time recently on a few
really irritating bugs, I'm quite happy to see that the new development
cycle starts to pay off. It's probably the first ever new branch that
gets 107 patches 2 weeks after having been opened, and the majority are
updates and not bug fixes.

First, let's have a quick round on the bugs fixed since last release
(2.3.2 should be issued shortly). The latest SSL changes in 2.3 had a
little bit of crt-list breakage that was quickly addressed. A bug in
the http-after-response rules could possibly cause random crashes. An
old bug in the SPOE with a dangling pointer could cause random crashes
(Many thanks to Maciej Zdeb for working hard for two months to isolate
this one). Checks could crash if a "proto" directive was set on the
server lines. And finally the last one, a dangling session pointer in
the idle connections was sometimes used after the session had been
detached, causing random crashes. These were detected on 2.3 which
amplifies the issue. While it's possible to crash 2.2 on the same issue
by applying a minor patch, it's impossible to say if a similar code path
is used without that patch. And a few crash reports there look suspiciously
related, so given that backporting this one requires extreme care, I'd
appreciate it if the rare ones who occasionally experience a crash in 2.2
could run an instance on 2.4-dev1 and report if they think their issue is

Now the new stuff. First there was a nice liftup of the CI to migrate to
GitHub actions (thanks Tim and Ilya for the work). We now have a larger
test matrix which seems more reliable and is more controllable than the
one we previously used on Travis. It was also an opportunity to start to
make the SSL build process more resistant to the non-linear evolutions of
the various OpenSSL forks.

Some changes were made to the pattern code to stop freezing the whole
process each time a del-acl or del-map action is performed. While it used
to be OK when dealing with only a few hundreds of thousands of entries,
it's not fun anymore with maps containing 20 million IP addresses where it
used to cause long pauses that sometimes managed to trigger the watchdog!
With this change, the ACL/map entries are now versionned and atomically
updated, so that it is possible to perform a delete in the background in
small batches, and even if it requires a full scan for certain types, it
will be done in small batches.

The cache used not to fully comply with the standards, as it would cache
an object that didn't have an explicit expiration time nor validator. Rémi
fixed this. Normally nobody should notice anything because such objects are
almost non-existent nowadays. However maybe some broken applications will
not be cached anymore, but for good, in that it was not really possible to
check for validity there.

It used to be possible to change a server's IP address at run time form the
CLI but it was not possible to enable SSL at run time because it required
to allocate an SSL context. William Dauchy worked on this so that this
limitation doesn't exist anymore.

The memory of the old process usage during reloads should significantly
on systems supporting malloc_trim() (i.e. glibc for now). While working
on optimizing the patterns I was annoyed by the huge memory usage after
replacing a whole map and have been looking for a way to compact unused
memory. I discovered this malloc_trim() that does exactly what one would
expect, i.e. unmap all unused pages from the allocator's caches. My old
process went down form 1.7 GB to 260 MB! Those doing frequent reloads
might be interested in giving it a try.

Amaury added some SSL stats so that it will now be possible to count
handshakes an errors on the two sides. More detailed info will likely
come over time but for me this will be related to the ability to report
better SSL logs as well.

Christopher and Baptiste finally finished their work on the MQTT and FIX
parsers. These can be used to extract information from initial messages
and steer the traffic to one server or another (or to drop it).

Fred added some traces to the peers so that exchanges can now be observed.
This is essentially useful for debugging, but always interesting to see
what flows between nodes during tables synchronization. This should be
improved over time.

Maciej implemented the "-m" argument to the "del-header" action, that was
initially planned for 2.2 and that everyone forgot about. This one allows
to specify if the argument to del-header designates a full header name,
a substring, a prefix, a suffix, or even a regex. This is something a lot
of users have been missing after "rspdel" was removed. I suspect that his
work is safe for backporting, so if anyone currently uses 2.2 or 2.3 and
is using ugly tricks (or Lua) just to remove header names by prefix for
example, just raise your hand to ask for a backport, we'll see what can
be done.

I also want to thank those who contributed new regression tests, we've got
12 new ones after 2.3, this is very useful and significantly contributes
to the code's quality and reliability.

Last point, for those who always want to live on the bleeding edge, the
changes in this version are still pretty minor and relatively safe, I'm
going to put it on haproxy.org, feel free to do so as well (but not on
all servers, as usual).

Please find the usual URLs below :
   Site index       : http://www.haproxy.org/
   Discourse        : http://discourse.haproxy.org/
   Slack channel    : https://slack.haproxy.org/
   Issue tracker    : https://github.com/haproxy/haproxy/issues
   Wiki             : https://github.com/haproxy/wiki/wiki
   Sources          : http://www.haproxy.org/download/2.4/src/
   Git repository   : http://git.haproxy.org/git/haproxy.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy.git
   Changelog        : http://www.haproxy.org/download/2.4/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/

Complete changelog :
Amaury Denoyelle (10):
      MINOR: ssl: instantiate stats module
      MINOR: ssl: count client hello for stats
      MINOR: ssl: add counters for ssl sessions
      BUG/MINOR: stats: free dynamically stats fields/lines on shutdown
      BUG/MEDIUM: stats: prevent crash if counters not alloc with dummy one
      BUG/MEDIUM: check: reuse srv proto only if using same mode
      MINOR: check: report error on incompatible proto
      MINOR: check: report error on incompatible connect proto
      MINOR: ssl: remove client hello counters
      MEDIUM: stats: add counters for failed handshake

Baptiste Assmann (2):
      MINOR: sample: Add converters to parse FIX messages
      MINOR: sample: Add converts to parses MQTT messages

Christopher Faulet (21):
      MINOR: ist: Add istend() function to return a pointer to the end of the 
      REGTEST: converter: Add a regtest for fix converters
      REGTEST: converter: Add a regtest for MQTT converters
      MINOR: http-htx: Add understandable errors for the errorfiles parsing
      DOC: config: Fix a typo on ssl_c_chain_der
      BUG/MINOR: http-fetch: Fix calls w/o parentheses of the cookie sample 
      BUG/MINOR: http-htx: Handle warnings when parsing http-error and 
      BUG/MAJOR: spoe: Be sure to remove all references on a released spoe 
      MINOR: spoe: Don't close connection in sync mode on processing timeout
      BUG/MINOR: tcpcheck: Don't warn on unused rules if check option is after
      MINOR: init: Fix the prototype for per-thread free callbacks
      MINOR: config/mux-h2: Return ERR_ flags from init_h2() instead of a status
      CLEANUP: config: Return ERR_NONE from config callbacks instead of 0
      REGTEST: make ssl_client_samples and ssl_server_samples require to 2.2
      BUG/MEDIUM: filters: Forward all filtered data at the end of http 
      BUG/MINOR: http-ana: Don't wait for the body of CONNECT requests
      CLEANUP: flt-trace: Remove unused random-parsing option
      MINOR: flt-trace: Add an option to inhibits trace messages
      MINOR: flt-trace: Use a bitfield for the trace options
      REGTESTS: Add a script to test the random forwarding with several filters
      BUG/MEDIUM: http-ana: Don't eval http-after-response ruleset on empty 

Eric Salama (1):
      MINOR: cfgparse: tighten the scope of newnameserver variable, free it on 

Frédéric Lécaille (3):
      MINOR: peers: Add traces to peer_treat_updatemsg().
      BUG/MINOR: peers: Do not ignore a protocol error for dictionary entries.
      BUG/MINOR: peers: Missing TX cache entries reset.

Ilya Shipitsin (9):
      CI: travis-ci: remove amd64, osx builds
      CI: travis-ci: arm64 are not allowed to fail anymore
      BUILD: ssl: use SSL_MODE_ASYNC macro instead of OPENSSL_VERSION
      CI: Github Actions: enable prometheus exporter
      CI: Github Actions: remove LibreSSL-3.0.2 builds
      CI: Github Actions: enable BoringSSL builds
      CI: travis-ci: remove builds migrated to GH actions
      CI: Github Action: run "apt-get update" before packages restore
      BUILD: SSL: guard TLS13 ciphersuites with HAVE_SSL_CTX_SET_CIPHERSUITES

Jerome Magnin (1):
      CLEANUP: cfgparse: remove duplicate registration for transparent build 

Joao Morais (1):
      DOC: clarify how to create a fallback crt

Maciej Zdeb (3):
      BUG/MINOR: http-fetch: Extract cookie value even when no cookie name
      BUG/MINOR: http_htx: Fix searching headers by substring
      MINOR: http_act: Add -m flag for del-header name matching method

Matthieu Guegan (1):
      BUILD: makefile: enable crypt(3) for OpenBSD

Remi Tricot-Le Breton (1):
      MEDIUM: cache: Change caching conditions

Thierry Fournier (2):
      BUG/MINOR: pattern: a sample marked as const could be written
      BUG/MINOR: lua: set buffer size during map lookups

Tim Duesterhus (6):
      CI: Expand use of GitHub Actions for CI
      REGTESTS: Add sample_fetches/cook.vtc
      CI: Stop hijacking the hosts file
      CI: Make the h2spec workflow more consistent with the VTest workflow
      CI: Pass the github.event_name to matrix.py
      CI: Clean up Windows CI

William Dauchy (3):
      REGTESTS: converter: add url_dec test
      MINOR: ssl: create common ssl_ctx init
      MEDIUM: cli/ssl: configure ssl on server at runtime

William Lallemand (10):
      REGTEST: ssl: test wildcard and multi-type + exclusions
      BUG/MEDIUM: ssl/crt-list: correctly insert crt-list line if crt already 
      REGTEST: ssl: mark reg-tests/ssl/ssl_crt-list_filters.vtc as broken
      DOC: add missing 3.10 in the summary
      REGTEST: server/cli_set_ssl.vtc requires OpenSSL
      BUG/MINOR: ssl: segv on startup when AKID but no keyid
      BUG/MEDIUM: ssl/crt-list: bundle support broken in crt-list
      BUG/MEDIUM: ssl: error when no certificate are found
      BUG/MINOR: ssl/crt-list: load bundle in crt-list only if activated
      BUG/MEDIUM: ssl/crt-list: fix error when no file found

Willy Tarreau (33):
      MINOR: compat: automatically include malloc.h on glibc
      MEDIUM: pools: call malloc_trim() from pool_gc()
      MEDIUM: pattern: call malloc_trim() on pat_ref_reload()
      MINOR: pattern: move the update revision to the pat_ref, not the 
      CLEANUP: pattern: delete the back refs at once during pat_ref_reload()
      MINOR: pattern: new sflag PAT_SF_REGFREE indicates regex_free() is needed
      MINOR: pattern: make the delete and prune functions more generic
      MEDIUM: pattern: link all final elements from the reference
      MEDIUM: pattern: change the pat_del_* functions to delete from the 
      MINOR: pattern: remerge the list and tree deletion functions
      MINOR: pattern: perform a single call to pat_delete_gen() under the 
      CLEANUP: acl: don't reference the generic pattern deletion function 
      CLEANUP: pattern: remove pat_delete_fcts[] and pattern_head->delete()
      MINOR: pattern: introduce pat_ref_delete_by_ptr() to delete a valid 
      MINOR: pattern: store a generation number in the reference patterns
      MEDIUM: pattern: only match patterns that match the current generation
      MINOR: pattern: add pat_ref_commit() to commit a previously inserted 
      MINOR: pattern: implement pat_ref_load() to load a pattern at a given 
      MINOR: pattern: add pat_ref_purge_older() to purge old entries
      MEDIUM: pattern: make pat_ref_prune() rely on pat_ref_purge_older()
      MINOR: pattern: during reload, delete elements frem the ref, not the 
      MINOR: pattern: prepare removal of a pattern from the list head
      MEDIUM: pattern: turn the pattern chaining to single-linked list
      BUG/MINOR: ssl: don't report 1024 bits DH param load error when it's 
      MINOR: server: remove idle lock in srv_cleanup_connections
      BUILD: ssl: silence build warning on uninitialised counters
      BUILD: http-htx: fix build warning regarding long type in printf
      BUG/MEDIUM: peers: fix decoding of multi-byte length in stick-table 
      REGTESTS: mark the abns test as broken again
      DOC: coding-style: update a few rules about pointers
      CLEANUP: connection: do not use conn->owner when the session is known
      BUG/MAJOR: connection: reset conn->owner when detaching from session list
      REGTESTS: mark proxy_protocol_random_fail as broken


Reply via email to