HAProxy 2.9-dev5 was released on 2023/09/08. It added 108 new commits
after version 2.9-dev4.

About 30 bugs and build fixes were addressed, essentially the same as
those that were fixed in 2.8.3 so I won't rehash them here. One concerns
the way the cluster-secret is used to produce the retry tokens in QUIC.
It was relying on a nul-terminated string with a fallback on a few random
bytes meaning that short or even empty strings could be used. This was
addressed by always calculating a hash early from it, but this means that
retry tokens produced by this version will not be usable as-is with an
haproxy process running in an older version. This means that those who
may have multiple haproxy nodes dealing with QUIC behind an L4 load
balancer may have to upgrade all nodes at once to support retries (these
are used during network floods). This is a little bit annoying but it's
better to address this right now and not have to think about it anymore,
especially when considering that it's only used during attack mitigations.
We'll also backport this into the next 2.8 so the same precautions will
be needed and then it will only be an old memory.

Aside this, some new stuff was brought:
  - CI cleanups and updates (musl, github actions v4, etc)

  - minimal support for Linux capabilities. We've wanted to do this for
    a long time for transparent proxying, interface binding etc to avoid
    running the process as root at runtime but it was never done. Finally
    we've seen some reports (thank you Tristan) where it was obvious that
    QUIC was bound to privileged port and running as an unprivileged user
    in the default socket-per-connection mode. And in this case the bind()
    that we're doing in order to perform sort of a UDP accept() fails. It
    was clear that such now mainstream use cases cannot warrant running as
    root anymore so capabilities were added so that the cap_net_bind_service
    capability can be preserved when switching from root to the final uid.

  - Proxy-protocol can now expose the extra TLVs that are received so that
    a new sample fetch (fc_pp_tlv) can retrieve them. If a front component
    makes use of in-house TLVs to pass extra info, you can now exploit it
    in haproxy.

  - health checks: we've seen a config where there were so many servers
    that health checks could not all complete in time at boot and resulted
    in checks timing out and all servers turning down while the process was
    at 100% CPU for a long period. Since 2.7, health checks can already
    migrate between threads, so now we migrate them more aggressively when
    other threads are less loaded. We can also benefit from the global
    "spread-checks" value to artifically extend the interval to slow them
    down on overloaded threads. Finally, a similar queuing mechanism as
    the server's maxconn was implemented so that each thread engages in no
    more than a certain number of checks at once, and extra ones wait in
    queue. It's not enabled by default but the limit can be set using the
    global "tune.max-checks-per-thread" setting. This managed to totally
    address the problem even at 7k sustained checks per second without any
    single failure.

  - some cleanups and internal rework was done on the log server init code
    in preparation for more changes to come. Such config checks used to be
    centralized and are now done in the context of their proxy. No visible
    change is expected.

  - the master process will now display a more human-friendly message when
    a worker crashes. It will also report the version and the link to the
    known bugs page

  - backend SSL: the SSL sessions are stored per-thread in order to limit
    locking, but as a side effect, each thread had to perform its own
    handshake if no connection could be reused. This was visible with
    health checks which could be responsible for as many handshakes as
    there are threads. It was not a problem in 1.8 when this was done,
    considering that by then 4 threads was considered large and 8 threads
    huge. But now with 64-thread configs becoming more and more common it
    was really becoming inefficient. Thus now a thread will automatically
    reuse the last updated SSL session if it doesn't have one. This also
    participated to speeding up health checks convergence and reducing
    the servers' load upon haproxy reload.

  - some sink/ring code refactor (should not have any visible effect, so
    please do report any unexpected change).

  - for configs showing blatantly inconsistent CPU bindings, a warning
    will be reported. The first case is that when thread sets are bound
    to smaller CPU sets, thus forcing contention to happen, which will
    ruin performance. The second case is when only some threads are
    referenced in cpu-map and others are left anywhere, which might
    very well be the same CPUs as the first ones. It will also produce
    a warning. Please note that not all inconsistent bindings can be
    detected, the combinations are too large to be resolved, but the
    most common ones that happen by mistake are well detected.

  - the HTTP client now allows to configure the default connect timeout
    and retries count, so that failures can be reported much earlier for
    communications inside the datacenter with short RTTs.

  - SSL: the "curves" keyword is now supported on the "server" lines,
    and a global "ssl-default-server-curves" keyword is added to set
    the default.

  - SSL: the build with the AWS-LC TLS stack should now work fine when
    building with "USE_OPENSSL_AWSLC=1", however I seem to understand
    that some algorithms necessary for QUIC are still not enabled.

The rest is mostly low-risk internal stuff to prepare future changes.

Please find the usual URLs below :
   Site index       : https://www.haproxy.org/
   Documentation    : https://docs.haproxy.org/
   Wiki             : https://github.com/haproxy/wiki/wiki
   Discourse        : https://discourse.haproxy.org/
   Slack channel    : https://slack.haproxy.org/
   Issue tracker    : https://github.com/haproxy/haproxy/issues
   Sources          : https://www.haproxy.org/download/2.9/src/
   Git repository   : https://git.haproxy.org/git/haproxy.git/
   Git Web browsing : https://git.haproxy.org/?p=haproxy.git
   Changelog        : https://www.haproxy.org/download/2.9/src/CHANGELOG
   Dataplane API    : 
   Pending bugs     : https://www.haproxy.org/l/pending-bugs
   Reviewed bugs    : https://www.haproxy.org/l/reviewed-bugs
   Code reports     : https://www.haproxy.org/l/code-reports
   Latest builds    : https://www.haproxy.org/l/dev-packages

Complete changelog :
Alexander Stephan (6):
      CLEANUP/MINOR: connection: Improve consistency of PPv2 related constants
      MEDIUM: connection: Generic, list-based allocation and look-up of PPv2 
      MEDIUM: sample: Add fetch for arbitrary TLVs
      MINOR: sample: Refactor fc_pp_authority by wrapping the generic TLV fetch
      MINOR: sample: Refactor fc_pp_unique_id by wrapping the generic TLV fetch
      MINOR: sample: Add common TLV types as constants for fc_pp_tlv

Andrew Hopkins (6):
      BUILD: ssl: Build with new cryptographic library AWS-LC
      REGTESTS: ssl: skip ssl_dh test with AWS-LC
      CI: scripts: add support to build-ssl.sh to download and build AWS-LC
      CI: add support to matrix.py to determine the latest AWS-LC release
      CI: Update matrix.py so all code is contained in functions.
      CI: github: Add a weekly CI run building with AWS-LC

Aurelien DARRAGON (18):
      BUG/MINOR: hlua/action: incorrect message on E_YIELD error
      MINOR: http_ana: position the FINAL flag for http_after_res execution
      MINOR: ring: add a function to compute max ring payload
      BUG/MEDIUM: ring: adjust maxlen consistency check
      MINOR: sink: simplify post_sink_resolve function
      MINOR: log/sink: detect when log maxlen exceeds sink size
      MINOR: sink: inform the user when logs will be implicitly truncated
      MEDIUM: sink: don't perform implicit truncations when maxlen is not set
      MINOR: log: move log-forwarders cleanup in log.c
      MEDIUM: httpclient/logs: rely on per-proxy post-check instead of global 
      MINOR: log: add dup_logsrv() helper function
      MEDIUM: log/sink: make logsrv postparsing more generic
      MEDIUM: fcgi-app: properly postresolve logsrvs
      MEDIUM: spoe-agent: properly postresolve log rings
      MINOR: sink: add helper function to deallocate sink struct
      MEDIUM: sink/ring: introduce high level ring creation helper function
      MEDIUM: sink: add sink_finalize() function
      CLEANUP: log: remove unnecessary trim in __do_send_log

Chris Staite (1):
      BUG/MEDIUM: h1-htx: Ensure chunked parsing with full output buffer

Christopher Faulet (13):
      DEBUG: applet: Properly report opposite SC expiration dates in traces
      BUG/MEDIUM: stconn: Update stream expiration date on blocked sends
      BUG/MINOR: stconn: Don't report blocked sends during connection 
      BUG/MEDIUM: stconn: Wake applets on sending path if there is a pending 
      BUG/MEDIUM: stconn: Don't block sends if there is a pending shutdown
      BUG/MINOR: stconn: Don't inhibit shutdown on connection on error
      BUG/MEDIUM: applet: Fix API for function to push new data in channels 
      BUG/MEDIUM: stconn: Report read activity when a stream is attached to 
front SC
      BUG/MEDIUM: applet: Report an error if applet request more room on 
aborted SC
      BUG/MEDIUM: stconn/stream: Forward shutdown on write timeout
      NUG/MEDIUM: stconn: Always update stream's expiration date after I/O
      BUG/MINOR: applet: Always expect data when CLI is waiting for a new 
      BUG/MINOR: ring/cli: Don't expect input data when showing events

Frédéric Lécaille (11):
      BUG/MINOR: quic: Possible skipped RTT sampling
      MINOR: quic: Add a trace to quic_release_frm()
      BUG/MAJOR: quic: Really ignore malformed ACK frames.
      BUG/MINOR: quic: Unchecked pointer to packet number space dereferenced
      BUILD: quic: Compilation issue on 32-bits systems with 
      BUG/MINOR: quic: Unchecked pointer to Handshake packet number space
      BUG/MINOR: quic: Wrong RTT adjusments
      BUG/MINOR: quic: Wrong RTT computation (srtt and rrt_var)
      BUG/MINOR: quic: Dereferenced unchecked pointer to Handshke packet number 
      BUG/MINOR: quic: Wrong cluster secret initialization
      CLEANUP: quic: Remove useless free_quic_tx_pkts() function.

Ilya Shipitsin (2):
      CI: musl: highlight section if there are coredumps
      CI: musl: drop shopt in workflow invocation

Miroslav Zagorac (1):
      MINOR: properly mark the end of the CLI command in error messages

Remi Tricot-Le Breton (1):
      MINOR: cache: Change hash function in default normalizer used in case of 

Tim Duesterhus (1):
      CI: Update to actions/checkout@v4

William Lallemand (7):
      BUG/MINOR: ssl/cli: can't find ".crt" files when replacing a certificate
      DOC: configuration: update examples for req.ver
      MINOR: global: export the display_version() symbol
      MEDIUM: mworker: display a more accessible message when a worker crash
      MINOR: httpclient: allow to configure the retries
      MINOR: httpclient: allow to configure the timeout.connect
      MINOR: ssl: add support for 'curves' keyword on server lines

Willy Tarreau (41):
      BUG/MEDIUM: mux-h2: fix crash when checking for reverse connection after 
      BUILD: import: guard plock.h against multiple inclusion
      BUILD: pools: import plock.h to build even without thread support
      BUG/MINOR: stream: protect stream_dump() against incomplete streams
      DOC: config: mention uid dependency on the tune.quic.socket-owner option
      MEDIUM: capabilities: enable support for Linux capabilities
      MINOR: ssl_sock: avoid iterating realloc(+1) on stored context
      DOC: ssl: add some comments about the non-obvious session allocation stuff
      CLEANUP: ssl: keep a pointer to the server in ssl_sock_init()
      MEDIUM: ssl_sock: always use the SSL's server name, not the one from the 
      MEDIUM: server/ssl: place an rwlock in the per-thread ssl server session
      MINOR: server/ssl: maintain an index of the last known valid SSL session
      MINOR: server/ssl: clear the shared good session index on failure
      MEDIUM: server/ssl: pick another thread's session when we have none yet
      MINOR: activity: report the current run queue size
      BUG/MINOR: checks: do not queue/wake a bounced check
      MINOR: checks: start the checks in sleeping state
      MINOR: checks: pin the check to its thread upon wakeup
      MINOR: check: remember when we migrate a check
      MINOR: check/activity: collect some per-thread check activity stats
      MINOR: checks: maintain counters of active checks per thread
      MINOR: check: also consider the random other thread's active checks
      MEDIUM: checks: search more aggressively for another thread on overload
      MEDIUM: checks: implement a queue in order to limit concurrent checks
      MINOR: checks: also consider the thread's queue for rebalancing
      BUG/MEDIUM: connection: fix pool free regression with recent ppv2 TLV 
      BUG/MINOR: stream: further protect stream_dump() against incomplete 
      BUILD: bug: make BUG_ON() void to avoid a rare warning
      BUILD: checks: shut up yet another stupid gcc warning
      MINOR: cpuset: add ha_cpuset_isset() to check for the presence of a CPU 
in a set
      MINOR: cpuset: add ha_cpuset_or() to bitwise-OR two CPU sets
      MINOR: cpuset: centralize a reliable bound cpu detection
      MEDIUM: threads: detect incomplete CPU bindings
      MEDIUM: threads: detect excessive thread counts vs cpu-map
      MINOR: tasks/stats: report the number of niced tasks in "show info"
      MEDIUM: init: initialize the trash earlier
      MINOR: tools: add function read_line_to_trash() to read a line of a file
      MINOR: cfgparse: use read_line_from_trash() to read from /sys
      MEDIUM: cfgparse: assign NUMA affinity to cpu-maps
      MINOR: cpuset: dynamically allocate cpu_map
      REORG: cpuset: move parse_cpu_set() and parse_cpumap() to cpuset.c


Reply via email to