Hi,
HAProxy 2.5-dev9 was released on 2021/10/08. It added 162 new commits
after version 2.5-dev8.
This brings the last round of possibly breaking changes. From this point
we should be careful not to change significant stuff and only to finish
what was begun, fix bugs, and perform some cleanups and doc updates,
especially since there has been a growing number of issues lately, some
of which might have accumulated due to developers being busy finishing
their changes and also because we're seeing an increase of feature
requests that take time to review and/or qualify. Thus my hope for next
versions is to see this number of issues go down, and likely a lot of
the small pending stuff completed.
This version looks large but it's mostly due to some recent pain with
includes (recurring issue) that managed to put a halt to the progress on
thread-groups. However as usual, many files are touched to move stuff
around but if it builds it's doesn't bring anything, otherwise it breaks
and we discover that some fixes are missing :-) The nice part in this is
that the routine build times dropped by ~38%, showing that code hygiene
ultimately pays off. If we keep that stuff away, and the usual bugs fixed
in every version, we're left mostly with:
- initial support for a thread group in front of the "thread" keyword
on "bind" lines, and for the "thread-groups" directive in the config.
For now it has no visible effect (group limited to 1 by default) but
it will help maintain compatible configs with future versions, that
will ease migrations back and forth.
- HTTP/1 updates to comply with latest updates to the spec:
Transfer-Encoding should not appear with HTTP/1.0 and can be abused
depending on how other intermediaries parse it; now a request or
response featuring a Transfer-Encoding header will automatically be
the last one on the connection. Similarly, since Content-Length is
forbidden to send together with Transfer-Encoding, seeing them both
implies talking with a non-conforming agent. The connection will also
be closed after the transfer in this case. The "TE" header is sanitized
to make sure not to advertise unsupported encodings to the server. And
unsupported encodings in requests or responses will be rejected to
prevent cache pollution or corrupted transfers.
- A number of improvements and fixes were brought to the http client
(both Lua an native), mostly on resource freeing.
- a new batch of QUIC fixes was merged, which mainly focuses on resource
freeing.
- "show pools" on the CLI will indicate what part of the "used" value
represents free memory in thread-local caches; some users were confused
into thinking they were facing a leak, and it's not normal that we only
report confusing information there.
- the "ssl_bc_hsk_err" sample fetch introduced in 2.5-dev6 was renamed to
"ssl_bc_err" because it will report more than just handshake errors in
TLS 1.3. Now SSL errors should be more accurate, especially when they
involve a peer rejecting a certificate.
- 3 regtests were added and 3 other ones fixed and re-enabled.
- the "conn_cur" stick-table data is not learned anymore from other peers.
This was a flaw since this element became replicable, which has caused
a number of questions (and even fixes). It represents a gauge that
corresponds to the number of currently active connections tracking a
key on the local peer, or on the one that pushed it last. Writing a
value from another peer here only results in the entry reaching zero
before the end, or worse, not being able to reach zero because the value
is higher than the local number of connections on the key. This happens
quite often during reloads or in active-backup setups so let's put an
end to this mistake. It's still emitted though, in case users developed
monitoring systems based on the protocol, they will continue to work.
The patch is trivial to backport, if some users are annoyed enough by
the current behavior, we could discuss about backporting it (but not
too far, say 2.4 max).
- usual small batch of doc updates
For the pending stuff, I have a local list of small trivial things to be
done that are independent on the release and that can get merged as they
are done. There's the set-src/set-dst stuff to be fixed (discussed in issue
#1303), enabling support for set-var() in "tcp-request connection", and
checking with Björn if we can get MPTCP finished in time (I think so but
as usual there's still some work to be done for both of us). This one
should be harmless as well so I'm fine with merging it late. I know that
Rémi was working on JWT decoding, and similarly it should integrate
seamlessly, so if it arrives late it should be a problem.
And the rest will have to be for 2.6. So far, so good. Please do not send
new features that require reviews at this point so that we can all stay
focused on what still has to be finished.
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse : http://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Wiki : https://github.com/haproxy/wiki/wiki
Sources : http://www.haproxy.org/download/2.5/src/
Git repository : http://git.haproxy.org/git/haproxy.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy.git
Changelog : http://www.haproxy.org/download/2.5/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
Willy
---
Complete changelog :
Amaury Denoyelle (10):
BUG/MINOR: task: fix missing include with DEBUG_TASK
BUG/MEDIUM: mux-quic: reinsert all streams in by_id tree
BUG/MAJOR: xprt-quic: do not queue qc timer if not set
MINOR: mux-quic: release connection if no more bidir streams
BUG/MAJOR: quic: remove qc from receiver cids tree on free
MINOR: qpack: do not encode invalid http status code
MINOR: qpack: support non-indexed http status code encoding
MINOR: qpack: fix memory leak on huffman decoding
CLEANUP: mux-quic: remove unused code
BUG/MINOR: quic: fix includes for compilation
Christopher Faulet (16):
MINOR: log: Try to get the status code when MUX_EXIT_STATUS is retrieved
MINOR: mux-h1: Set error code if possible when MUX_EXIT_STATUS is returned
MINOR: mux-h1: Be able to set custom status code on parsing error
MEDIUM: mux-h1: Reject HTTP/1.0 GET/HEAD/DELETE requests with a payload
MEDIUM: h1: Force close mode for invalid uses of T-E header
BUG/MINOR: mux-h1/mux-fcgi: Sanitize TE header to only send "trailers"
MINOR: http: Add 422-Unprocessable-Content error message
MINOR: h1: Change T-E header parsing to fail if chunked encoding is found
twice
BUG/MEDIUM: mux-h1/mux-fcgi: Reject messages with unknown transfer
encoding
REGTESTS: Add script to validate T-E header parsing
MINOR: arg: Be able to forbid unresolved args when building an argument
list
BUG/MINOR: tcpcheck: Don't use arg list for default proxies during parsing
BUG/MINOR: tcp-rules: Stop content rules eval on read error and
end-of-input
BUG/MEDIUM: filters: Fix a typo when a filter is attached blocking the
release
BUG/MEDIUM: http-ana: Clear request analyzers when applying redirect rule
BUG/MEDIUM: mux_h2: Handle others remaining read0 cases on partial frames
Emeric Brun (1):
DOC: peers: fix doc "enable" statement on "peers" sections
Frédéric Lécaille (4):
MINOR: quic: Distinguish packet and SSL read enc. level in traces
MINOR: quic: Add a function to dump SSL stack errors
MINOR: quic: BUG_ON() SSL errors.
MINOR: quic: Fix SSL error issues (do not use ssl_bio_and_sess_init())
Remi Tricot-Le Breton (3):
MINOR: ssl: Set connection error code in case of SSL read or write fatal
failure
MINOR: ssl: Rename ssl_bc_hsk_err to ssl_bc_err
MINOR: ssl: Store the last SSL error code in case of read or write failure
Thayne McCombs (1):
DOC: configuration: add clarification on escaping in keyword arguments
William Lallemand (18):
head-truc
REGTESTS: lua: test the httpclient:get() feature
Revert "head-truc"
BUG/MEDIUM: httpclient: replace ist0 by istptr
MINOR: Makefile: add MEMORY_POOLS to the list of DEBUG_xxx options
REGTESTS: ssl: enable show_ssl_ocspresponse.vtc again
REGTESTS: ssl: enable ssl_crt-list_filters.vtc again
REGTESTS: ssl: show_ssl_ocspresponse w/ freebsd won't use base64
REGTESTS: ssl: wrong feature cmd in show_ssl_ocspresponse.vtc
MINOR: httpclient: destroy() must free the headers and the ists
MINOR: httpclient: set HTTPCLIENT_F_ENDED only in release
MINOR: httpclient: stop_and_destroy() ask the applet to autokill
MINOR: httpclient: test if started during stop_and_destroy()
MINOR: httpclient/lua: implement garbage collection
BUG/MEDIUM: httpclient/lua: crash because of b_xfer and get_trash_chunk()
MINOR: httpclient: destroy checks if a client was started but not stopped
BUG/MINOR: httpclient/lua: does not process headers when failed
MINOR: httpclient/lua: supports headers via named arguments
Willy Tarreau (109):
MINOR: config: use a standard parser for the "nbthread" keyword
CLEANUP: init: remove useless test against MAX_THREADS in affinity loop
MEDIUM: init: de-uglify the per-thread affinity setting
MINOR: init: extract the setup and end of threads to their own functions
REORG: pools: move default settings to defaults.h
BUG/MEDIUM: lua: fix wakeup condition from sleep()
BUG/MAJOR: lua: use task_wakeup() to properly run a task once
MINOR: tasks: catch TICK_ETERNITY with BUG_ON() in __task_queue()
CLEANUP: tasks: remove the long-unused work_lists
MINOR: task: provide 3 task_new_* wrappers to simplify the API
MINOR: time: uninline report_idle() and move it to task.c
REORG: sched: move idle time calculation from time.h to task.h
REORG: sched: move the stolen CPU time detection to sched_entering_poll()
CLEANUP: server: always include the storage for SSL settings
CLEANUP: sample: rename sample_conv_var2smp() to *_sint
CLEANUP: sample: uninline sample_conv_var2smp_str()
MINOR: sample: provide a generic var-to-sample conversion function
BUG/MEDIUM: sample: properly verify that variables cast to sample
BUILD: action: add the relevant structures for function arguments
BUILD: extcheck: needs to include stream-t.h
BUILD: hlua: needs to include stream-t.h
BUILD: stats: define several missing structures in stats.h
BUILD: resolvers: define missing types in resolvers.h
BUILD: httpclient: include missing ssl_sock-t
BUILD: sample: include openssl-compat
BUILD: http_ana: need to include proxy-t to get redirect_rule
BUILD: http_rules: requires http_ana-t.h for REDIRECT_*
BUILD: vars: need to include xxhash
BUILD: peers: need to include eb{32/mb/pt}tree.h
BUILD: ssl_ckch: include ebpttree.h in ssl_ckch.c
BUILD: compiler: add the container_of() and container_of_safe() macros
BUILD: idleconns: include missing ebmbtree.h at several places
BUILD: connection: connection.h needs list.h and server.h
BUILD: tree-wide: add missing http_ana.h from many places
BUILD: cfgparse-ssl: add missing errors.h
BUILD: tcp_sample: include missing errors.h and session-t.h
BUILD: mworker: mworker-prog needs time.h for the 'now' variable
BUILD: tree-wide: add several missing activity.h
BUILD: compat: fix -Wundef on SO_REUSEADDR
CLEANUP: pools: pools-t.h doesn't need to include thread-t.h
REORG: pools: uninline the UAF allocator and force-inline the rest
REORG: thread: uninline the lock-debugging code
MINOR: thread/debug: replace nsec_now() with now_mono_time()
CLEANUP: remove some unneeded includes from applet-t.h
REORG: listener: move bind_conf_alloc() and listener_state_str() to
listener.c
CLEANUP: listeners: do not include openssl-compat
CLEANUP: servers: do not include openssl-compat
REORG: ssl: move ssl_sock_is_ssl() to connection.h and rename it
CLEANUP: mux_fcgi: remove dependency on ssl_sock
CLEANUP: ssl/server: move ssl_sock_set_srv() to srv_set_ssl() in server.c
REORG: ssl-sock: move the sslconns/totalsslconns counters to global
REORG: sample: move the crypto samples to ssl_sample.c
REORG: sched: moved samp_time and idle_time to task.c as well
REORG: time/ticks: move now_ms and global_now_ms definitions to ticks.h
CLEANUP: tree-wide: remove unneeded include time.h in ~20 files
REORG: activity: uninline activity_count_runtime()
REORG: acitvity: uninline sched_activity_entry()
CLEANUP: stream: remove many unneeded includes from stream-t.h
CLEANUP: stick-table: no need to include socket nor in.h
MINOR: connection: use uint64_t for the hashes
REORG: connection: move the hash-related stuff to connection.c
REORG: connection: uninline conn_notify_mux() and conn_delete_from_tree()
REORG: server: uninline the idle conns management functions
REORG: ebtree: split structures into their own file ebtree-t.h
CLEANUP: tree-wide: only include ebtree-t from type files
REORG: connection: move the largest inlines from connection.h to
connection.c
CLEANUP: connection: do not include http_ana!
CLEANUP: connection: remove unneeded tcpcheck-t.h and use only session-t.h
REORG: connection: uninline the rest of the alloc/free stuff
REORG: task: uninline the loop time measurement code
CLEANUP: time: move a few configurable defines to defaults.h
CLEANUP: fd: do not include time.h
REORG: fd: uninline compute_poll_timeout()
CLENAUP: wdt: use ha_tkill() instead of accessing pthread directly
REORG: thread: move the thread init/affinity/stop to thread.c
REORG: thread: move ha_get_pthread_id() to thread.c
MINOR: thread: use a dedicated static pthread_t array in thread.c
CLEANUP: thread: uninline ha_tkill/ha_tkillall/ha_cpu_relax()
MINOR: pools: report the amount used by thread caches in "show pools"
BUILD: connection: avoid a build warning on FreeBSD with SO_USER_COOKIE
BUILD: init: avoid a build warning on FreeBSD with USE_PROCCTL
REORG: time: move time-keeping code and variables to clock.c
REORG: clock: move the updates of cpu/mono time to clock.c
MINOR: activity: get the run_time from the clock updates
CLEANUP: clock: stop exporting before_poll and after_poll
REORG: clock: move the clock_id initialization to clock.c
REORG: clock/wdt: move wdt timer initialization to clock.c
MINOR: clock: move the clock_ids to clock.c
MINOR: wdt: move wd_timer to wdt.c
CLEANUP: wdt: do not remap SI_TKILL to SI_LWP, test the values directly
REORG: thread/sched: move the task_per_thread stuff to thread_ctx
REORG: thread/clock: move the clock parts of thread_info to thread_ctx
REORG: thread/sched: move the thread_info flags to the thread_ctx
REORG: thread/sched: move the last dynamic thread_info to thread_ctx
MINOR: thread: make "ti" a const pointer and clean up thread_info a bit
MINOR: threads: introduce a minimalistic notion of thread-group
MINOR: global: add a new "thread-groups" directive
MINOR: global: add a new "thread-group" directive
MINOR: threads: make tg point to the current thread's group
MEDIUM: threads: automatically assign threads to groups
MINOR: threads: set the group ID and its bit in the thread group
MINOR: threads: set the tid, ltid and their bit in thread_cfg
MEDIUM: threads: replace ha_set_tid() with ha_set_thread()
MINOR: threads: add the current group ID in thread-local "tgid" variable
MINOR: debug: report the group and thread ID in the thread dumps
MEDIUM: listeners: support the definition of thread groups on bind lines
MINOR: threads: add a new function to resolve config groups and masks
MEDIUM: config: resolve relative threads on bind lines to absolute ones
MEDIUM: stick-table: never learn the "conn_cur" value from peers
---
