Hi,
HAProxy 2.2-dev4 was released on 2020/03/09. It added 124 new commits
after version 2.2-dev3.
That's another round of updates and cleanups accumulated over the last
two weeks. The updates mostly focused on 6 fronts this time:
- ACL: the unique-id generation used to be extremely slow (O(n^2)) and
take a lot of time to start when dealing with many ACL patterns. That
was reworked by Carl Henrik Lunde and is now typically 100+ times
faster. It will likely be backported to 2.0 once it has been shown not
to cause any side effect.
- Lua: Lua-declared actions can now yield, not in the Lua way but at
least in the haproxy way, in that, just like other actions, they may
signal they can't proceed and need to wait so that they are evaluated
again later. In addition it is now possible to write filters in Lua
so that it should require less ugly and unreliable tricks (don't ask
me how however, I haven't looked yet).
- SSL/CLI: dynamic update of certificates in crt-list used to be limited
to those without filters. Now those involving filters can be updated
as well using "set ssl cert".
- random/UUID: we've got some insightful complaints that UUID were far
from being unique in multi-process environments! And this is true,
since the randoms were poorly seeded and were not re-seeded after the
fork, due to the fact that randoms were only used to spread health
checks in the past. In addition I discovered that most random()
implementations were not even thread-safe. So we now have a pseudo-
random number generator that's thread safe and which properly uses
different seeds on different processes. This will be backported as
far as 2.0 where UUID was backported.
- polling: we still used to observe a performance degradation compared
to 1.7 on the number of syscalls used per requests on a connection
in keep-alive mode (there were several epollctl() calls per request).
These were significantly improved now so that we can now have less
calls by avoiding needlessly unregistering events. The difference is
visible in extreme scenarios where there were thousands times more
connections than run-queue-depth and these connections were
experiencing frequent requests, like when using very interactive
services.
- watchdog: the watchdog that detects deadlocks and kills a runaway
process used to depend both on Linux and threads. Now it also works
on FreeBSD and/or when threads are disabled. It's important since
there seems to be a fair number of users on FreeBSD, so we can now
improve the reliability there as well.
- backtraces: on operating systems where this is possible and relevant,
when the watchdog triggers, a call trace will be produced in best
effort. Till know we only used to know what task was running, this
was a bit limited and a number of backtraces still couldn't be
exploited without gdb and a core. Now at least on Linux/x86_64 and
arm64 by default, and FreeBSD/x86_64 when compiled with USE_BACKTRACE=1
we'll get a detailed backtrace with function names+offsets and/or
pointers. This should improve issue reports where known bugs will have
more chances of being recognized and this might help developers
understand the issue without bothering the reporter asking for a
core dump.
There's still quite some work to do before 2.2 (roughly 2.5 months ahead).
If you still have secret patches on your side, it's the last moment before
the window closes at the end of the month with dev5, after which only the
ongoing stuff will be merged. I noticed Tim's extensions to pass unique
IDs in PPv2 on the list, I'm also aware of some ongoing work on idle
connections and SSL, and I remember that a few other less impacting points
were discussed, such as syslog over TCP and a few adjustments to errorfile
and return directives.
Now let's beat it hard.
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse : http://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Sources : http://www.haproxy.org/download/2.2/src/
Git repository : http://git.haproxy.org/git/haproxy.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy.git
Changelog : http://www.haproxy.org/download/2.2/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
Willy
---
Complete changelog :
Bjoern Jacke (1):
DOC: fix typo about no-tls-tickets
Björn Jacke (1):
DOC: improve description of no-tls-tickets
Carl Henrik Lunde (1):
OPTIM: startup: fast unique_id allocation for acl.
Christopher Faulet (30):
MINOR: contrib/prometheus-exporter: Add the last heathcheck duration
metric
BUG/MINOR: http-htx: Do case-insensive comparisons on Host header name
MINOR: mux-h1: Remove useless case-insensitive comparisons
MINOR: buf: Add function to insert a string at an absolute offset in a
buffer
MINOR: htx: Add a function to return a block at a specific offset
MINOR: htx: Use htx_find_offset() to truncate an HTX message
MINOR: flt_trace: Use htx_find_offset() to get the available payload
length
BUG/MINOR: filters: Use filter offset to decude the amount of forwarded
data
BUG/MINOR: filters: Forward everything if no data filters are called
BUG/MEDIUM: cache/filters: Fix loop on HTX blocks caching the response
payload
BUG/MEDIUM: compression/filters: Fix loop on HTX blocks compressing the
payload
BUG/MINOR: http-ana: Reset request analysers on a response side error
BUG/MINOR: lua: Abort when txn:done() is called from a Lua action
BUG/MINOR: lua: Ignore the reserve to know if a channel is full or not
MINOR: lua: Add function to know if a channel is a response one
MINOR: lua: Stop using the lua txn in hlua_http_get_headers()
MINOR: lua: Stop using the lua txn in hlua_http_rep_hdr()
MINOR: lua: Stop using lua txn in hlua_http_del_hdr() and
hlua_http_add_hdr()
MINOR: lua: Remove the flag HLUA_TXN_HTTP_RDY
MINOR: lua: Rename hlua_action_wake_time() to hlua_set_wake_time()
BUG/MINOR: lua: Init the lua wake_time value before calling a lua function
BUG/MINOR: http-rules: Return ACT_RET_ABRT to abort a transaction
BUG/MINOR: http-rules: Preserve FLT_END analyzers on reject action
BUG/MINOR: http-rules: Fix a typo in the reject action function
MINOR: cache/filters: Initialize the cache filter when stream is created
MINOR: compression/filters: Initialize the comp filter when stream is
created
BUG/MINOR: rules: Preserve FLT_END analyzers on silent-drop action
BUG/MINOR: rules: Return ACT_RET_ABRT when a silent-drop action is
executed
BUG/MINOR: rules: Increment be_counters if backend is assigned for a
silent-drop
BUG/MINOR: http-rules: Abort transaction when a redirect is applied on
response
Emmanuel Hocdet (5):
MINOR: ssl: move find certificate chain code to its own function
MINOR: ssl: resolve issuers chain later
MINOR: ssl: resolve ocsp_issuer later
MINOR: ssl/cli: "show ssl cert" command should print the "Chain Filename:"
MINOR: ssl: add "ca-verify-file" directive
Ilya Shipitsin (5):
BUILD: cirrus-ci: suppress OS version check when installing packages
DOC: configuration.txt: fix various typos
DOC: assorted typo fixes in the documentation and Makefile
BUILD: cirrus-ci: get rid of unstable freebsd images
DOC: assorted typo fixes in the documentation
Jerome Magnin (1):
BUG/MINOR: http_ana: make sure redirect flags don't have overlapping bits
Lukas Tribus (1):
BUG/MINOR: dns: ignore trailing dot
Miroslav Zagorac (2):
CLEANUP: contrib/spoa_example: Fix several typos
CLEANUP: remove unused code in 'my_ffsl/my_flsl' functions
Tim Duesterhus (13):
BUG/MINOR: sample: Make sure to return stable IDs in the unique-id fetch
REGTEST: Add unique-id reg-test
MINOR: stream: Add stream_generate_unique_id function
MINOR: stream: Use stream_generate_unique_id
MINOR: ist: Add `IST_NULL` macro
MINOR: ist: Add `int isttest(const struct ist)`
MINOR: ist: Add `struct ist istalloc(size_t)` and `void istfree(struct
ist*)`
CLEANUP: Use `isttest()` and `istfree()`
MINOR: ist: Add `struct ist istdup(const struct ist)`
MINOR: proxy: Make `header_unique_id` a `struct ist`
MEDIUM: stream: Make the `unique_id` member of `struct stream` a `struct
ist`
BUG/MAJOR: proxy_protocol: Properly validate TLV lengths
CLEANUP: proxy_protocol: Use `size_t` when parsing TLVs
William Lallemand (3):
BUG/MEDIUM: ssl: chain must be initialized with sk_X509_new_null()
MINOR: ssl/cli: support crt-list filters
MINOR: ssl: reach a ckch_store from a sni_ctx
Willy Tarreau (61):
MEDIUM: buffer: remove the buffer_wq lock
BUG/MINOR: h2: reject again empty :path pseudo-headers
MINOR: wdt: always clear sigev_value to make valgrind happy
MINOR: epoll: always initialize all of epoll_event to please valgrind
CLEANUP: fd: remove the FD_EV_STATUS aggregate
CLEANUP: fd: remove some unneeded definitions of FD_EV_* flags
MINOR: fd: merge the read and write error bits into RW error
MINOR: rawsock: always mark the FD not ready when we're certain it happens
MEDIUM: connection: make the subscribe() call able to wakeup if ready
MEDIUM: connection: don't stop receiving events in the FD handler
MEDIUM: mux-h1: do not blindly wake up the tasklet at end of request
anymore
BUG/MINOR: arg: don't reject missing optional args
MINOR: tools: make sure to correctly check the returned 'ms' in
date2std_log
MINOR: debug: report the task handler's pointer relative to main
BUG/MEDIUM: debug: make the debug_handler check for the thread in
threads_to_dump
MINOR: haproxy: export main to ease access from debugger
MINOR: haproxy: export run_poll_loop
MINOR: task: export run_tasks_from_list
BUILD: tools: remove obsolete and conflicting trace() from standard.c
MINOR: tools: add new function dump_addr_and_bytes()
MINOR: tools: add resolve_sym_name() to resolve function pointers
MINOR: debug: use resolve_sym_name() to dump task handlers
MINOR: cli: make "show fd" rely on resolve_sym_name()
MEDIUM: debug: add support for dumping backtraces of stuck threads
MINOR: debug: call backtrace() once upon startup
BUG/MINOR: wdt: do not return an error when the watchdog couldn't be
enabled
BUILD: Makefile: include librt before libpthread
MEDIUM: wdt: fall back to CLOCK_REALTIME if CLOCK_THREAD_CPUTIME is not
available
MINOR: wdt: do not depend on USE_THREAD
MINOR: debug: report the number of entries in the backtrace
MINOR: debug: improve backtrace() on aarch64 and possibly other systems
MINOR: debug: use our own backtrace function on clang+x86_64
MINOR: debug: dump the whole trace if we can't spot the starting point
BUILD: tools: unbreak resolve_sym_name() on non-GNU platforms
BUILD: tools: rely on __ELF__ not USE_DL to enable use of dladdr()
BUILD: makefile: do not modify the build options during make reg-tests
BUG/MEDIUM: connection: stop polling for sending when the event is ready
MEDIUM: stream-int: make sure to try to immediately validate the
connection
MINOR: tcp/uxst/sockpair: only ask for I/O when really waiting for a
connect()
MEDIUM: connection: only call ->wake() for connect() without I/O
OPTIM: connection: disable receiving on disabled events when the run
queue is too high
OPTIM: mux-h1: subscribe rather than waking up at a few other places
BUG/MINOR: connection/debug: do not enforce !event_type on subscribe()
anymore
DOC: fix incorrect indentation of http_auth_*
BUG/MINOR: ssl-sock: do not return an uninitialized pointer in
ckch_inst_sni_ctx_to_sni_filters
MINOR: debug: add CLI command "debug dev write" to write an arbitrary size
BUG/MINOR: init: make the automatic maxconn consider the max of soft/hard
limits
BUILD: buffer: types/{ring.h,checks.h} should include buf.h, not buffer.h
BUILD: ssl: include mini-clist.h
BUILD: global: must not include common/standard.h but only
types/freq_ctr.h
BUILD: freq_ctr: proto/freq_ctr needs to include common/standard.h
BUILD: listener: types/listener.h must not include standard.h
BUG/MEDIUM: random: initialize the random pool a bit better
BUG/MEDIUM: random: implement per-thread and per-process random sequences
Revert "BUG/MEDIUM: random: implement per-thread and per-process random
sequences"
MINOR: tools: add 64-bit rotate operators
BUG/MEDIUM: random: implement a thread-safe and process-safe PRNG
MINOR: backend: use a single call to ha_random32() for the random LB algo
BUG/MINOR: checks/threads: use ha_random() and not rand()
MINOR: sample: make all bits random on the rand() sample fetch
MINOR: tools: add a generic function to generate UUIDs
---
