HAProxy 1.9.7 was released on 2019/04/25. It added 100 new commits after version

After a month since the last release, a huge number of bugs were addressed into
this release. The most significant are fixes of 100% CPU usage issues. Olivier
and Willy worked hard on it. They fixed several bugs in the task scheduler and
improved it. Another issue concerned the streams and the way aborts was
handled. The flag reporting the abort on the stream-interface was cleared too
early. So the stream was not properly released, looping infinitely on
itself. Thanks to Maciej for helping Olivier investigating on this
issue. Finally, two bugs leading to a loop were fixed in the H2 multiplexer. The
first one occurred when an incomplete frame was received and no more data was
available. The second one when the subscription to the send events was made
twice, which should never happen. This last issue was also able to lead to
random crashes.

Willy, with the help of Maksim, fixed a locking problem in the round-robin LB
causing occasional crashes of HAProxy because of a divide by zero when the next
server's position was computed. Willy also fixed an AB/BA locking issue in the
listener's accept code thanks to Olivier and his self-locked list. At the end,
all this part was reworked. The lock on the listener queue disappeared and the
accept code was significantly improved. As mentioned, Olivier implemented
self-locked lists. It is now safe to manipulate such lists with multiple threads
without having to worry about concurrency issues.

A very old bug on how HTTP sample fetches work was fixed. All HTTP sample
fetches were buggy because the channel used was chosen depending on the sample
direction and not on the keyword really used. The request channel was used when
called during the request analysis and the response one was used when called
during the response analysis, regardless the sample really called. It could
cause a whole bunch of bugs, from undefined behavior because the data were
extracted from the wrong buffer to crash of HAProxy.

Many bugs in the HTX part were fixed. The most significant was a problem of data
corruption when an header was added while some data were already received. It
was addressed by defragmenting the HTX message. It is a quick and dirty fix,
waiting for a better solution. The TCP splicing was fixed in the H1
multiplexer. The handling of the CONNECT method was refactored to work when h1
and h2 are mixed (an h1 client sending requests to an h2 server or this
opposite). And finally, the option "http-tunnel" was deprecated in HTX. It is
still supported in legacy HTTP, but in HTX, this options is now ignored.

Otherwise, as always, there was a bunch of bugfixes and changes here and
there. The most noticeable are:

   - A memory leak was fixed in the map converter. And now when a map file is
     loaded, the default value is parsed only when it is present. This fixes
     segfaults at parsing time when no default value is provided.

   - Pattern IDs are now assigned after checking the config validity. It fixes a
     bug where some map identifiers were not assigned (appearing as -1 in show
     map). Thanks to Pavlos to report this bug.

   - Several bugs was fixed on the SPOE, mainly on the way fragmented frames was
     handled internally.

   - A bug was fixed in the peers. Peer sessions were not always cleanly reset
     on release, resulting in a bad state for new sessions. In addition, the CLI
     command "show peers" was added. It dumps info about the peers configured in
     "peers" sections.

   - Nenad fixed a security issue in the TLS ticket key rotation. Whenever
     HAProxy was reloaded with rotated keys, the resumption would be broken for
     previous encryption key. CVE-2019-11323 was assigned to this bug.

   - Build of HAProxy on AIX 5.1 was fixed.

   - Some small improvements were brought to the master-worker mode. The busy
     polling was disabled on the master process to not waste a dedicated
     processor on it. And an unambiguous status is now set on exit.

   - The global directive "set-dumpable" was added to enable core dumps. It's
     always a pain to get a core dump when enabling user/group setting (which
     disables the dumpable flag on Linux), when using a chroot and/or when
     HAProxy is started by a service management tool which requires complex
     operations to just raise the core dump limit. It becomes much easier to
     retrieve a core file by using this new directive. Temporarily moving the
     chroot to a user-writable place generally enough.

   - Missing locks was added in set-map and add-acl HTTP rules.

   - The multithreading mode is now enabled on wurlf. And it is now compatible
     with the new buffer API, so it builds again on 1.9.

WARNING: Device detection modules (51Degrees, DeviceAtlas and Wurfl) are not
compatible with the HTX mode. Respective maintainers works on it. But for this
release, don't enable the HTX if you need to use one of them.

There is still an unresolved issue when the SSL is heavily used with multiple
threads. It is unclear for now what the problem is, but an extra latency has
been observed by a user. After digging, we suspect a problem with OpenSSL. You
can have more details on this issue on Github (issue #67).

Please find the usual URLs below :
    Site index       : http://www.haproxy.org/
    Discourse        : http://discourse.haproxy.org/
    Slack channel    : https://slack.haproxy.org/
    Issue tracker    : https://github.com/haproxy/haproxy/issues
    Sources          : http://www.haproxy.org/download/1.9/src/
    Git repository   : http://git.haproxy.org/git/haproxy-1.9.git/
    Git Web browsing : http://git.haproxy.org/?p=haproxy-1.9.git
    Changelog        : http://www.haproxy.org/download/1.9/src/CHANGELOG
    Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/

Complete changelog :
Christopher Faulet (28):
       BUG/MINOR: htx: Preserve empty HTX messages with an unprocessed parsing 
       BUG/MEDIUM: spoe: Queue message only if no SPOE applet is attached to 
the stream
       BUG/MEDIUM: spoe: Return an error if nothing is encoded for fragmented 
       BUG/MINOR: spoe: Be sure to set tv_request when each message fragment is 
       BUG/MEDIUM: htx: Defrag if blocks position is changed and the payloads 
       BUG/MEDIUM: htx: Don't crush blocks payload when append is done on a 
data block
       MEDIUM: htx: Deprecate the option 'http-tunnel' and ignore it in HTX
       MINOR: proto_htx: Don't adjust transaction mode anymore in HTX analyzers
       BUG/MEDIUM: htx: Fix the process of HTTP CONNECT with h2 connections
       MINOR: mux-h1: Simplify handling of 1xx responses
       BUG/MINOR: mux-h1: Handle the flag CS_FL_KILL_CONN during a shutdown 
       BUG/MINOR: mux-h1: Process input even if the input buffer is empty
       BUG/MINOR: mux-h1: Don't switch the parser in busy mode if other side 
has done
       BUG/MEDIUM: mux-h1: Notify the stream waiting for TCP splicing if ibuf 
is empty
       BUG/MEDIUM: mux-h1: Enable TCP splicing to exchange data only
       MINOR: mux-h1: Handle read0 during TCP splicing
       BUG/MEDIUM: htx: Don't return the start-line if the HTX message is empty
       BUG/MAJOR: http_fetch: Get the channel depending on the keyword used
       BUG/MINOR: http_fetch/htx: Allow permissive sample prefetch for the HTX
       BUG/MEDIUM: h1: Don't parse chunks CRLF if not enough data are available
       BUG/MEDIUM: thread/http: Add missing locks in set-map and add-acl HTTP 
       BUG/MEDIUM: stream: Don't request a server connection if a shutw was 
       BUG/MINOR: 51d: Get the request channel to call 
       BUG/MINOR: da: Get the request channel to call CHECK_HTTP_MESSAGE_FIRST()
       BUG/MEDIUM: stream: Fix the way early aborts on the client side are 
       BUG/MINOR: spoe: Don't systematically wakeup SPOE stream in the applet 
       BUG/MAJOR: muxes: Use the HTX mode to find the best mux for HTTP proxies 
       BUG/MINOR: htx: Exclude TCP proxies when the HTX mode is handled during 

David Carlier (1):
       BUILD/MINOR: listener: Silent a few signedness warnings.

Emeric Brun (3):
       BUG/MEDIUM: peers: fix a case where peer session is not cleanly reset on 
       MINOR: peers: adds counters on show peers about tasks calls.
       MINOR: ssl/cli: async fd io-handlers printable on show fd

Frédéric Lécaille (2):
       MINOR: peers: Add a new command to the CLI for peers.
       DOC: update for "show peers" CLI command.

Ilya Shipitsin (1):
       REGTESTS: exclude tests that require ssl, pcre if no such feature is 

Nenad Merdanovic (2):
       BUG/MEDIUM: map: Fix memory leak in the map converter
       BUG/MINOR: ssl: Fix 48 byte TLS ticket key rotation

Olivier Houchard (15):
       BUG/MEDIUM: h2: Don't attempt to recv from h2_process_demux if we 
       BUG/MEDIUM: streams: Don't remove the SI_FL_ERR flag in si_update_both().
       BUG/MEDIUM: streams: Store prev_state before calling si_update_both().
       BUG/MEDIUM: stream: Don't clear the stream_interface flags in 
       BUG/MEDIUM: streams: Only re-run process_stream if we're in a connected 
       BUG/MEDIUM: stream_interface: Don't bother doing chk_rcv/snd if not 
       MINOR: lists: Implement locked variations.
       BUG/MEDIUM: lists: Properly handle the case we're removing the first elt.
       MINOR: initcall: Don't forget to define the __start/stop_init_##stg 
       BUG/MEDIUM: h2: Make sure we're not already in the send_list in 
       BUG/MEDIUM: tasks: Make sure we modify global_tasks_mask with the 
       MINOR: tasks: Don't consider we can wake task with tasklet_wakeup().
       MEDIUM: tasks: No longer use rq.node.leaf_p as a lock.
       MINOR: tasks: Don't set the TASK_RUNNING flag when adding in the tasklet 
       BUG/MEDIUM: applets: Don't use task_in_rq().

Robin H. Johnson (1):
       MINOR: skip get_gmtime where tm is unused

William Lallemand (4):
       BUG/MINOR: cli: correctly handle abns in 'show cli sockets'
       MINOR: cli: start addresses by a prefix in 'show cli sockets'
       BUG/MINOR: mworker: don't exit with an ambiguous value
       BUG/MINOR: mworker: ensure that we still quits with SIGINT

Willy Tarreau (40):
       BUILD: makefile: work around an old bug in GNU make-3.80
       BUILD: http: properly mark some struct as extern
       BUILD: chunk: properly declare pool_head_trash as extern
       BUILD: cache: avoid a build warning with some compilers/linkers
       MINOR: tools: make memvprintf() never pass a NULL target to vsnprintf()
       BUILD: re-implement an initcall variant without using executable sections
       BUILD: makefile: fix build of IPv6 header on aix51
       BUILD: makefile: add _LINUX_SOURCE_COMPAT to build on AIX-51
       BUILD: Makefile: disable shared cache on AIX 5.1
       BUILD: fix backport of initcall variant
       BUILD: use inttypes.h instead of stdint.h
       BUILD: connection: fix naming of ip_v field
       BUG/MEDIUM: htx: fix random premature abort of data transfers
       BUG/MEDIUM: pattern: assign pattern IDs after checking the config 
       BUG/MEDIUM: task/threads: address a fairness issue between local and 
global tasks
       BUG/MINOR: tasks: make sure the first task to be queued keeps its nice 
       BUG/MINOR: threads: fix the process range of thread masks
       BUG/MEDIUM: list: fix the rollback on addq in the locked liss
       BUG/MEDIUM: list: fix LIST_POP_LOCKED's removal of the last pointer
       BUG/MEDIUM: list: add missing store barriers when updating elements and 
       MINOR: list: make the delete and pop operations idempotent
       BUG/MEDIUM: list: correct fix for LIST_POP_LOCKED's removal of last 
       BUG/MEDIUM: list: fix again LIST_ADDQ_LOCKED
       BUG/MEDIUM: list: fix incorrect pointer unlocking in LIST_DEL_LOCKED()
       MAJOR: listener: do not hold the listener lock in listener_accept()
       BUG/MEDIUM: listener: use a self-locked list for the dequeue lists
       BUG/MEDIUM: listener: make sure the listener never accepts too many conns
       BUILD: htx: fix a used uninitialized warning on is_cookie2
       BUG/MAJOR: lb/threads: fix insufficient locking on round-robin LB
       MEDIUM: tasks: improve fairness between the local and global queues
       BUILD: task/thread: fix single-threaded build of task.c
       MEDIUM: tasks: only base the nice offset on the run queue depth
       MINOR: tasks: restore the lower latency scheduling when niced tasks are 
       BUG/MEDIUM: tasks: Make sure we set TASK_QUEUED before adding a task to 
the rq.
       BUG/MAJOR: task: make sure never to delete a queued task
       BUG/MINOR: mworker: disable busy polling in the master process
       BUG/MEDIUM: maps: only try to parse the default value when it's present
       BUG/MINOR: acl: properly detect pattern type SMP_T_ADDR
       MINOR: init: add a "set-dumpable" global directive to enable core dumps
       BUG/MAJOR: lb/threads: fix AB/BA locking issue in round-robin LB

paulborile (3):
       BUILD: wurfl: build fix for 1.9/2.0 code base
       MINOR: wurfl: enabled multithreading mode
       DOC: wurfl: added point of contact in MAINTAINERS file

Christopher Faulet

Reply via email to