Hi, HAProxy 2.0.9 was released on 2019/11/15. It added 33 new commits after version 2.0.8.
Several problematic bugs still affecting 2.0 were found since 2.0.8 thus it's better to get rid of them now before everyone has already updated and has to do it again. The main one affects the way outgoing connections are validated. We used to face several issues when dealing with retries during the development of 2.0, which have stacked upon each other until we figured they were wrong. Indeed, Christopher found a case where haproxy could enter an endless loop while trying to perform a connection retry after a protocol failure (typically try to speak H2 to a server responding in H1). Well, the dog was watching, quickly biting that offending loop, but still... Another one concerns idle connections with threads. There is a very difficult to meet but definitely present race in the code closing a session and releasing the last connection to a server form this session. We managed to reproduce it by mixing queues, random server errors and server session terminations, all at maximum rate. The result is a double free of a struct srv_list which crashes haproxy. It was also reported that splicing was broken with chunked encoding, and this revealed that we have a bit more complex work to do for 2.2 to fix it. For now it's simply disabled for chunked encoding, which is rarely noticeable in practice since most often, chunks do not come large enough to enable dynamic splicing. Once in a while, someone reports that one (or a few) thread eats 100% CPU mostly in system, showing an strace output in which it's visible that epoll_wait() reports activity for a listener but nothing is done. This bug was finally identified, it could happen when at least two distinct listeners are used to fill the process' connection limit. In this case, the one which has reached saturation last would return without disabling itself, and be called again immediately. Note that in such a case, the CPU usage is just a byproduct of some limit already being reached, but it would definitely make the troubleshooting harder. Connection retries over H2 connections experiencing a failed handshake or a GOAWAY frame were not possible because the data had already left. This was now fixed. The rest is a bit less important and has less impact. For those running on 2.1-dev, no need to downgrade, I'm going to issue another 2.1-dev ASAP. Please find the usual URLs below : Site index : http://www.haproxy.org/ Discourse : http://discourse.haproxy.org/ Slack channel : https://slack.haproxy.org/ Issue tracker : https://github.com/haproxy/haproxy/issues Sources : http://www.haproxy.org/download/2.0/src/ Git repository : http://git.haproxy.org/git/haproxy-2.0.git/ Git Web browsing : http://git.haproxy.org/?p=haproxy-2.0.git Changelog : http://www.haproxy.org/download/2.0/src/CHANGELOG Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/ Last minute note, for those tracking the git repo, I messed up my initial git push and had to do it again in force. Sorry about this. Thus do not worry in case one automated git-pull script reports an error, the error was on my side. Willy --- Complete changelog : Baptiste Assmann (2): BUG/MINOR: action: do-resolve now use cached response BUG: dns: timeout resolve not applied for valid resolutions Christopher Faulet (7): BUG/MINOR: mux-h2: Don't pretend mux buffers aren't full anymore if nothing sent BUG/MAJOR: stream-int: Don't receive data from mux until SI_ST_EST is reached BUG/MEDIUM: mux-h1: Disable splicing for chunked messages BUG/MEDIUM: stream: Be sure to support splicing at the mux level to enable it BUG/MEDIUM: stream: Be sure to release allocated captures for TCP streams BUG/MEDIUM: filters: Don't call TCP callbacks for HTX streams BUG/MINOR: mux-h1: Don't set CS_FL_EOS on a read0 when receiving data to pipe Joao Morais (1): BUG/MINOR: config: Update cookie domain warn to RFC6265 Jérôme Magnin (2): DOC: management: document reuse and connect counters in the CSV format DOC: management: document cache_hits and cache_lookups in the CSV format Lukas Tribus (1): MINOR: doc: http-reuse connection pool fix Olivier Houchard (4): MINOR: mux: Add a new method to get informations about a mux. BUG/MEDIUM: stream_interface: Only use SI_ST_RDY when the mux is ready. BUG/MEDIUM: servers: Only set SF_SRV_REUSED if the connection if fully ready. BUG/MEDIUM: Make sure we leave the session list in session_free(). William Dauchy (1): MINOR: tcp: avoid confusion in time parsing init William Lallemand (1): BUG/MINOR: cli: don't call the kw->io_release if kw->parse failed Willy Tarreau (14): MINOR: config: warn on presence of "\n" in header values/replacements BUG/MINOR: mux-h2: do not emit logs on backend connections BUG/MINOR: spoe: fix off-by-one length in UUID format string BUG/MEDIUM: mux-h2: report no available stream on a connection having errors BUG/MEDIUM: mux-h2: immediately remove a failed connection from the idle list BUG/MEDIUM: mux-h2: immediately report connection errors on streams DOC: management: fix typo on "cache_lookups" stats output BUG/MINOR: queue/threads: make the queue unlinking atomic BUG/MEDIUM: listeners: always pause a listener on out-of-resource condition CLEANUP: session: slightly simplify idle connection cleanup logic MINOR: memory: also poison the area on freeing BUILD: contrib/da: remove an "unused" warning BUG/MINOR: log: limit the size of the startup-logs [RELEASE] Released version 2.0.9 ---
