Hi, HAProxy 1.9-dev4 was released on 2018/10/21. It added 97 new commits after version 1.9-dev3.
There's not much user-visible here, it's mostly another merge of some pending infrastructure changes. The most sensitive changes consist in the finalization of the connection reorientation from top to bottom, so that we don't need the conn_want_* tricks from the upper layers nor the update_poll() calls anymore. Everything is attempted directly and a subscription to the lower layer is made in case of failure. The perf is slightly better than with dev3, but more importantly the code becomes much cleaner and straightforward. An optimization was made in the scheduler regarding the wait queues, most of which are lockfree now. Another one concerns the FD lock which is taken less often as well. All in all the overall multi-thread performance has increased quite a bit. I measured a gain of 60% over 1.8 using only H2 on 4 threads. A number of bugs in various areas were addressed (master-worker, rdv point, h2, streams). Some preliminary changes to the HTTP/1 engine were made. One of them concerns the connection header processing which revealed that some absurd cases were supported in the configuration, and were possibly working as expected for some users... depending on what they were expecting. These ones involved "option http-pretend-keepalive" in frontends and "option http-tunnel" in backends, both of which make no sense. These ones will now be ignored and will emit a warning if encountered. The support for TLS 1.3 ciphersuites was merged. If you play with it, please report successes or failures, as this was backported to 1.8. Regarding TLS, certificates can now be generated on the fly on BoringSSL as well. Build reports about warnings were all handled, and we tried to address all of them on gcc 3.4, 4.4, 4.7, 5.3, 6.4, 7.2, 8.1, as well as Clang 3.4, 3.8 and a very recent one that I forgot. The build now looks OK both on Linux and FreeBSD so that it's possible to add -Werror. By the way I wanted to add a Makefile option to enable -Werror easily and I forgot, it will be for later. Again, if you get some warnings, feel free to pass them on. Ah, I almost forgot, on systems featuring clock_gettime() with both CLOCK_THREAD_CPUTIME_ID and CLOCK_MONOTONIC (ie Linux >= 2.6.12 and maybe other OSes), "show activity" will indicate the number of milliseconds of CPU that were stolen to each thread by other processes or threads running on the machine. Normally this indicates improperly bound threads being parasited by something else. This is expected to help when some users observe very abnormal performance patterns when using threads. Regarding the next steps, we're not that bad in the end, thanks to the efforts of all participants. From what I've seen hopefully that next week we'll be able to take a first round of the new native HTTP representation in experimental state with some limitations. We should probably have a nice update on the master-worker model, and some updates on the cache. If we manage to get all this in good shape and merged soon, I'm willing to push the release date a bit further and have a longer stabilization phase so that we release 1.9 in better shape than 1.8, hoping that post-release bugs will have less impact and will not require as much energy to address. As usual, this development version is mostly aimed at developers, but I'm starting to be tempted to deploy it just to test it further, though I'll possibly do it with dev5 next week. Please don't put it on sensitive production yet if you want to play with it, as the finalization of the connection changes have already uncovered some old nasty corner cases, it could likely trigger a few other ones. Please find the usual URLs below : Site index : http://www.haproxy.org/ Discourse : http://discourse.haproxy.org/ Sources : http://www.haproxy.org/download/1.9/src/ Git repository : http://git.haproxy.org/git/haproxy.git/ Git Web browsing : http://git.haproxy.org/?p=haproxy.git Changelog : http://www.haproxy.org/download/1.9/src/CHANGELOG Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/ Have fun, Willy --- Complete changelog : Bertrand Jacquin (1): DOC: Fix a few typos Christopher Faulet (10): MINOR: http: Move comment about some HTTP macros in the right header file MINOR: stats: Add missing include MINOR: http: Export some functions and do cleanup to prepare HTTP refactoring MEDIUM: http: Ignore http-pretend-keepalive option on frontend MEDIUM: http: Ignore http-tunnel option on backend MINOR: http: Use same flag for httpclose and forceclose options MINOR: h1: Add EOH marker during headers parsing MINOR: conn-stream: Add CL_FL_NOT_FIRST flag MINOR: h1: Change the union h1_sl to use indirect strings to store infos MINOR: h1: Add the flag H1_MF_NO_PHDR to not add pseudo-headers during parsing Dirkjan Bussink (4): MEDIUM: ssl: add support for ciphersuites option for TLSv1.3 CLEANUP: haproxy: Remove unused variable CLEANUP: h1: Fix debug warnings for h1 headers CLEANUP: stick-tables: Remove unneeded double (()) around conditional clause Emeric Brun (2): BUG/MEDIUM: Cur/CumSslConns counters not threadsafe. BUG/MEDIUM: mworker: segfault receiving SIGUSR1 followed by SIGTERM. Emmanuel Hocdet (2): MINOR: ssl: cleanup old openssl API call MINOR: ssl: generate-certificates for BoringSSL Fabrice Fontaine (1): BUILD: Allow configuration of pcre-config path Ilya Shipitsin (1): BUG/MINOR: connection: avoid null pointer dereference in send-proxy-v2 Lukas Tribus (2): DOC: clarify force-private-cache is an option DOC: fix reference to map files in MAINTAINERS Olivier Houchard (19): BUG/MEDIUM: buffers: Make sure we don't wrap in ci_insert_line2/b_rep_blk. MINOR: connections: Introduce an unsubscribe method. MEDIUM: connections: Change struct wait_list to wait_event. BUG/MEDIUM: h2: Make sure we're not in the send list on flow control. BUG/MEDIUM: stream: Make sure to unsubscribe before si_release_endpoint. MINOR: server: Use memcpy() instead of strncpy(). MINOR: build: Disable -Wstringop-overflow. MINOR: cfgparse: Write 130 as 128 as 0x82 and 0x80. MINOR: peers: use defines instead of enums to appease clang. BUG/MEDIUM: pools: Fix the usage of mmap()) with DEBUG_UAF. BUILD: memory: fix free_list pointer declaration again for atomic CAS BUG/MEDIUM: h2: Close connection if no stream is left an GOAWAY was sent. BUG/MEDIUM: connections: Remove subscription if going in idle mode. BUG/MEDIUM: stream: Make sure polling is right on retry. MINOR: h2: Make sure to return 1 in h2_recv() when needed. MEDIUM: connections: Don't directly mess with the polling from the upper layers. MINOR: streams: Call tasklet_free() after si_release_endpoint(). MINOR: connection: Add a SUB_CALL_UNSUBSCRIBE event. MINOR: h2: Don't run tasks that are waiting to send if mux in full. PiBa-NL (1): REGTEST/MINOR: compatibility: use unix@ instead of abns@ sockets William Lallemand (1): BUG/MEDIUM: mworker: don't poll on LI_O_INHERITED listeners Willy Tarreau (51): REORG: http: move the code to different files REORG: http: move HTTP rules parsing to http_rules.c CLEANUP: http: remove some leftovers from recent cleanups BUILD: Makefile: add a "make opts" target to simply show the build options BUILD: Makefile: speed up compiler options detection BUG/MINOR: backend: check that the mux installed properly BUG/MEDIUM: h2: check that the connection is still valid at the end of init() BUG/MEDIUM: h2: make h2_stream_new() return an error on memory allocation failure MEDIUM: task: perform a single tree lookup per run queue batch BUG/MINOR: threads: move declaration of capabilities to config.h OPTIM: tools: optimize my_ffsl() for x86_64 MINOR: log: make sess_log() support sess=NULL MINOR: chunk: add chunk_cpy() and chunk_cat() MEDIUM: h2: stop relying on H2_SS_IDLE / H2_SS_CLOSED CLEANUP: h2: rename h2c_snd_settings() to h2c_send_settings() MINOR: h2: don't try to send data before preface MINOR: h2: unify the mux init function MINOR: h2: retrieve the front proxy from the caller instead of the session MINOR: h2: split h2c_stream_new() into h2s_new() + h2c_frt_stream_new() MINOR: h2: add a new flag to quickly distinguish front vs back connection BUG/MEDIUM: stream: don't crash on out-of-memory BUILD: compiler: add a new statement "__unreachable()" BUILD: lua: silence some compiler warnings about potential null derefs BUILD: ssl: fix null-deref warning in ssl_fc_cipherlist_str sample fetch BUILD: ssl: fix another null-deref warning in ssl_sock_switchctx_cbk() BUILD: stick-table: make sure not to fail on task_new() during initialization BUILD: peers: check allocation error during peers_init_sync() MINOR: tools: add a new function atleast2() to test masks for more than 1 bit MINOR: config: use atleast2() instead of my_popcountl() where relevant MEDIUM: fd/threads: only grab the fd's lock if the FD has more than one thread MAJOR: tasks: create per-thread wait queues OPTIM: tasks: group all tree roots per cache line MINOR: pools: allocate most memory pools from an array MINOR: pools: split pool_free() in the lockfree variant MEDIUM: pools: implement a thread-local cache for pool entries BUG/MEDIUM: threads: fix thread_release() at the end of the rendez-vous point Revert "BUILD: lua: silence some compiler warnings about potential null derefs" BUILD: lua: silence some compiler warnings about potential null derefs (#2) MINOR: lua: all functions calling lua_yieldk() may return BUILD: lua: silence some compiler warnings after WILL_LJMP BUILD: Makefile: silence an option conflict warning with clang CLEANUP: state-file: make the path concatenation code a bit more consistent MINOR: fd: centralize poll timeout computation in compute_poll_timeout() MINOR: poller: move time and date computation out of the pollers BUILD: memory: fix pointer declaration for atomic CAS BUILD: Makefile: add USE_RT to pass -lrt for clock_gettime() and friends MINOR: time: add now_mono_time() and now_cpu_time() MEDIUM: time: measure the time stolen by other threads BUILD: memory: fix free_list pointer declaration again for atomic CAS BUILD: compiler: rename __unreachable() to my_unreachable() MINOR: ebtree: save 8 bytes in struct eb32sc_node mildis (2): BUG/MINOR: h2: null-deref BUG/MINOR: checks: queues null-deref ---
