Hi all! Give some food, drink, disk space, a quiet place, a keyboard and Git to a developer and he will code forever... All good things must come to an end, and I decided that it had to be right now. After all, we initially announced the end of developments around end of September and a release around October or November. We've missed the September deadline, but I know by experience that nobody notices missed deadlines as long as they are not missed by more than one month. And one month ends today.
So what do we have here ? A *preview* of what the final 1.8 will be like. As a reminder, our -rc don't mean they're totally ready, but that we're really done with development and only performing the last adjustments, fixes, cleanups, documentation updates etc and that NEW FEATURES ARE NOT WELCOME ANYMORE FOR THIS VERSION. There are still some issues (I found a few after the release) but overall there's enough in various areas to satisfy the curious. And we needed to merge our various branches to take the time to fix conflicts and adapt our respective code (mainly to threads, which were merged first). In practice, since 1.8-dev3, the following main features were merged : - multi-thread support : for certain workloads like massive SSL rekeying it provides comparable performance to multi-process, but with all the load handled in a single process, hence single health checks and server states, single stats, single CLI etc. There are some known scalability limitations coming from arbitrations we had to do for 1.8 and which we will address during 1.9 (and maybe some before 1.8-final). The feature is enabled by default on linux2628 and freebsd, and disabled by default on other targets, though it can be forced enabled using USE_THREAD=1 or forced disabled using USE_THREAD= (empty string). Once enabled, it's possible to start several threads in the configuration using the "nbthread" directive in the global section. It's possible to force to map threads to CPUs but we're not completely satisfied with the current configuration options and will be working on them (any feedback is welcome). Another point is that we tried to start to take a look at the device detection extensions and it didn't appear trivial enough to fit in the tight schedule so this was postponed. Their maintainers are welcome to take a look as they know this much better than anyone else, and the thread developers will be happy to give some help on the subject. Important note: I just found that health-checks are totally broken (recursive locks and missed unlocks), so please disable checks when testing threads for now. We'll see how to address this. - small object cache : this is what I've been calling the "favicon cache" for several years now. The idea is always the same, when haproxy is deployed in front of a slow application server having to deliver a few small static objects, it costs a lot to this server to deliver them. A small cache of a few megabytes caching small objects for a short time with zero administration definitely helps here. As we don't want this cache to cause trouble, it's pessimistic : if any risk is suspected, an object is not cached ; and by default it caches for a short time (1 minute I believe) so that even after a failed deployment, it takes less time to wait than to wake up the LB admin to clear the cache. To be very clear, this is not meant to replace any real cache you might already be using. Maybe it could offload it from some dumb files at best. It's only meant to improve the situation for those not having a cache. The cache is enabled using "http-request cache-use" and "http-response cache-store" directives like below (the doc will come soon) : listen frt mode http http-response cache-store foobar http-request cache-use foobar cache foobar total-max-size 4 # size in megabytes - client-facing HTTP/2 : that's HTTP/2 support on the frontend. It's much better after the completely new rewrite than the first attempt a few months ago, and in the end I'm really happy with the outcome despite the pain it was. It's now almost complete, it supports POST, 1xx responses, chunked responses. In fact it's easier to enumerate what it does not support : CONTINUATION frames are not yet implemented (I may have found how to do it), PRIORITY frames are ignored (that's allowed by the spec), there's no equivalent of chunked encoding in requests, and trailers from the response are discarded. What it needs most is real world exposure now to spot corner cases. I wanted to place it on haproxy.org until I failed on the thread problem described above and had to revert. In order to enable it, simply add "alpn http/1.1,h2" on a "bind" line present in an HTTP mode frontend, and if the client advertises ALPN "h2" it will be used, otherwise it will fall back to HTTP/1.1. You'll need to have 16kB or larger buffers (that's the default value and may be adjusted using tune.bufsize). It's only supported over TLS for now. Note, on openssl versions before 1.0.2, you need to use "npn" instead of "alpn", but I don't know how long browsers will support it. - TLS1.3 0-RTT done right : while there is a lot of demand for 0-RTT support everywhere (which people often just call TLS1.3 by the way), it's important to remember that by default 0-RTT makes TLS vulnerable to replay attacks and that the upper protocol must take care of this. The IETF HTTP working group has been working on a draft to define how to safely map HTTP on top of 0-RTT to provide both performance and security (https://tools.ietf.org/html/draft-ietf-httpbis-replay-00). HAProxy 1.8-rc1 is the first server-side component to implement this draft, and we'll soon run some interoperability tests with a well known browser which just implemented it as well on the client side. This can be enabled using openssl-1.1.1 (still in development as well) and the "allow-0rtt" directive on the bind line. That's more or less all for the user-visible changes, the rest are important infrastructure changes supporting this work. That's why it was important to merge right now, so that we all have a common base to start sorting out the various issues that will inevitably emerge, and finish the documentation and the configuration mechanisms. I'm not listing all the things that were already merged in earlier versions (JSON stats, server-templates, SRV records etc), I'll try to recap everything for the release. If you have suggestions (especially on the configuration perspective), feel free to suggest ideas here on the list so that everyone can participate (and please use a different thread to address different topics so that participants are not forced to receive the parts they're not interested in). Ah last thing. Yes I know it's fun to deploy a fresh new load balancer in production, and I was about to do it on haproxy.org. I was lucky it took only 2 seconds to enter an endless loop, it's less fun if it happens when you're driving on your way back home. So use with care, on dedicated servers only, with non-critical traffic only, and keep an eye on it. Do not hesitate to report strange things here, we all know it encourages the "me too" behaviour. I hope we'll be able to quickly sort out the thread+check problem and emit an rc2, in the worst case in a week or two. Given that the code merge was *much* faster than anticipated, I wouldn't be surprized if we can release 1.8 by end of November with everything working (=nobody knows how to break it yet). Please find the usual URLs below : Site index : http://www.haproxy.org/ Discourse : http://discourse.haproxy.org/ Sources : http://www.haproxy.org/download/1.8/src/ Git repository : http://git.haproxy.org/git/haproxy-1.8.git/ Git Web browsing : http://git.haproxy.org/?p=haproxy-1.8.git Changelog : http://www.haproxy.org/download/1.8/src/CHANGELOG Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/ Have fun, Willy -- feeling exhausted like a marathoner :-) --- Complete changelog : Baptiste Assmann (1): MINOR: lua: add uuid to the Class Proxy Christopher Faulet (69): BUG/MINOR: spoe: Don't compare engine name and SPOE scope when both are NULL BUG/MINOR: spoa: Update pointer on the end of the frame when a reply is encoded MINOR: action: Add trk_idx inline function MINOR: action: Use trk_idx instead of tcp/http_trk_idx MINOR: action: Add a function pointer in act_rule struct to check its validity MINOR: action: Add function to check rules using an action ACT_ACTION_TRK_* MINOR: action: Add a functions to check http capture rules MINOR: action: Factorize checks on rules calling check_ptr if defined MINOR: acl: Pass the ACLs as an explicit parameter of build_acl_cond MEDIUM: spoe: Add support of ACLS to enable or disable sending of SPOE messages MINOR: spoe: Check uniqness of SPOE engine names during config parsing MEDIUM: spoe: Parse new "spoe-group" section in SPOE config file MEDIUM: spoe/rules: Add "send-spoe-group" action for tcp/http rules MINOR: spoe: Move message encoding in its own function MINOR: spoe: Add a type to qualify the message list during encoding MINOR: spoe: Add a generic function to encode a list of SPOE message MEDIUM: spoe/rules: Process "send-spoe-group" action BUG/MINOR: dns: Fix CLI keyword declaration MAJOR: dns: Refactor the DNS code BUG/MINOR: mailers: Fix a memory leak when email alerts are released MEDIUM: mailers: Init alerts during conf parsing and refactor their processing MINOR: mailers: Use pools to allocate email alerts and its tcpcheck_rules MINOR: standard: Add memvprintf function MINOR: log: Save alerts and warnings emitted during HAProxy startup MINOR: cli: Add "show startup-logs" command MINOR: startup: Extend the scope the MODE_STARTING flag MINOR: threads: Add THREAD_LOCAL macro MEDIUM: threads: Add hathreads header file MINOR: threads: Add mechanism to register per-thread init/deinit functions MINOR: threads: Add nbthread parameter MEDIUM: threads: Adds a set of functions to handle sync-point MAJOR: threads: Start threads to experiment multithreading MINOR: threads: Define the sync-point inside run_poll_loop MEDIUM: threads/buffers: Define and register per-thread init/deinit functions MEDIUM: threads/chunks: Transform trash chunks in thread-local variables MEDIUM: threads/time: Many global variables from time.h are now thread-local MEDIUM: threads/logs: Make logs thread-safe MEDIUM: threads/pool: Make pool thread-safe by locking all access to a pool MAJOR: threads/fd: Make fd stuffs thread-safe MINOR: threads/fd: Add a mask of threads allowed to process on each fd in fdtab array MEDIUM: threads/fd: Initialize the process mask during the call to fd_insert MINOR: threads/fd: Process cached events of FDs depending on the process mask MINOR: threads/polling: pollers now handle FDs depending on the process mask WIP: SQUASH WITH SYNC POINT MEDIUM: threads/signal: Add a lock to make signals thread-safe MEDIUM: threads/listeners: Make listeners thread-safe MEDIUM: threads/proxy: Add a lock per proxy and atomically update proxy vars MEDIUM: threads/server: Make connection list (priv/idle/safe) thread-safe MEDIUM: threads/server: Add a lock per server and atomically update server vars MINOR: threads/server: Add a lock to deal with insert in updates_servers list MEDIUM: threads/lb: Make LB algorithms (lb_*.c) thread-safe MEDIUM: threads/queue: Make queues thread-safe MEDIUM: threads/freq_ctr: Make the frequency counters thread-safe MEDIUM: thread/vars: Make vars thread-safe MEDIUM: threads/filters: Add init/deinit callback per thread MINOR: threads/filters: Update trace filter to add _per_thread callbacks MEDIUM: threads/compression: Make HTTP compression thread-safe MEDIUM: thread/spoe: Make the SPOE thread-safe MEDIUM: thread/dns: Make DNS thread-safe MINOR: threads: Add thread-map config parameter in the global section MINOR: threads/checks: Add a lock to protect the pid list used by external checks MINOR: threads/checks: Set the task process_mask when a check is executed MINOR: threads/mailers: Add a lock to protect queues of email alerts MINOR: threads: Don't start when device a detection module is used BUG/MEDIUM: threads: Run the poll loop on the main thread too BUG/MINOR: threads: Add missing THREAD_LOCAL on static here and there MAJOR: threads: Offically enable the threads support in HAProxy BUG/MAJOR: threads/time: Store the time deviation in an 64-bits integer BUG/MEDIUM: threads: Initialize the sync-point Dragan Dosen (4): IMPORT: sha1: import SHA1 functions MINOR: sample: add the sha1 converter MINOR: sample: add the hex2i converter BUG/MEDIUM: prevent buffers being overwritten during build_logline() execution Emeric Brun (15): MINOR: threads: Prepare makefile to link with pthread MINOR: threads: Add atomic-ops and plock includes in import dir MAJOR: threads/task: handle multithread on task scheduler MEDIUM: threads/stick-tables: handle multithreads on stick tables MINOR: threads/sample: Change temp_smp into a thread local variable MEDIUM: threads/http: Make http_capture_bad_message thread-safe MINOR: threads/regex: Change Regex trash buffer into a thread local variable MAJOR: threads/applet: Handle multithreading for applets MAJOR: threads/peers: Make peers thread safe MAJOR: threads/buffer: Make buffer wait queue thread safe MEDIUM: threads/stream: Make streams list thread safe MAJOR: threads/ssl: Make SSL part thread-safe MAJOR: threads/map: Make acls/maps thread safe MEDIUM: threads/server: Use the server lock to protect health check and cli concurrency BUG/MAJOR: threads/freq_ctr: fix lock on freq counters. Emmanuel Hocdet (9): BUILD: ssl: support OPENSSL_NO_ASYNC #define MINOR: ssl: build with recent BoringSSL library BUG/MINOR: ssl: OCSP_single_get0_status can return -1 MEDIUM: ssl: convert CBS (BoringSSL api) usage to neutral code MINOR: ssl: support Openssl 1.1.1 early callback for switchctx MINOR: ssl: generated certificate is missing in switchctx early callback MINOR: update proxy-protocol-v2 #define MINOR: merge ssl_sock_get calls for log and ppv2 MINOR: add ALPN information to send-proxy-v2 Lukas Tribus (2): BUG/MINOR: cli: restore "set ssl tls-key" command CLEANUP: cli: remove undocumented "set ssl tls-keys" command Olivier Houchard (12): BUG/MEDIUM: server: Allocate tmptrash before using it. BUG/MINOR: checks: Don't forget to release the connection on error case. MINOR: http: Mark the 425 code as "Too Early". MEDIUM: ssl: Handle early data with OpenSSL 1.1.1 MINOR: ssl/proto_http: Add keywords to take care of early data. MINOR: ssl: Don't abuse ssl_options. BUG/MINOR: dns: Fix SRV records with the new thread code. MINOR: ssl: Remove the global allow-0rtt option. MINOR: connection: introduce conn_stream MINOR: mux: add more methods to mux_ops MINOR: mux_pt: implement remaining mux_ops methods MAJOR: connection : Split struct connection into struct connection and struct conn_stream. Thierry FOURNIER (8): MINOR: hlua: Add regex class BUG/MINOR: lua: const attribute of a string is overridden MEDIUM: threads/lua: Makes the jmpbuf and some other buffers local to the current thread. MEDIUM: threads/lua: Add locks around the Lua execution parts. MEDIUM: threads/lua: Ensure that the launched tasks runs on the same threads than me MEDIUM: threads/lua: Cannot acces to the socket if we try to access from another thread. MEDIUM: threads/xref: Convert xref function to a thread safe model MEDIUM: threads/tasks: Add lock around notifications William Lallemand (13): MEDIUM: cfgparse: post section callback MEDIUM: cfgparse: post parsing registration CLEANUP: shctx: get ride of the shsess_packet{_hdr} structures MEDIUM: lists: list_for_each_entry{_safe}_from functions REORG: shctx: move lock functions and struct MEDIUM: shctx: allow the use of multiple shctx REORG: shctx: move ssl functions to ssl_sock.c MEDIUM: shctx: separate ssl and shctx MINOR: shctx: rename lock functions MEDIUM: shctx: forbid shctx to read more than expected MEDIUM: cache: configuration parsing and initialization MEDIUM: cache: store objects in cache MEDIUM: cache: deliver objects from cache Willy Tarreau (110): CONTRIB: trace: add the possibility to place trace calls in the code CONTRIB: trace: try to display the function's return value on exit CONTRIB: trace: report the base name only for file names MINOR: stream-int: stop checking for useless connection flags in chk_snd_conn MINOR: ssl: don't abort after sending 16kB MINOR: connection: move the cleanup of flag CO_FL_WAIT_ROOM MINOR: connection: add flag CO_FL_WILL_UPDATE to indicate when updates are granted MEDIUM: connection: make use of CO_FL_WILL_UPDATE in conn_sock_shutw() MINOR: raw_sock: make use of CO_FL_WILL_UPDATE MINOR: ssl_sock: make use of CO_FL_WILL_UPDATE MINOR: buffer: add the buffer input manipulation functions BUILD: Makefile: disable -Wunused-label MEDIUM: h1: ensure that 1xx, 204 and 304 don't have a payload body MINOR: h1: store the status code in the H1 message BUILD: stick-tables: silence an uninitialized variable warning CLEANUP: threads: replace the last few 1UL<<tid with tid_bit CLEANUP: threads: rename process_mask to thread_mask MINOR: h1: add a function to measure the trailers length MINOR: threads: add a portable barrier for threads and non-threads BUG/MAJOR: threads/freq_ctr: use a memory barrier to detect changes MEDIUM: connection: start to introduce a mux layer between xprt and data MINOR: connection: implement alpn registration of muxes MINOR: mux: register the pass-through mux for any ALPN string MEDIUM: session: use the ALPN token and proxy mode to select the mux MINOR: connection: report the major HTTP version from the MUX for logging (fc_http_major) MINOR: connection: introduce the conn_stream manipulation functions MINOR: connection: make conn_stream users also check for per-stream error flag MINOR: conn_stream: new shutr/w status flags MINOR: conn_stream: modify cs_shut{r,w} API to pass the desired mode MEDIUM: connection: make conn_sock_shutw() aware of lingering MINOR: connection: add cs_close() to close a conn_stream MEDIUM: mux_pt: make cs_shutr() / cs_shutw() properly close the connection MEDIUM: connection: replace conn_full_close() with cs_close() MEDIUM: connection: make mux->detach() release the connection MEDIUM: stream: do not forcefully close the client connection anymore MEDIUM: checks: exclusively use cs_destroy() to release a connection MEDIUM: connection: add a destroy callback MINOR: session: release the listener with the session, not the stream MEDIUM: session: make use of the connection's destroy callback CONTRIB: hpack: implement a reverse huffman table generator for hpack MINOR: hpack: implement the HPACK Huffman table decoder MINOR: hpack: implement the header tables management MINOR: hpack: implement the decoder MEDIUM: hpack: implement basic hpack encoding MINOR: h2: centralize all HTTP/2 protocol elements and constants MINOR: h2: create a very minimalistic h2 mux MINOR: h2: expose tune.h2.header-table-size to configure the table size MINOR: h2: expose tune.h2.initial-window-size to configure the window size MINOR: h2: expose tune.h2.max-concurrent-streams to limit the number of streams MINOR: h2: create the h2c struct and allocate its pool MINOR: h2: create the h2s struct and the associated pool MINOR: h2: handle two extra stream states for errors MINOR: h2: add a frame header descriptor for incoming frames MEDIUM: h2: allocate and release the h2c context on connection init/end MEDIUM: h2: implement basic recv/send/wake functions MEDIUM: h2: dynamically allocate the demux buffer on Rx MEDIUM: h2: implement the mux buffer allocator MINOR: h2: add the connection and stream flags listing the causes for blocking MINOR: h2: add function h2s_id() to report a stream's ID MINOR: h2: small function to know when the mux is busy MINOR: h2: new function h2c_error to mark an error on the connection MINOR: h2: new function h2s_error() to mark an error on a stream MINOR: h2: add h2_set_frame_size() to update the size in a binary frame MINOR: h2: new function h2_peek_frame_hdr() to retrieve a new frame header MINOR: h2: add a few functions to retrieve contents from a wrapping buffer MINOR: h2: add stream lookup function based on the stream ID MINOR: h2: create dummy idle and closed streams MINOR: h2: add the function to create a new stream MINOR: h2: update the {MUX,DEM}_{M,D}ALLOC flags on buffer availability MEDIUM: h2: start to consider the H2_CF_{MUX,DEM}_* flags for polling MINOR: h2: also terminate the connection on shutr MEDIUM: h2: properly consider all conditions for end of connection MEDIUM: h2: wake the connection up for send on pending streams MEDIUM: h2: start to implement the frames processing loop MINOR: h2: add a function to send a GOAWAY error frame MINOR: h2: match the H2 connection preface on init MEDIUM: h2: enable connection polling for send when a cs wants to emit MEDIUM: h2: enable reading again on the connection if it was blocked on stream buffer full MEDIUM: h2: process streams pending for sending MINOR: h2: send a real SETTINGS frame based on the configuration MEDIUM: h2: detect the presence of the first settings frame MINOR: h2: create a stream parser for the demuxer MINOR: h2: implement PING frames MEDIUM: h2: decode SETTINGS frames and extract relevant settings MINOR: h2: lookup the stream during demuxing MEDIUM: h2: honor WINDOW_UPDATE frames MINOR: h2: implement h2_send_rst_stream() to send RST_STREAM frames MINOR: h2: handle CONTINUATION frames MEDIUM: h2: partial implementation of h2_detach() MEDIUM: h2: unblock a connection when its current stream detaches MEDIUM: h2: basic processing of HEADERS frame MEDIUM: h2: don't use trash to decode headers! MEDIUM: h2: implement the response HEADERS frame to encode the H1 response MEDIUM: h2: send the H1 response body as DATA frames MEDIUM: h2: skip the response trailers if any MEDIUM: h2: properly continue to parse header block when facing a 1xx response MEDIUM: h2: send WINDOW_UPDATE frames for connection MEDIUM: h2: handle request body in DATA frames MINOR: h2: handle RST_STREAM frames MEDIUM: h2: send DATA+ES or RST_STREAM on shutw/shutr MINOR: h2: use a common function to signal some and all streams. MEDIUM: h2: handle GOAWAY frames MINOR: h2: centralize the check for the idle streams MINOR: h2: centralize the check for the half-closed(remote) streams MEDIUM: h2: silently ignore frames higher than last_id after GOAWAY MINOR: h2: properly reject PUSH_PROMISE frames coming from the client MEDIUM: h2: perform a graceful shutdown on "Connection: close" MEDIUM: h2: send a GOAWAY frame when dealing with an empty response MEDIUM: h2: apply a timeout to h2 connections BUG/MEDIUM: h2: fix incorrect timeout handling on the connection ---