Hi,
HAProxy 2.0.0 was released on 2019/06/16. It added 63 new commits
after version 2.0-dev7.
There were a few last-minute bug reports that started to make me worry
a bit but in the end these were nothing dramatic and quickly addressed.
Aside the usual bug fixes, this version mostly contains some documentation
and build updates. The most visible change will concern Linux users (most
users in fact), do to the removal of the totally obsolete "linux22",
"linux24", "linux24e" and "linux2628" targets in the Makefile. Now there
is a single "linux-glibc" one provided for environments running a recent
(read "still supported") Linux kernel with a similarly recent glibc. No
more worries about what was still enabled in 2.6.28 or for your libc! As
a side effect of this, namespaces, TCP Fast Open and getaddrinfo() are now
enabled by default on this combination. The very few remaining improvements
were in any order :
- a small change on the state file to use server name only and not a
more-or-less random combination of the server ID and name anymore,
as clearly people were confused by this in the past and we now have
everything to rely solely on names across a cluster now ;
- some HTX updates to pass the H2 scheme between the two sides so that
applications that expect to be called as "http:" will receive it ;
- a new "http-request replace-uri" directive to more easily get rid of
any remaining reqrep users may still have in their configurations and
which will be removed from 2.1 ;
- report of each process' version in "show proc" on the master process
command line (nice to detect failed upgrades) ;
- 51degrees now supports HTX and provided a dummy library so that there
is no more code that we cannot build in travis, and I hope we can now
continue to enforce such a rule ;
I must say I am very happy with this release, especially with how smooth
it went when approaching the release over the last few weeks. Contributors
were reasonable in general, avoiding to send risky patches, and testers
did a great coverage, being able to report interesting issues. Seeing the
type of issues we've had last is very encouraging and is what makes me
want to release now without waiting more. Thanks to all these improvements,
I think we now have the cleanest version ever issued at the dot-zero release.
I sincerely hope new versions will continue on this trend that benefits a
lot from the new development cycle.
For those who haven't followed the development cycle closely, I'll try to
quickly summarize the changes since the previous LTS version (1.8). As
most of you know, 1.9 will not be maintained for a long time and should
mostly be seen as a technological preview or technical foundation for 2.0.
Thus what was added since 1.8 :
- new internal native HTTP representation called HTX, was already in 1.9
and is now enabled by default in 2.0 ;
- end-to-end HTTP/2 support including trailers and continuation frames,
as needed for gRPC ; HTTP/2 may also be upgraded from HTTP/1.1 using
the H2 preface;
- server connection pooling and more advanced reuse, with ALPN protocol
negotiation (already in 1.9) ;
- layer 7 retries, allowing to use 0-RTT and TCP Fast Open to the servers
as well as on the frontend ;
- much more scalable multi-threading, which is even enabled by default on
platforms where it was successfully tested ; by default, as many threads
are started as the number of CPUs haproxy is allowed to run on. This
removes a lot of configuration burden in VMs and containers ;
- automatic maxconn setting for the process and the frontends, directly
based on the number of available FDs (easier configuration in containers
and with systemd) ;
- logging to stdout for use in containers and systemd (already in 1.9).
Logs can now provide micro-second resolution for some events ;
- peers now support SSL, declaration of multiple stick-tables directly in
the peers section, and synchronization of server names, not just IDs ;
- In master-worker mode, the master process now exposes its own CLI and
can communicate with all other processes (including the stopping ones),
even allowing to connect to their CLI and check their state. It is also
possible to start some sidecar programs and monitor them from the master,
and the master can automatically kill old processes that survived too
many reloads ;
- the incoming connections are load-balanced between all threads depending
on their load to minimize the processing time and maximize the capacity
(already in 1.9) ;
- the SPOE connection load-balancing was significantly improved in order
to reduce high percentiles of SPOA response time (already in 1.9) ;
- the "random" load balancing algorithm and a power-of-two-choices variant
were introduced ;
- statistics improvements with per-thread counters for certain things, and
a prometheus exporter for all our statistics;
- lots of debugging help, it's easier to produce a core dump, there are
new commands on the CLI to control various things, there is a watchdog
to fail cleanly when a thread deadlock or a spinning task are detected,
so overall it should provide a better experience in field and less
round trips between users and developers (hence less stress during an
incident).
- all 3 device detection engines are now compatible with multi-threading
and can be build-tested without any external dependencies ;
- "do-resolve" http-request action to perform a DNS resolution on any,
sample, and resolvers now support relying on /etc/resolv.conf to match
the local resolver ;
- log sampling and balancing : it's now possible to send 1 log every 10
to a server, or to spread the logging load over multiple log servers;
- a new SPOA agent (spoa_server) allows to interface haproxy with Python
and Lua programs ;
- support for Solaris' event ports (equivalent of kqueue or epoll) which
will significantly improve the performance there when dealing with
numerous connections ;
- some warnings are now reported for some deprecated options that will
be removed in 2.1. Since 2.0 is long term supported, there's no
emergency to convert them, however if you see these warnings, you
need to understand that you're among their extremely rare users and
just because of this you may be taking risks by keeping them ;
- A new SOCKS4 server-side layer was provided ; it allows outgoing
connections to be forwarded through a SOCKS4 proxy (such as ssh -D).
- priority- and latency- aware server queues : it is possible now to
assign priorities to certain requests and/or to give them a time
bonus or penalty to refine control of the traffic and be able to
engage on SLAs.
- internally the architecture was significantly redesigned to allow to
further improve performance and make it easier to implement protocols
that span over multiple layers (such as QUIC). This work started in
1.9 and will continue with 2.1.
- the I/O, applets and tasks now share the same multi-threaded scheduler,
giving a much better responsiveness and fairness between all tasks as
is visible with the CLI which always responds instantly even under
extreme loads (started in 1.9) ;
- the internal buffers were redesigned to ease zero-copy operations, so
that it is possible to sustain a high bandwidth even when forwarding
HTTP/1 to/from HTTP/2 (already in 1.9) ;
- new test suite based on Vtest (formerly varnish-test) allowing each
developer to quickly verify that he didn't break anything. This has
already caught many bugs that could have been otherwise released by
sticking to the previous process, and is directly responsible for
the increase in quality of this release ;
- continuous integration using Travis-CI allows us to improve build
quality and to detect regressions much faster ;
This list is huge and I built it by reading my previous announcement
e-mails so it is certain I missed some items which may be of importance
to some of you, or that you have contributed yourself (sorry about that,
please voice in here if so). Some of my coworkers have spent a much longer
time setting up a detailed list with examples in an article posted here :
https://www.haproxy.com/blog/haproxy-2-0-and-beyond/
[disclaimer: I only quickly glanced over it for now so if you find missing
items in either list, well, consider that none of us invented anything
and that what matters is the union of these lists :-)]
I was also told that some of my HapTech colleagues have been working
tirelessly on a dataplane API to more dynamically reconfigure HAProxy,
on a Kubernetes Ingress Controller and on an SPOA agent to perform some
traffic replication to a development server, sometimes called "traffic
shadowing". It's not a secret that I'm clueless in these areas which is
why I haven't followed these efforts closely, but my understanding is
that we'll get some imminent news about them.
The development will go on with 2.1 which will not be LTS, so it will
experience quite some breakage to prepare 2.2 which will be LTS and
expected approximately at the same date next year.
I'm going to deploy this shiny new version right now on the haproxy.org
servers (not kidding, we're already on dev3), and I don't expect any bad
joke! I hope we won't need a 2.0.1 before one week or two.
Now let's see if I managed to mangle a link or an upload again, as is
the tradition with about every single new major release :-)
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse : http://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Sources : http://www.haproxy.org/download/2.0/src/
Git repository : http://git.haproxy.org/git/haproxy-2.0.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy-2.0.git
Changelog : http://www.haproxy.org/download/2.0/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
Willy
---
Complete changelog from 2.0-dev7 :
Baptiste Assmann (1):
MEDIUM: server: server-state only rely on server name
Ben51Degrees (3):
BUG/MINOR: 51d/htx: The _51d_fetch method, and the methods it calls are
now HTX aware.
MINOR: 51d: Added dummy libraries for the 51Degrees module for testing.
BUILD/MINOR: 51d: Updated build registration output to indicate thatif
the library is a dummy one or not.
Christopher Faulet (12):
BUG/MINOR: fl_trace/htx: Be sure to always forward trailers and EOM
BUG/MINOR: channel/htx: Call channel_htx_full() from channel_full()
BUG/MINOR: http: Use the global value to limit the number of parsed
headers
BUG/MINOR: htx: Detect when tail_addr meet end_addr to maximize free rooms
BUG/MEDIUM: htx: Don't change position of the first block during HTX
analysis
CLEANUP: channel: Remove channel_htx_fwd_payload() and
channel_htx_fwd_all()
BUG/MEDIUM: proto_htx: Introduce the state ENDING during forwarding
MINOR: htx: Add 3 flags on the start-line to deal with the request schemes
MINOR: h2: Set flags about the request's scheme on the start-line
MINOR: mux-h1: Set flags about the request's scheme on the start-line
MINOR: mux-h2: Forward clients scheme to servers checking start-line flags
BUG/MINOR: mux-h1: Wake busy mux for I/O when message is fully sent
Ilya Shipitsin (2):
BUILD: travis-ci: add 51Degree device detection, update openssl to 1.1.1c
BUILD: travis-ci: TFO and GETADDRINFO are now enabled by default
Kazuo Yagi (1):
MINOR: doc: Remove -Ds option in man page
Olivier Houchard (7):
MINOR: fd: Don't use atomic operations when it's not needed.
BUG/MEDIUM: h1: Don't wait for handshake if we had an error.
BUG/MEDIUM: h1: Wait for the connection if the handshake didn't complete.
BUG/MEDIUM: connections: Don't call shutdown() if we want to disable
linger.
BUG/MEDIUM: connections: Don't use ALPN to pick mux when in mode TCP.
BUG/MEDIUM: connections: Don't try to send early data if we have no mux.
BUG/MEDIUM: ssl: Make sure we initiate the handshake after using early
data.
Tim Duesterhus (2):
BUILD: Silence gcc warning about unused return value
DOC: Fix typos in CONTRIBUTING
William Lallemand (7):
DOC: mworker-prog: documentation for the program section
MINOR: mworker: change formatting in uptime field of "show proc"
MINOR: mworker: add the HAProxy version in "show proc"
MINOR: doc: add master-worker in the man page
MINOR: doc: mention HAPROXY_LOCALPEER in the man
MINOR: doc: update the manpage and usage message about -S
DOC: add some environment variables in section 2.3
Willy Tarreau (28):
MINOR: http: add a new "http-request replace-uri" action
CLEANUP: 51d: move the 51d dummy lib to contrib/51d/src to match the real
lib
BUG/MINOR: task: prevent schedulable tasks from starving under high I/O
activity
CLEANUP: connection: rename the wait_event.task field to .tasklet
CLEANUP: tasks: rename task_remove_from_tasklet_list() to tasklet_remove_*
BUILD: makefile: clarify the "help" output and list options
BUG: tasks: fix bug introduced by latest scheduler cleanup
BUG/MEDIUM: mux-h2: fix early close with option abortonclose
BUG/MEDIUM: mux-h2: properly account for the appended data in HTX
BUILD: makefile: further clarify the "help" output and list targets
BUILD: makefile: rename "linux2628" to "linux-glibc" and remove older
targets
BUILD: travis-ci: switch to linux-glibc instead of linux2628
DOC: update few references to the linux* targets and change them to
linux-glibc
BUILD: makefile: detect and reject recently removed linux targets
BUILD: makefile: enable linux namespaces by default on linux
BUILD: makefile: enable TFO on linux platforms
BUILD: makefile: enable getaddrinfo on the linux-glibc target
DOC: small updates to the CONTRIBUTING file
CLEANUP: removed obsolete examples an move a few to better places
DOC: update the outdated ROADMAP file
DOC: create a BRANCHES file to explain the life cycle
DOC: mention in INSTALL haproxy 2.0 is a long-term supported stable
version
BUILD: makefile: make the obsolete target detection compatible with
make-3.80
BUILD: tools: work around an internal compiler bug in gcc-3.4
BUILD: pattern: work around an internal compiler bug in gcc-3.4
BUILD: makefile: enable USE_RT on Solaris
BUILD: makefile: do not use echo -n
DOC: mention a few common build errors in the INSTALL file
---
