Hi,
HAProxy 2.2-dev2 was released on 2020/02/07. It added 115 new commits
after version 2.2-dev1.
Over the last two weeks, 36 bugs and a few build warnings were addressed.
On the features front, I mainly noted these :
- CLI: Adis added support for multiple filters on the "show table"
output. I haven't tried it yet but it looks promising as I guess I'll
be able to simplify a few anti-abuse scripts here and there that
heavily rely on grep.
- connection: the ongoing code cleanup work was continued. It should
normally be harmless (famous last words). Once this is complete, I
intend to write some doc about it so that we know this tricky area
better in the future.
- HTTP: Christopher implemented about all the things we've been debating
over the last 5 years around the "return" directive. Now we can forge
a response at any instant based on a status code, an error file, a
body from an argument string, a body from a log-format argument, a
body from a file, a body processed from a log-format stored in a file
(effectively making it a response template), a set of headers, and I
think that's about all. We may have to remind people more often that
haproxy is not a file server, but at least it will be extremely
convenient for some to be able to return rich reject pages or tailored
sets of headers for rate shaping based on 503+retry for example. This
should allow some of us to finally remove the dirty hacks consisting
in using an backend with a specific 503 error file, or something similar
based on a deny rule, just for the sake of delivering a robots, favicon,
or any ".well-known" content. Responses remain limited to the size of a
buffer, and I'm not willing to see this change for now.
- HTTP: Christopher also added a new "http-after-response" ruleset to
manipulate headers after the final response is sent. This is mainly
used to append headers after redirects or haproxy's error responses.
The main use case definitely concerns HSTS, but given that all regular
actions were implemented, one could also think about using this to
delete some Server headers for example.
- Lua: it is now possible to directly build a response to be injected
from an HTTP action by passing a reply object to txn:done(). In the
past it used to be only possible from services. This means that some
new HTTP actions could first be implemented in Lua for the time it
takes to get a broad consensus on them, before doing them natively.
- SSL: William brought significant startup time savings when using
large amounts of certificates thanks to a new option indicating
what extensions to look for. By default, for backwards compatibility
we look for ".ocsp", ".sctl", ".issuer" and all cert types extensions.
But when you know exactly what you're using and know it's pointless
to check for the ones above, you can now explicitly tell haproxy not
to look them up, and all these extra syscalls start to account for
real when you have 100k certs.
- Lua: Tim added options to prepend the lookup path for Lua modules.
- a bunch of dead code cleanups and/or minor fixes by Ilya and
William Dauchy (I noticed a few other ones arrived since the release).
- splicing: a thread-local pool of recently used pipes was added to
improve cache locality and eliminate locking on allocation, resulting
in ~5-6% performance increase on spliced traffic.
- scheduler: the scheduler now becomes latency aware. I was particularly
irritated by seeing some pathological cases in which a "show info" on
the CLI could take tens of seconds on a machine saturated under high
traffic rates just because I/O tasks requeue themselves and find some
new data available. Now we have 3 latency classes and tasks are placed
there based on their behavior. The result is that now even on a machine
saturating 16 threads at 100% forwarding 90 Gbps of traffic, the CLI
responds in 70ms and not one minute anymore. And the small objects now
experience a much lower latency on mixed traffic.
Last, a few potentially user-visible changes:
- you'll now get an error if an ACL is called "or" since you'll never be
able to match it, as the "or" word is taken by the expression parser.
It was backported but will only warn in stable versions.
- too large error files that fill the rewrite reserved area will warn
you at load time, as they are potentially incompatible with
http-after-reponse rules.
- the number of connections reported in the logs output of a quitting
proxy now clearly indicates that the value is the cumulated conns
and not the active conns. I wouldn't be surprised to learn that some
people parse this output and got it wrong as I did a few times :-)
This branch is starting to be interesting, I'll deploy it on haproxy.org
and see if it allows me to simplify some of the configuration. Let's
continue to get good stuff like this merged till end of March, and let's
try to get dev3 in two weeks.
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse : http://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Sources : http://www.haproxy.org/download/2.2/src/
Git repository : http://git.haproxy.org/git/haproxy.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy.git
Changelog : http://www.haproxy.org/download/2.2/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
Willy
---
Complete changelog :
Adis Nezirovic (3):
MEDIUM: cli: Allow multiple filter entries for "show table"
BUG/MINOR: cli: Missing arg offset for filter data values.
MINOR: cli: Report location of errors or any extra data for "show table"
Christopher Faulet (30):
BUG/MINOR: http-ana: Increment the backend counters on the backend
BUG/MINOR: stream: Be sure to have a listener to increment its counters
BUG/MINOR: http-rules: Always init log-format expr for common HTTP actions
BUG/MINOR: http-act: Use the good message to test strict rewritting mode
MINOR: global: Set default tune.maxrewrite value during global structure
init
MINOR: http-rules: Set SF_ERR_PRXCOND termination flag when a header
rewrite fails
MINOR: http-htx: Emit a warning if an error file runs over the buffer's
reserve
MINOR: htx: Add a function to append an HTX message to another one
MINOR: htx/channel: Add a function to copy an HTX message in a channel's
buffer
BUG/MINOR: http-ana: Don't overwrite outgoing data when an error is
reported
MINOR: dns: Dynamically allocate dns options to reduce the act_rule size
MINOR: dns: Add function to release memory allocated for a do-resolve rule
BUG/MINOR: http-ana: Reset HTX first index when HAPRoxy sends a response
BUG/MINOR: http-ana: Set HTX_FL_PROXY_RESP flag if a server perform a
redirect
MINOR: http-rules: Add a flag on redirect rules to know the rule direction
MINOR: http-rules: Handle the rule direction when a redirect is evaluated
MINOR: http-ana: Rely on http_reply_and_close() to handle server error
MINOR: http-ana: Add a function for forward internal responses
MINOR: http-ana/http-rules: Use dedicated function to forward internal
responses
MEDIUM: http: Add a ruleset evaluated on all responses just before
forwarding
MEDIUM: http-rules: Add the return action to HTTP rules
MEDIUM: http-rules: Support extra headers for HTTP return actions
CLEANUP: lua: Remove consistency check for sample fetches and actions
BUG/MINOR: http-ana: Increment failed_resp counters on invalid response
MINOR: lua: Get the action return code on the stack when an action
finishes
MINOR: lua: Create the global 'act' object to register all action return
codes
MINOR: lua: Add act:wake_time() function to set a timeout when an action
yields
MEDIUM: lua: Add ability for actions to intercept HTTP messages
REGTESTS: Add reg tests for the HTTP return action
REGTESTS: Add a reg test for http-after-response rulesets
Emmanuel Hocdet (2):
BUG/MINOR: ssl: ssl_sock_load_pem_into_ckch is not consistent
BUG/MINOR: ssl/cli: ocsp_issuer must be set w/ "set ssl cert"
Frédéric Lécaille (1):
BUG/MINOR: ssl: Possible memleak when allowing the 0RTT data buffer.
Ilya Shipitsin (2):
BUILD: CI: temporarily mark openssl-1.0.2 as allowed failure
BUILD: CI: move cygwin builds to Github Actions
Jerome Magnin (1):
DOC: word converter ignores delimiters at the start or end of input string
Olivier Houchard (16):
BUG/MEDIUM: netscaler: Don't forget to allocate storage for conn->src/dst.
MEDIUM: streams: Always create a conn_stream in connect_server().
MEDIUM: connections: Get ride of the xprt_done callback.
BUG/MEDIUM: connections: Set CO_FL_CONNECTED in conn_complete_session().
BUG/MEDIUM: 0rtt: Only consider the SSL handshake.
BUG/MEDIUM: streams: Move the conn_stream allocation outside #IF
USE_OPENSSL.
MINOR: ssl: Remove dead code.
BUG/MEDIUM: ssl: Don't forget to free ctx->ssl on failure.
BUG/MEDIUM: stream: Don't install the mux in back_handle_st_con().
MEDIUM: streams: Don't close the connection in back_handle_st_con().
MEDIUM: streams: Don't close the connection in back_handle_st_rdy().
BUG/MEDIUM: connections: Don't forget to unlock when killing a connection.
BUG/MEDIUM: memory_pool: Update the seq number in pool_flush().
MINOR: memory: Only init the pool spinlock once.
BUG/MEDIUM: memory: Add a rwlock before freeing memory.
BUG/MAJOR: memory: Don't forget to unlock the rwlock if the pool is empty.
Tim Duesterhus (7):
MINOR: lua: Add hlua_prepend_path function
MINOR: lua: Add lua-prepend-path configuration option
MINOR: lua: Add HLUA_PREPEND_C?PATH build option
CLEANUP: peers: Remove unused static function `free_dcache`
CLEANUP: peers: Remove unused static function `free_dcache_tx`
MINOR: acl: Warn when an ACL is named 'or'
BUG/MINOR: acl: Fix type of log message when an acl is named 'or'
William Dauchy (3):
BUG/MINOR: connection: fix ip6 dst_port copy in make_proxy_line_v2
BUG/MINOR: dns: allow 63 char in hostname
MINOR: proxy: clarify number of connections log when stopping
William Lallemand (7):
BUG/MINOR: ssl/cli: free the previous ckch content once a PEM is loaded
BUG/MINOR: ssl: increment issuer refcount if in chain
BUG/MINOR: ssl: memory leak w/ the ocsp_issuer
BUG/MINOR: ssl: typo in previous patch
BUG/MINOR: ssl/cli: fix unused variable with openssl < 1.0.2
MINOR: ssl: ssl-load-extra-files configure loading of files
BUG/MINOR: ssl: clear the SSL errors on DH loading failure
Willy Tarreau (43):
BUILD: stick-table: fix build errors introduced by last stick-table change
CLEANUP: changelog: remove the duplicate entry for 2.2-dev1
CLEANUP: backend: remove useless test for inexistent connection
CLEANUP: backend: shut another false null-deref in back_handle_st_con()
CLEANUP: stats: shut up a wrong null-deref warning from gcc 9.2
MEDIUM: connection: remove CO_FL_CONNECTED and only rely on CO_FL_WAIT_*
MINOR: stream-int: always report received shutdowns
MINOR: connection: remove CO_FL_SSL_WAIT_HS from CO_FL_HANDSHAKE
MEDIUM: connection: use CO_FL_WAIT_XPRT more consistently than
L4/L6/HANDSHAKE
MINOR: connection: remove checks for CO_FL_HANDSHAKE before I/O
MINOR: connection: do not check for CO_FL_SOCK_RD_SH too early
MINOR: connection: don't check for CO_FL_SOCK_WR_SH too early in
handshakes
MINOR: raw-sock: always check for CO_FL_SOCK_WR_SH before sending
MINOR: connection: remove some unneeded checks for CO_FL_SOCK_WR_SH
BUG/MINOR: stktable: report the current proxy name in error messages
BUG/MEDIUM: mux-h2: make sure we don't emit TE headers with anything but
"trailers"
BUILD: cfgparse: silence a bogus gcc warning on 32-bit machines
REGTESTS: make the set_ssl_cert test require version 2.2
BUILD: CI: disable slow regtests on Travis
BUG/MINOR: tcpchecks: fix the connect() flags regarding delayed ack
MEDIUM: raw-sock: remove obsolete calls to fd_{cant,cond,done}_{send,recv}
MEDIUM: pipe/thread: reduce the locking overhead
MEDIUM: pipe/thread: maintain a per-thread local cache of recently used
pipes
BUG/MEDIUM: pipe/thread: fix atomicity of pipe counters
MINOR: tasks: move the list walking code to its own function
MEDIUM: tasks: implement 3 different tasklet classes with their own queues
MEDIUM: tasks: automatically requeue into the bulk queue an already
running tasklet
OPTIM: task: refine task classes default CPU bandwidth ratios
MINOR: task: permanently flag tasklets waking themselves up
MINOR: task: make sched->current also reflect tasklets
MINOR: task: detect self-wakeups on tl==sched->current instead of
TASK_RUNNING
OPTIM: task: readjust CPU bandwidth distribution since last update
MINOR: task: don't set TASK_RUNNING on tasklets
SCRIPTS: add a new "backport" script to simplify long series of backports
BUG/MINOR: ssl: we may only ignore the first 64 errors
SCRIPTS: use /usr/bin/env bash instead of /bin/bash for scripts
CLEANUP: hpack: remove a redundant test in the decoder
CONTRIB: debug: add missing flags SF_HTX and SF_MUX
CONTRIB: debug: add the possibility to decode the value as certain types
only
CONTRIB: debug: support reporting multiple values at once
BUILD: lua: silence a warning on systems where longjmp is not marked as
noreturn
CONTRIB: debug: also support reading values from stdin
SCRIPTS: backport: use short revs and resolve the initial commit
---
