Re: [ANNOUNCE] haproxy-2.4.9
On Thu, Nov 25, 2021 at 01:29:13PM +0300, Dmitry Sivachenko wrote: > > > On 25 Nov 2021, at 13:09, Willy Tarreau wrote: > > > > Please try the two attached patches. They re-backport something that > > we earlier failed to backport that simplifies the ugly ifdefs everywhere > > that virtually break every single backport related to SSL. > > > > For me they work with/without SSL and with older versions (tested as far > > as 0.9.8). > > > > Thanks, > > Willy > > <0001-CLEANUP-servers-do-not-include-openssl-compat.patch><0002-CLEANUP-server-always-include-the-storage-for-SSL-se.patch> > > > These two patches do fix the build. OK thanks Dmitry. For now we'll probably just keep the workaround that Amaury pushed in the mean time, but I'm pretty sure that sooner or later we'll see yet another breakage, and if so it's likely that we decide to merge them. Cheers, Willy
Re: [ANNOUNCE] haproxy-2.4.9
> On 25 Nov 2021, at 13:29, Amaury Denoyelle wrote: > > Dmitry, the patches that Willy provided you should fix the issue. Now, > do you need a 2.4.10 to be emitted early with it or is it possible for > you to keep the patches in your tree so we can have a more substantial > list of change for a new version ? > As for me there is no hurry: I'll add patches to FreeBSD ports collection.
Re: [ANNOUNCE] haproxy-2.4.9
On Thu, Nov 25, 2021 at 11:42:01AM +0300, Dmitry Sivachenko wrote:
> On 24 Nov 2021, at 12:57, Christopher Faulet wrote:
> > > > Hi,
> > > HAProxy 2.4.9 was released on 2021/11/23. It added 36 new commits
> > after version 2.4.8.
> >
> Hello,
> version 2.4.9 fails to build with OpenSSL turned off:
> src/server.c:207:51: error: no member named 'ssl_ctx' in 'struct server'
> if (srv->mux_proto || srv->use_ssl != 1 || !srv->ssl_ctx.alpn_str) {
> ~~~ ^
> src/server.c:241:37: error: no member named 'ssl_ctx' in 'struct server'
> const struct ist alpn = ist2(srv->ssl_ctx.alpn_str,
> ~~~ ^
> src/server.c:242:37: error: no member named 'ssl_ctx' in 'struct server'
> srv->ssl_ctx.alpn_len);
> ~~~ ^
> Version 2.4.8 builds fine.
>
>
Dmitry, the patches that Willy provided you should fix the issue. Now,
do you need a 2.4.10 to be emitted early with it or is it possible for
you to keep the patches in your tree so we can have a more substantial
list of change for a new version ?
--
Amaury Denoyelle
Re: [ANNOUNCE] haproxy-2.4.9
> On 25 Nov 2021, at 13:09, Willy Tarreau wrote: > > Please try the two attached patches. They re-backport something that > we earlier failed to backport that simplifies the ugly ifdefs everywhere > that virtually break every single backport related to SSL. > > For me they work with/without SSL and with older versions (tested as far > as 0.9.8). > > Thanks, > Willy > <0001-CLEANUP-servers-do-not-include-openssl-compat.patch><0002-CLEANUP-server-always-include-the-storage-for-SSL-se.patch> These two patches do fix the build. Thanks!
Re: [ANNOUNCE] haproxy-2.4.9
On Thu, Nov 25, 2021 at 11:02:52AM +0100, Amaury Denoyelle wrote:
> On Thu, Nov 25, 2021 at 11:42:01AM +0300, Dmitry Sivachenko wrote:
> > On 24 Nov 2021, at 12:57, Christopher Faulet wrote:
> > > > > Hi,
> > > > HAProxy 2.4.9 was released on 2021/11/23. It added 36 new commits
> > > after version 2.4.8.
> > >
> > Hello,
> > version 2.4.9 fails to build with OpenSSL turned off:
> > src/server.c:207:51: error: no member named 'ssl_ctx' in 'struct server'
> > if (srv->mux_proto || srv->use_ssl != 1 || !srv->ssl_ctx.alpn_str) {
> > ~~~ ^
> > src/server.c:241:37: error: no member named 'ssl_ctx' in 'struct server'
> > const struct ist alpn = ist2(srv->ssl_ctx.alpn_str,
> > ~~~ ^
> > src/server.c:242:37: error: no member named 'ssl_ctx' in 'struct server'
> > srv->ssl_ctx.alpn_len);
> > ~~~ ^
> > Version 2.4.8 builds fine.
> >
> >
>
> Thanks for your report. One of my commit to handle properly websocket on
> the server side introduces this issue. I'm working on a fix.
Please try the two attached patches. They re-backport something that
we earlier failed to backport that simplifies the ugly ifdefs everywhere
that virtually break every single backport related to SSL.
For me they work with/without SSL and with older versions (tested as far
as 0.9.8).
Thanks,
Willy
>From ce5ca630697a069ffbd81169663e5dbeb554179a Mon Sep 17 00:00:00 2001
From: Willy Tarreau
Date: Wed, 6 Oct 2021 11:23:32 +0200
Subject: CLEANUP: servers: do not include openssl-compat
This is exactly the same as for listeners, servers only include
openssl-compat to provide the SSL_CTX type to use as two pointers to
contexts, and to detect if NPN, ALPN, and cipher suites are supported,
and save up to 5 pointers in the ssl_ctx struct if not supported. This
is pointless, as these ones have all been supported for about a decade,
and including this file comes with a long dependency chain that impacts
lots of other files. The ctx was made a void*.
Now the build time was significantly reduced, from 9.2 to 8.1 seconds,
thanks to opensslconf.h being included "only" 456 times instead of 2424
previously!
The total number of lines of code compiled was reduced by 15%.
(cherry picked from commit 340ef2502eae2a37781e460d3590982c0e437fbd)
[wt: this is backported to get rid of the painful #ifdef around SSL
fields that regularly break backports]
Signed-off-by: Willy Tarreau
---
include/haproxy/server-t.h | 10 +-
1 file changed, 1 insertion(+), 9 deletions(-)
diff --git a/include/haproxy/server-t.h b/include/haproxy/server-t.h
index 429195388..32b649bf3 100644
--- a/include/haproxy/server-t.h
+++ b/include/haproxy/server-t.h
@@ -35,9 +35,7 @@
#include
#include
#include
-#include
#include
-#include
#include
#include
#include
@@ -341,7 +339,7 @@ struct server {
#ifdef USE_OPENSSL
char *sni_expr; /* Temporary variable to store a sample
expression for SNI */
struct {
- SSL_CTX *ctx;
+ void *ctx;
struct {
unsigned char *ptr;
int size;
@@ -353,9 +351,7 @@ struct server {
__decl_thread(HA_RWLOCK_T lock); /* lock the cache and SSL_CTX
during commit operations */
char *ciphers; /* cipher suite to use if
non-null */
-#ifdef HAVE_SSL_CTX_SET_CIPHERSUITES
char *ciphersuites; /* TLS 1.3 cipher suite
to use if non-null */
-#endif
int options;/* ssl options */
int verify; /* verify method (set of
SSL_VERIFY_* flags) */
struct tls_version_filter methods; /* ssl methods */
@@ -363,14 +359,10 @@ struct server {
char *ca_file; /* CAfile to use on verify */
char *crl_file; /* CRLfile to use on verify */
struct sample_expr *sni;/* sample expression for SNI */
-#ifdef OPENSSL_NPN_NEGOTIATED
char *npn_str; /* NPN protocol string */
int npn_len;/* NPN protocol string length */
-#endif
-#ifdef TLSEXT_TYPE_application_layer_protocol_negotiation
char *alpn_str; /* ALPN protocol string */
int alpn_len; /* ALPN protocol string length
*/
-#endif
} ssl_ctx;
#ifdef USE_QUIC
struct quic_transport_params quic_params; /* QUIC transport parameters
*/
--
2.28.0
>From 6d395b766fd816cf2e7feea3286a689e635e35f9 Mon Sep 17 00:00:00 2001
From: Willy Tarreau
Date: Wed, 6 Oct 2021 14:48:37 +0200
Subject: CLEANUP: server: always include the storage for SSL settings
The SSL stuff in struct server takes less than 3% of it
Re: [ANNOUNCE] haproxy-2.4.9
On Thu, Nov 25, 2021 at 11:42:01AM +0300, Dmitry Sivachenko wrote:
> On 24 Nov 2021, at 12:57, Christopher Faulet wrote:
> > > > Hi,
> > > HAProxy 2.4.9 was released on 2021/11/23. It added 36 new commits
> > after version 2.4.8.
> >
> Hello,
> version 2.4.9 fails to build with OpenSSL turned off:
> src/server.c:207:51: error: no member named 'ssl_ctx' in 'struct server'
> if (srv->mux_proto || srv->use_ssl != 1 || !srv->ssl_ctx.alpn_str) {
> ~~~ ^
> src/server.c:241:37: error: no member named 'ssl_ctx' in 'struct server'
> const struct ist alpn = ist2(srv->ssl_ctx.alpn_str,
> ~~~ ^
> src/server.c:242:37: error: no member named 'ssl_ctx' in 'struct server'
> srv->ssl_ctx.alpn_len);
> ~~~ ^
> Version 2.4.8 builds fine.
>
>
Thanks for your report. One of my commit to handle properly websocket on
the server side introduces this issue. I'm working on a fix.
--
Amaury Denoyelle
Re: [ANNOUNCE] haproxy-2.4.9
On 24 Nov 2021, at 12:57, Christopher Faulet wrote:
>
>
> Hi,
>
> HAProxy 2.4.9 was released on 2021/11/23. It added 36 new commits
> after version 2.4.8.
>
Hello,
version 2.4.9 fails to build with OpenSSL turned off:
src/server.c:207:51: error: no member named 'ssl_ctx' in 'struct server'
if (srv->mux_proto || srv->use_ssl != 1 || !srv->ssl_ctx.alpn_str) {
~~~ ^
src/server.c:241:37: error: no member named 'ssl_ctx' in 'struct server'
const struct ist alpn = ist2(srv->ssl_ctx.alpn_str,
~~~ ^
src/server.c:242:37: error: no member named 'ssl_ctx' in 'struct server'
srv->ssl_ctx.alpn_len);
~~~ ^
Version 2.4.8 builds fine.
[ANNOUNCE] haproxy-2.4.9
Hi, HAProxy 2.4.9 was released on 2021/11/23. It added 36 new commits after version 2.4.8. In the previous release, fixes about shutdowns management in the muxes have exposed some hidden bugs. Since the muxes were introduced, in the 1.8, shutdowns at the conn-stream level were not fully idempotent. Until recently, it was not an issue. But in the 2.4.8, some users observed delays to close client connections on the HAProxy side corresponding to the client timeout because the silent mode was used instead of the clean one to shutdown the connection. In addition, true silent shutdowns were not properly handled in the H1 multiplexer when outgoing data were blocked, leading too to delay to close connections. A H2 multiplexer fix to drain data and be sure to send GOAWAY frame was announced in the 2.4.8. However a patch was missing. Another side effect of this missing patch was the TLS sessions were not cached as expected. It is now fixed. Still on the H2 multiplexer, an old fix for H2 partial frames was incomplete and caused some high CPU usages in h2_io_cb() on some rare occasions. Some users reported occasional crashes in the cache (#1284 and #1451). We finally had an explanation (a missing break). This was fixed. "show cache" cli command was also fixed to be thread-safe. Under high load, it was possible to dereference a node already reassigned, leading to crash. Finally, parsing of "max-age" or "s-maxage" was improved to properly ignore unparsable value in quotes. A bug with the "program" post-parser was fixed. It could be called with an empty programs list in case of a config parsing error on reload after another error, and could crash. Recent adjustments about the backend support for WebSocket over HTTP/2 were backported. They allow to fallback on a HTTP/1 connection if the WebSockets are not support in HTTP/2. In addition the server keyword "ws" can be used to tune this. http-response rulesets evaluation was not aligned with what is said in the documentation. It was possible to inhibit the frontend rules evaluation with an "allow" rule in the backend section while it should instead only stop backend rules evaluation. This bug exists since the beginning and only concerns the "allow" rule. It was fixed and http-after-response rulesets evaluation was also fixed in the same way. The support for backend aggregated server check status in the Prometheus exporter was backported. Thanks to this feature, the number of server per health-check status are now reported at the backend level. William fixed some bugs in the SSL part. First, outgoing TLS connections involving SNI couldn't be resumed in TLS 1.3 because the call to SSL_get_servername() on a resumed connection doesn't return the previous SNI with TLS 1.3. Then, the wrong error was reported during SSL handshake when a non-matching SNI was found with the strict-sni option enabled because the clientHello callback was returning with a success code. An "handshake failure" was reported instead of "unrecognized name". As a side effect of this bug, the connections was accepted in case of TLS resume. Finally, thanks to Willy, the SSL counter are now atomically updated. The detection of the need for libatomic in the makefile was modified so that it's not hard-coded on the architecture but instead detects what the compiler says it needs. This allowed to remove the arm/aarch64 hacks on linux and also allows MIPS and RISCV to work as expected. In addition it's now trivial to force it if desired. In addition, the usual bunch of some of small fixes and cleanups. The 2.3.16 will be emitted quite soon. The next 2.2 and 2.0 releases are planned for the next week. Thanks everyone for your help and your contributions! Please find the usual URLs below : Site index : http://www.haproxy.org/ Discourse: http://discourse.haproxy.org/ Slack channel: https://slack.haproxy.org/ Issue tracker: https://github.com/haproxy/haproxy/issues Wiki : https://github.com/haproxy/wiki/wiki Sources : http://www.haproxy.org/download/2.4/src/ Git repository : http://git.haproxy.org/git/haproxy-2.4.git/ Git Web browsing : http://git.haproxy.org/?p=haproxy-2.4.git Changelog: http://www.haproxy.org/download/2.4/src/CHANGELOG Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/ --- Complete changelog : Amaury Denoyelle (7): MINOR: mux-h2: add trace on extended connect usage BUG/MEDIUM: mux-h2: reject upgrade if no RFC8441 support MINOR: stream/mux: implement websocket stream flag MINOR: connection: implement function to update ALPN MINOR: connection: add alternative mux_ops param for conn_install_mux_be MEDIUM: server/backend: implement websocket protocol selection MINOR: server: add ws keyword Christopher Faulet (10): DOC: config: Fix typo in ssl_fc_unique_id description BUG/MINOR: http-ana: Apply stop to the current section for http-response
