Re: [PATCH 1/1] BUG/MINOR: lua: remove loop initial declarations

2021-11-24 Thread Tim Düsterhus

Bertrand,

On 11/24/21 10:16 PM, Bertrand Jacquin wrote:

No backport needed as this issue was introduced in v2.5-dev10~69 with
commit 9e5e586e35c5 ("BUG/MINOR: lua: Fix lua error handling in
`hlua_config_prepend_path()`")


Oh no, that's mine :-( Actually a backport is needed, because 2.5 was 
released yesterday.


Acked-by: Tim Duesterhus 

Best regards
Tim Düsterhus



[PATCH 1/1] BUG/MINOR: lua: remove loop initial declarations

2021-11-24 Thread Bertrand Jacquin
HAProxy is documented to support gcc >= 3.4 as per INSTALL file, however
hlua.c makes use of c11 only loop initial declarations leading to build
failure when using gcc-4.9.4:

  x86_64-unknown-linux-gnu-gcc -Iinclude  -Wchar-subscripts -Wcomment -Wformat 
-Winit-self -Wmain -Wmissing-braces -Wno-pragmas -Wparentheses -Wreturn-type 
-Wsequence-point -Wstrict-aliasing -Wswitch -Wtrigraphs -Wuninitialized 
-Wunknown-pragmas -Wunused-label -Wunused-variable -Wunused-value 
-Wpointer-sign -Wimplicit -pthread -fdiagnostics-color=auto -D_LARGEFILE_SOURCE 
-D_FILE_OFFSET_BITS=64 -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -O3 -msse 
-mfpmath=sse -march=core2 -g -fPIC -g -Wall -Wextra -Wundef 
-Wdeclaration-after-statement -fwrapv -Wno-unused-label -Wno-sign-compare 
-Wno-unused-parameter -Wno-clobbered -Wno-missing-field-initializers 
-Wtype-limits  -DUSE_EPOLL  -DUSE_NETFILTER   -DUSE_PCRE2 -DUSE_PCRE2_JIT 
-DUSE_POLL -DUSE_THREAD -DUSE_BACKTRACE   -DUSE_TPROXY -DUSE_LINUX_TPROXY 
-DUSE_LINUX_SPLICE -DUSE_LIBCRYPT -DUSE_CRYPT_H -DUSE_GETADDRINFO -DUSE_OPENSSL 
-DUSE_LUA -DUSE_ACCEPT4   -DUSE_SLZ -DUSE_CPU_AFFINITY -DUSE_TFO -DUSE_NS 
-DUSE_DL -DUSE_RT  -DUSE_PRCTL  -DUSE_THREAD_DUMP-DUSE_PCRE2 
-DPCRE2_CODE_UNIT_WIDTH=8  -I/usr/local/include 
-DCONFIG_HAPROXY_VERSION=\"2.5.0\" -DCONFIG_HAPROXY_DATE=\"2021/11/23\" -c -o 
src/connection.o src/connection.c
  src/hlua.c: In function 'hlua_config_prepend_path':
  src/hlua.c:11292:2: error: 'for' loop initial declarations are only allowed 
in C99 or C11 mode
for (size_t i = 0; i < 2; i++) {
^
  src/hlua.c:11292:2: note: use option -std=c99, -std=gnu99, -std=c11 or 
-std=gnu11 to compile your code

This commit moves loop iterator to an explicit declaration.

No backport needed as this issue was introduced in v2.5-dev10~69 with
commit 9e5e586e35c5 ("BUG/MINOR: lua: Fix lua error handling in
`hlua_config_prepend_path()`")
---
 src/hlua.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/src/hlua.c b/src/hlua.c
index 08735374af77..8dea91e75832 100644
--- a/src/hlua.c
+++ b/src/hlua.c
@@ -11249,6 +11249,7 @@ static int hlua_config_prepend_path(char **args, int 
section_type, struct proxy
char *path;
char *type = "path";
struct prepend_path *p = NULL;
+   size_t i;
 
if (too_many_args(2, args, err, NULL)) {
goto err;
@@ -11289,7 +11290,7 @@ static int hlua_config_prepend_path(char **args, int 
section_type, struct proxy
 * thread. The remaining threads will be initialized based on
 * prepend_path_list.
 */
-   for (size_t i = 0; i < 2; i++) {
+   for (i = 0; i < 2; i++) {
lua_State *L = hlua_states[i];
const char *error;
 



[PATCH]: MEDIUM: pool little FreeBSD support improvement.

2021-11-24 Thread David CARLIER
Hi

here a little patch for FreeBSD to support memory arenas trimming.

Thanks.

regards.
From 1d6386a626f56ca64c25e2dfbf2f9d90a81bd7ae Mon Sep 17 00:00:00 2001
From: David Carlier 
Date: Wed, 24 Nov 2021 20:02:41 +
Subject: [PATCH] MEDIUM: pool: trimming arenas on FreeBSD.

FreeBSD uses a slighty simplified version of jemalloc as libc allocator
since many years (there is thoughts to eventually switch to snmalloc
 but not before a long time).
We detect the libc in the least hacky way in this case aiming as jemalloc
 specific API then we try to purge arenas as much as we can.
---
 include/haproxy/compat.h |  2 +-
 src/pool.c   | 32 
 2 files changed, 25 insertions(+), 9 deletions(-)

diff --git a/include/haproxy/compat.h b/include/haproxy/compat.h
index 25b15a1f0..daa58be5d 100644
--- a/include/haproxy/compat.h
+++ b/include/haproxy/compat.h
@@ -269,7 +269,7 @@ typedef struct { } empty_t;
 #endif
 
 /* FreeBSD also has malloc_usable_size() but it requires malloc_np.h */
-#if defined(USE_MEMORY_PROFILING) && defined(__FreeBSD__) && (__FreeBSD_version >= 72)
+#if defined(__FreeBSD__) && (__FreeBSD_version >= 72)
 #include 
 #endif
 
diff --git a/src/pool.c b/src/pool.c
index af46b4469..f3ea8c7a7 100644
--- a/src/pool.c
+++ b/src/pool.c
@@ -42,8 +42,8 @@ int mem_poison_byte = -1;
 static int mem_fail_rate = 0;
 #endif
 
-#if defined(HA_HAVE_MALLOC_TRIM)
 static int using_libc_allocator = 0;
+#if defined(HA_HAVE_MALLOC_TRIM)
 
 /* ask the allocator to trim memory pools */
 static void trim_all_pools(void)
@@ -82,26 +82,42 @@ static void detect_allocator(void)
 
 	using_libc_allocator = !!memcmp(, , sizeof(mi1));
 }
-
-static int is_trim_enabled(void)
-{
-	return using_libc_allocator;
-}
 #else
 
+#if defined(__FreeBSD__)
+extern void sdallocx(void *, size_t, int) __attribute__((weak));
+#endif
+
 static void trim_all_pools(void)
 {
+#if defined(__FreeBSD__)
+	if (using_libc_allocator) {
+		unsigned int narenas = 0;
+		size_t len = sizeof(narenas);
+
+		if (mallctl("arenas.narenas", , , NULL, 0) == 0) {
+			for (unsigned int i = 0; i < narenas; i ++) {
+char mib[32] = {0};
+snprintf(mib, sizeof(mib), "arena.%u.purge", i);
+(void)mallctl(mib, NULL, NULL, NULL, 0);
+			}
+		}
+	}
+#endif
 }
 
 static void detect_allocator(void)
 {
+#if defined(__FreeBSD__)
+	using_libc_allocator = (sdallocx != NULL);
+#endif
 }
+#endif
 
 static int is_trim_enabled(void)
 {
-	return 0;
+	return using_libc_allocator;
 }
-#endif
 
 /* Try to find an existing shared pool with the same characteristics and
  * returns it, otherwise creates this one. NULL is returned if no memory
-- 
2.33.1



[ANNOUNCE] haproxy-2.3.16

2021-11-24 Thread Christopher Faulet

Hi,

HAProxy 2.3.16 was released on 2021/11/24. It added 18 new commits
after version 2.3.15.

As announced for the 2.4.9, this release contains fixes about hidden bugs
recently exposed about the shutdowns management at the conn-stream
level. The client connections close could be delayed by the client
timeout. In addition, because of a failed backport, affecting the 2.2 too,
H1 responses could be truncated. All these bugs was fixed.

The H2 multiplexer fix to drains data and be sure to send GOAWAY frame was
finally backported. It was erroneously announced for the 2.3.15. As side
effect, the caching of TLS sessions is now fixed for H2 connections. Still
on the H2 multiplexer, an incomplete old fix for H2 partial frames was
fixed. It caused some high CPU usages in h2_io_cb() on some rare occasions.

Issues reported about occasional crashed in the cache (#1284 and #1451) was
fixed. A missing break statement was the explanation.

A bug with the "program" post-parser was fixed. It could be called with an
empty programs list in case of a config parsing error on reload after
another error, and could crash.

http-response rulesets evaluation was not aligned with what is said in the
documentation. It was possible to inhibit the frontend rules evaluation with
an "allow" rule in the backend section while it should instead only stop
backend rules evaluation. This bug exists since the beginning and only
concerns the "allow" rule. It was fixed and http-after-response rulesets
evaluation was also fixed in the same way.

William's fixes about the SSL was backported. First, outgoing TLS
connections involving SNI can now be resumed in TLS 1.3. Then, the right
error is not reported during SSL handshake when a non-matching SNI is found
with the strict-sni option enabled. A "unrecognized name" error is returned
instead of "handshake failure". As a side effect, this fixes the TLS resume
for non-matching SNI, rejecting the connections.

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.3/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.3.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.3.git
   Changelog: http://www.haproxy.org/download/2.3/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Christopher Faulet (8):
  BUG/MEDIUM: mux-h1: Fix H1C_F_ST_SILENT_SHUT value
  DOC: config: Fix typo in ssl_fc_unique_id description
  BUG/MINOR: http-ana: Apply stop to the current section for http-response 
rules
  Revert "BUG/MINOR: http-ana: Don't eval front after-response rules if stopped 
on back"
  DOC: lua: Be explicit with the Reply object limits
  BUG/MEDIUM: conn-stream: Don't reset CS flags on close
  BUG/MINOR: mux-h2: Fix H2_CF_DEM_SHORT_READ value
  BUG/MINOR: stick-table/cli: Check for invalid ipv6 key

William Lallemand (3):
  BUG/MEDIUM: ssl: backend TLS resumption with sni and TLSv1.3
  BUG/MINOR: mworker: doesn't launch the program postparser
  BUG/MEDIUM: ssl: abort with the correct SSL error when SNI not found

Willy Tarreau (7):
  BUG/MEDIUM: connection: make cs_shutr/cs_shutw//cs_close() idempotent
  MINOR: connection: add a new CO_FL_WANT_DRAIN flag to force drain on close
  MINOR: mux-h2: perform a full cycle shutdown+drain on close
  BUG/MEDIUM: mux-h2: always process a pending shut read
  BUG/MEDIUM: shctx: leave the block allocator when enough blocks are found
  BUG/MINOR: shctx: do not look for available blocks when the first one is 
enough
  MINOR: shctx: add a few BUG_ON() for consistency checks

--
Christopher Faulet



[ANNOUNCE] haproxy-2.4.9

2021-11-24 Thread Christopher Faulet



Hi,

HAProxy 2.4.9 was released on 2021/11/23. It added 36 new commits
after version 2.4.8.

In the previous release, fixes about shutdowns management in the muxes have
exposed some hidden bugs. Since the muxes were introduced, in the 1.8,
shutdowns at the conn-stream level were not fully idempotent. Until
recently, it was not an issue. But in the 2.4.8, some users observed delays
to close client connections on the HAProxy side corresponding to the client
timeout because the silent mode was used instead of the clean one to
shutdown the connection. In addition, true silent shutdowns were not
properly handled in the H1 multiplexer when outgoing data were blocked,
leading too to delay to close connections.

A H2 multiplexer fix to drain data and be sure to send GOAWAY frame was
announced in the 2.4.8. However a patch was missing. Another side effect of
this missing patch was the TLS sessions were not cached as expected. It is
now fixed. Still on the H2 multiplexer, an old fix for H2 partial frames was
incomplete and caused some high CPU usages in h2_io_cb() on some rare
occasions.

Some users reported occasional crashes in the cache (#1284 and #1451). We
finally had an explanation (a missing break). This was fixed. "show cache"
cli command was also fixed to be thread-safe. Under high load, it was
possible to dereference a node already reassigned, leading to
crash. Finally, parsing of "max-age" or "s-maxage" was improved to properly
ignore unparsable value in quotes.

A bug with the "program" post-parser was fixed. It could be called with an
empty programs list in case of a config parsing error on reload after
another error, and could crash.

Recent adjustments about the backend support for WebSocket over HTTP/2 were
backported. They allow to fallback on a HTTP/1 connection if the WebSockets
are not support in HTTP/2. In addition the server keyword "ws" can be used
to tune this.

http-response rulesets evaluation was not aligned with what is said in the
documentation. It was possible to inhibit the frontend rules evaluation with
an "allow" rule in the backend section while it should instead only stop
backend rules evaluation. This bug exists since the beginning and only
concerns the "allow" rule. It was fixed and http-after-response rulesets
evaluation was also fixed in the same way.

The support for backend aggregated server check status in the Prometheus
exporter was backported. Thanks to this feature, the number of server per
health-check status are now reported at the backend level.

William fixed some bugs in the SSL part. First, outgoing TLS connections
involving SNI couldn't be resumed in TLS 1.3 because the call to
SSL_get_servername() on a resumed connection doesn't return the previous SNI
with TLS 1.3. Then, the wrong error was reported during SSL handshake when a
non-matching SNI was found with the strict-sni option enabled because the
clientHello callback was returning with a success code. An "handshake
failure" was reported instead of "unrecognized name". As a side effect of
this bug, the connections was accepted in case of TLS resume. Finally,
thanks to Willy, the SSL counter are now atomically updated.

The detection of the need for libatomic in the makefile was modified so that
it's not hard-coded on the architecture but instead detects what the
compiler says it needs. This allowed to remove the arm/aarch64 hacks on
linux and also allows MIPS and RISCV to work as expected. In addition it's
now trivial to force it if desired.

In addition, the usual bunch of some of small fixes and cleanups.

The 2.3.16 will be emitted quite soon. The next 2.2 and 2.0 releases are
planned for the next week.

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.4/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.4.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.4.git
   Changelog: http://www.haproxy.org/download/2.4/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Amaury Denoyelle (7):
  MINOR: mux-h2: add trace on extended connect usage
  BUG/MEDIUM: mux-h2: reject upgrade if no RFC8441 support
  MINOR: stream/mux: implement websocket stream flag
  MINOR: connection: implement function to update ALPN
  MINOR: connection: add alternative mux_ops param for conn_install_mux_be
  MEDIUM: server/backend: implement websocket protocol selection
  MINOR: server: add ws keyword

Christopher Faulet (10):
  DOC: config: Fix typo in ssl_fc_unique_id description
  BUG/MINOR: http-ana: Apply stop to the current section for http-response