Re: unsubscribe

[Aleksandar Lazic]

Hi.

Here can you find the right way to Unsubscribe from the list 
https://www.haproxy.org/#tact


Regards

Alex

On 2024-02-12 (Mo.) 23:02, Nicolas Grilly wrote:


*Nicolas Grilly*
Managing Partner
+33 6 03 00 25 34
Recrutez plus rapidement avec VocationCity.com 
Hire faster with VocationCity.com 

unsubscribe

[Nicolas Grilly]

*Nicolas Grilly*
Managing Partner
+33 6 03 00 25 34
Recrutez plus rapidement avec VocationCity.com

Hire faster with VocationCity.com 

Re: [PATCH] DOC/MINOR: userlists: musl performance

[Lukas Tribus]

On Mon, 12 Feb 2024 at 18:10, Nicolas CARPi  wrote:
>
> Dear Lukas, Willy,
>
> Please find another patch attached, addressing your comments.
>
> Willy: s/gcc/glibc/
>
> Lukas: I shifted the focus on the rounds/cost solution, while still
> mentioning the musl issue, as this problem is clearly more visible on
> Alpine Linux, as the github issues show.

Thank you, I agree.

Acked-by: Lukas Tribus 


Lukas

Re: [PATCH] DOC/MINOR: userlists: musl performance

[Nicolas CARPi]

Dear Lukas, Willy,

Please find another patch attached, addressing your comments.

Willy: s/gcc/glibc/

Lukas: I shifted the focus on the rounds/cost solution, while still 
mentioning the musl issue, as this problem is clearly more visible on 
Alpine Linux, as the github issues show.

Cheers,
~Nico
>From 005baaf717fc3e345ff02e9adcbfe169ae17dfb7 Mon Sep 17 00:00:00 2001
From: Nicolas CARPi 
Date: Mon, 12 Feb 2024 18:03:52 +0100
Subject: [PATCH] DOC/MINOR: userlists: mention solutions to high cpu with
 hashes

This change adds a paragraph to the documentation regarding "userlists"
and the use of hashed password value.

It indicates what a user can do to address the high CPU cost of
having to calculate the hash at each request, such as reducing the
number of rounds or the cost complexity, if the algorithm allows for it.

I believe it is necessary to mention how the musl C library
impacts performance of hashing functions, as this has already led to a
few issues:

https://github.com/haproxy/haproxy/issues/1298
https://github.com/haproxy/haproxy/issues/2008
https://github.com/haproxy/haproxy/issues/2251

The performance impact is significant enough to mention it.
---
 doc/configuration.txt | 10 +-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/doc/configuration.txt b/doc/configuration.txt
index 937d2a4b7..2eef819e2 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -3992,7 +3992,15 @@ user  [password|insecure-password ]
   designed to be expensive to compute to achieve resistance against brute
   force attacks. They do not simply salt/hash the clear text password once,
   but thousands of times. This can quickly become a major factor in HAProxy's
-  overall CPU consumption!
+  overall CPU consumption, and can even lead to application crashes!
+
+  To address the high CPU usage of hash functions, one approach is to reduce
+  the number of rounds of the hash function (SHA family algorithms) or decrease
+  the "cost" of the function, if the algorithm supports it.
+
+  As a side note, musl (e.g. Alpine Linux) implementations are known to be
+  slower than their glibc counterparts when calculating hashes, so you might
+  want to consider this aspect too.
 
   Example:
 userlist L1
-- 
2.43.0

Re: [PATCH] DOC/MINOR: userlists: musl performance

[Lukas Tribus]

On Mon, 12 Feb 2024 at 14:13, Nicolas CARPi  wrote:
>
> Hello everyone,
>
> Please find attached my very first patch to the documentation. Hope I
> did everything good! :)
>
> Based on a comment from @bugre:
> https://github.com/haproxy/haproxy/issues/2251#issuecomment-1716594046
>
> (and also because I've been bitten by this!)

This is getting confusing and I'm not sure if I agree with this patch.

The problem is neither the libc nor the hash itself, but the iterations.
Documenting that one libc performs worse or even much worse is besides
the point. The point is that strong hashes with high iteration counts
are designed to be a self-DoS, and that is exactly how they behave in
haproxy, on all libcs.

Worse, this suggests at least in some way that a configuration like
this acceptable on glibc.


Lukas

Re: [PATCH] DOC/MINOR: userlists: musl performance

[Willy Tarreau]

Hi Nicolas,

On Mon, Feb 12, 2024 at 02:13:18PM +0100, Nicolas CARPi wrote:
> Hello everyone,
> 
> Please find attached my very first patch to the documentation. Hope I 
> did everything good! :)

Thank you! I have one comment below:

> +  Furthermore, there is a significant performance penalty when using a musl
> +  variant (for instance with the Alpine linux distribution), as the musl 
> code for
> +  these hashing functions is less performant/optimized than its GCC
   ~~~
I guess you meant "glibc" instead of GCC above ?  :-)
If that's it, no worries, I can fix it and commit your patch so that
you don't need to resend it, otherwise feel free to respin another
version or to suggest an edit.

Thanks!
Willy

[PATCH] DOC/MINOR: userlists: musl performance

[Nicolas CARPi]

Hello everyone,

Please find attached my very first patch to the documentation. Hope I 
did everything good! :)

Based on a comment from @bugre: 
https://github.com/haproxy/haproxy/issues/2251#issuecomment-1716594046

(and also because I've been bitten by this!)

Best,
~Nicolas
>From 04844e6eee725ff8bfe4ec62578e24649f020ec9 Mon Sep 17 00:00:00 2001
From: Nicolas CARPi 
Date: Mon, 12 Feb 2024 12:45:15 +0100
Subject: [PATCH] DOC/MINOR: userlists: mention performance penalty of musl

This change adds a paragraph to the documentation regarding "userlists"
and the use of hashed password value. It is necessary to mention how the
Alpine/musl C library impacts performance of hashing functions, as this
has already led to a few issues:

https://github.com/haproxy/haproxy/issues/1298
https://github.com/haproxy/haproxy/issues/2008
https://github.com/haproxy/haproxy/issues/2251

This change indicates clearly the shortcomings of musl, and proposes
workarounds such as reducing the number of rounds or the cost
complexity, if the algorithm allows for it.
---
 doc/configuration.txt | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/doc/configuration.txt b/doc/configuration.txt
index 937d2a4b7..9864a49a5 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -3994,6 +3994,12 @@ user  [password|insecure-password ]
   but thousands of times. This can quickly become a major factor in HAProxy's
   overall CPU consumption!
 
+  Furthermore, there is a significant performance penalty when using a musl
+  variant (for instance with the Alpine linux distribution), as the musl code 
for
+  these hashing functions is less performant/optimized than its GCC
+  counterpart. When using musl, reducing the number of rounds or the "cost" of
+  the algorithm used can help a lot with the CPU usage.
+
   Example:
 userlist L1
   group G1 users tiger,scott
-- 
2.43.0

Re: WolfSSL builds for use with HAProxy

[William Lallemand]

On Fri, Feb 09, 2024 at 11:00:24PM +, Tristan wrote:
> Hi Ilya,
> 
> On 09/02/2024 20:31, Илья Шипицин wrote:
> > I run QUIC Interop from time to time, WolfSSL shows the best
> > compatibility compared to LibreSSL and aws-lc.
> > it really looks like a winner today
> 
> And in comparison to current QuicTLS?
> 
> > I'm afraid it practice it works in a different way.
> > First, you install WolfSSL to prod, next INSTALL/wiki got updated :)
> 
> I appreciate the joke... :)

Well, unfortunately they are a lot of unknows with wolfSSL, so we every
new user could discover them :-)

> 
> Either way, for now I'm waiting on OCSP support first (hi William, Rémi);
> hopefully someone else figures out the best build flags by the time that's
> dealt with.

I'll add that to my list but that's not the biggest priority for now :)


-- 
William Lallemand

Re: WolfSSL builds for use with HAProxy

[William Lallemand]

On Thu, Feb 08, 2024 at 02:46:46PM +, Tristan wrote:
> Hi all,
> 

Hello,

> With the ever-increasing threat of one day needing to give up on OpenSSL
> 1.1.1 (whenever the next bad CVE is found on QuicTLS 1.1.1w, essentially) I
> was looking at alternatives a bit closer.
> 
> Based on the wiki,
> https://github.com/openssl/openssl/issues/20286#issuecomment-1527869072, and
> that it has support for other features I'm interested in (notably ECH),
> WolfSSL seems by far my best bet at the moment.
> 

I have personnally more hope with aws-lc, given the fact that this is a
fork of a stack we know, and they maintain a stable ABI. But we don't
even have the clienthello callback in it yet, we have to invest some
time on this library too. But WolfSSL could a good alternative for you,
but this is still experimental.

> However, given that almost everything is compile time and defaults focus on
> suitability for constrained embedded environments rather than best
> big-proxy-server oriented performance, does anyone have pointers on what
> flags are important/traps/etc?
> 
> Besides the getrandom thing, HAProxy's INSTALL/wiki only vaguely mention
> that such build-time tuning is required, so I'm hoping someone might have
> gone through that already.
> 

--enable-haproxy is suppose to enable every features we support. you
could be needing --enable-keylog-export for debugging with the keylog
file, but it actually does not support TLS1.3 so that would be useless
for QUIC. You also need --enable-quic.

The other useful flags are optimizations depending on your CPU, for
example --enable-aesni , --enable-intelasm .


> This one is a bit extra, but considering that aiming for bleeding edge with
> WolfSSL is not entirely compatible with how most distros work (ie even if it
> was packaged, it's likely going to be perpetually quite far behind), what
> does the future look like in that regard from the distros' side?
> 

5.6.6 is already in debian testing and will be in the next ubuntu LTS
release, the problem is that it won't be compatible with haproxy... We
are trying to make them enable the haproxy features in --enable-distro
but it seems like it's already too late. At least --enable-quic is now
enable by default.

https://github.com/wolfSSL/wolfssl/issues/6834

So basically we reached a deadlock which will last 2 years. Regarding
Redhat I don't think they are even packaging it.

But yes, wolfssl cycles are not really compatible with LTS distro, so it
would need a PPA which provides the updated lib with the right flags.

-- 
William Lallemand