Hello Lukas
Output from 2.1.3
-------------------------------------------------------------------------------------------------------------------------
HA-Proxy version 2.1.3 2020/02/12 - https://haproxy.org/
Status: stable branch - will stop receiving fixes around Q1 2021.
Known bugs: http://www.haproxy.org/bugs/bugs-2.1.3.html
Build options :
TARGET = linux-glibc
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv
-Wno-unused-label -Wno-sign-compare -Wno-unused-parameter
-Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered
-Wno-missing-field-initializers -Wno-implicit-fallthrough
-Wno-stringop-overflow -Wno-cast-function-type -Wtype-limits
-Wshift-negative-value -Wshift-overflow=2 -Wduplicated-cond -Wnull-dereference
OPTIONS = USE_PCRE2=1 USE_PCRE2_JIT=1 USE_GETADDRINFO=1 USE_OPENSSL=1
USE_LUA=1 USE_ZLIB=1
Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER -PCRE -PCRE_JIT
+PCRE2 +PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED -REGPARM
-STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT
+CRYPT_H -VSYSCALL +GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 -MY_ACCEPT4 +ZLIB
-SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD
-OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS
Default settings :
bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with multi-threading support (MAX_THREADS=64, default=4).
Built with OpenSSL version : OpenSSL 1.1.1d 10 Sep 2019
Running on OpenSSL version : OpenSSL 1.1.1d 10 Sep 2019
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.3
Built with network namespace support.
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
IP_FREEBIND
Built with PCRE2 version : 10.32 2018-09-10
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.11
Running on zlib version : 1.2.11
Compression algorithms supported : identity("identity"), deflate("deflate"),
raw-deflate("deflate"), gzip("gzip")
Built with the Prometheus exporter as a service
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
h2 : mode=HTTP side=FE|BE mux=H2
fcgi : mode=HTTP side=BE mux=FCGI
<default> : mode=HTTP side=FE|BE mux=H1
<default> : mode=TCP side=FE|BE mux=PASS
Available services :
prometheus-exporter
Available filters :
[SPOE] spoe
[CACHE] cache
[FCGI] fcgi-app
[TRACE] trace
[COMP] compression
-------------------------------------------------------------------------------------------------------------------------
Output from 2.2.4
-------------------------------------------------------------------------------------------------------------------------
HA-Proxy version 2.2.4-de45672 2020/09/30 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2025.
Known bugs: http://www.haproxy.org/bugs/bugs-2.2.4.html
Running on: Linux 4.19.121-linuxkit #1 SMP Tue Dec 1 17:50:32 UTC 2020 x86_64
Build options :
TARGET = linux-glibc
CPU = generic
CC = gcc
CFLAGS = -O2 -g -Wall -Wextra -Wdeclaration-after-statement -fwrapv
-Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered
-Wno-missing-field-initializers -Wno-stringop-overflow -Wno-cast-function-type
-Wtype-limits -Wshift-negative-value -Wshift-overflow=2 -Wduplicated-cond
-Wnull-dereference
OPTIONS = USE_PCRE2=1 USE_PCRE2_JIT=1 USE_GETADDRINFO=1 USE_OPENSSL=1
USE_LUA=1 USE_ZLIB=1
Feature list : +EPOLL -KQUEUE +NETFILTER -PCRE -PCRE_JIT +PCRE2 +PCRE2_JIT
+POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED +BACKTRACE -STATIC_PCRE
-STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H
+GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 +ZLIB -SLZ +CPU_AFFINITY +TFO +NS
+DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER +PRCTL
+THREAD_DUMP -EVPORTS
Default settings :
bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with multi-threading support (MAX_THREADS=64, default=4).
Built with OpenSSL version : OpenSSL 1.1.1d 10 Sep 2019
Running on OpenSSL version : OpenSSL 1.1.1d 10 Sep 2019
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.3
Built with network namespace support.
Built with zlib version : 1.2.11
Running on zlib version : 1.2.11
Compression algorithms supported : identity("identity"), deflate("deflate"),
raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
IP_FREEBIND
Built with PCRE2 version : 10.32 2018-09-10
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with gcc compiler version 8.3.0
Built with the Prometheus exporter as a service
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
fcgi : mode=HTTP side=BE mux=FCGI
<default> : mode=HTTP side=FE|BE mux=H1
h2 : mode=HTTP side=FE|BE mux=H2
<default> : mode=TCP side=FE|BE mux=PASS
Available services :
prometheus-exporter
Available filters :
[SPOE] spoe
[COMP] compression
[TRACE] trace
[CACHE] cache
[FCGI] fcgi-app
-------------------------------------------------------------------------------------------------------------------------
The SSL configuration for 2.1.3
-------------------------------------------------------------------------------------------------------------------------
global
# Disable SSLv3 and, for now, TLS 1.3
ssl-default-bind-options no-sslv3 no-tlsv13
# For TLS <= 1.2 (if enabled)
# Per https://wiki.mozilla.org/Security/Server_Side_TLS, configuration: old
(backward compatibility), remove DHE-* and DES-*
ssl-default-bind-ciphers
ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA
# For TLS = 1.3 (if enabled)
# Per https://wiki.mozilla.org/Security/Server_Side_TLS, configuration:
modern (compatibility)
ssl-default-bind-ciphersuites
TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
tune.ssl.lifetime 24h
tune.ssl.cachesize 100000
-------------------------------------------------------------------------------------------------------------------------
The ssl configuration for 2.2.4
-------------------------------------------------------------------------------------------------------------------------
global
# Disable SSLv3 and, for now, TLS 1.3
ssl-default-bind-options ssl-min-ver TLSv1.0 ssl-max-ver TLSv1.2
no-tls-tickets
# Lower default dh param (remove this once java7 is no longer a problem)
tune.ssl.default-dh-param 1024
# For TLS <= 1.2 (if enabled)
# Per https://wiki.mozilla.org/Security/Server_Side_TLS, configuration: old
(backward compatibility), remove DHE-* and DES-*
ssl-default-bind-ciphers
ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA
# For TLS = 1.3 (if enabled)
# Per https://wiki.mozilla.org/Security/Server_Side_TLS, configuration:
modern (compatibility)
ssl-default-bind-ciphersuites
TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
tune.ssl.lifetime 24h
tune.ssl.cachesize 100000
-------------------------------------------------------------------------------------------------------------------------
As you can see we have enabled TLS1.0 and TLS1.1 in both configs.
I have some wireshark logs but I have to see if they contain some sensitive
information before I can post them here.
Best regards
Johan
-----Original Message-----
From: Lukas Tribus <[email protected]>
Sent: den 3 februari 2021 14:03
To: Johan Andersson <[email protected]>
Cc: [email protected]
Subject: Re: SSL session resumption
Hello Johan,
we are gonna need the outputs of "haproxy -vv" from both situations, as well as
at the very least *all* the ssl configuration parameters in haproxy that you
are using.
However, I do not believe it is likely that we can find the root cause, without
access to those handshakes, since it cannot be reproduced by openssl s_client.
What definitely changed in haproxy 2.2 is that the default minimum TLS version
is now 1.2. To rollback to TLS 1.0 you can configure:
global
ssl-default-bind-options ssl-min-ver TLSv1.0
Regards,
Lukas
On Wed, 3 Feb 2021 at 13:36, Johan Andersson <[email protected]> wrote:
>
> To whom it may concern
>
>
>
> We have recently upgraded out HAProxy version from 2.1.3 to 2.2.4.
>
> After the upgrade we got customer complaints that the data usage of their
> devices had gone up. Our company sells proprietary hardware that logs data
> and sends that to a web service which we host. These devices are often
> deployed remotely and connected via shaky 3G connections with data-capped SIM
> cards, so low data usage is very important.
>
> After some digging with Wireshark, we found that the SSL sessions are not
> resumed. Instead a new handshake is initiated every time the device sends
> data. Which is typically once an hour.
>
> We have set the global tune.ssl.lifetime parameter to 24h and the
> tune.ssl.cachesize to 100000 and this has worked since HAProxy version 1.6.9
> when we first introduced it.
>
> We have also tested with the latest 2.1.11 release of HAProxy and it behaves
> the same way as the 2.1.3 version. We have also tested with 2.2.0 and 2.2.8
> and they behave the same as 2.2.4.
>
>
>
> We have tried reproducing this with openssl s_client, saving the session id
> between requests but can’t reproduce it that way.
>
> We have also pored over the change logs between versions to see if there is
> some change that could make HAProxy behave this way.
>
>
>
> We’re at a loss here, what could cause this behavior, and how can we fix it?
>
>
>
>
>
> Best regards
>
>
>
> Johan Andersson
>
> Development Engineer
>
> Global Platforms Cloud Team
>
>
>
> HMS Industrial Networks AB
>
> Stationsgatan 37, Box 4126
>
> 300 04 Halmstad, Sweden
>
>
>
> Email: [email protected]
>
>
>
>
>
> HALMSTAD | BARCELONA | BEIJING | BOSTON | BUCHEN | CHICAGO | COVENTRY
> | DEN BOSCH | DUBAI | IGUALADA |
>
> KARLSRUHE | MILAN | MULHOUSE | NIVELLES | PUNE | RAVENSBURG | SEOUL |
> SINGAPORE | TOKYO | WETZLAR
>
>
