stable-bot: Bugfixes waiting for a release 2.2 (1), 2.1 (28), 2.0 (21), 1.8 (11)

2020-09-15 Thread stable-bot
Hi,

This is a friendly bot that watches fixes pending for the next haproxy-stable 
release!  One such e-mail is sent periodically once patches are waiting in the 
last maintenance branch, and an ideal release date is computed based on the 
severity of these fixes and their merge date.  Responses to this mail must be 
sent to the mailing list.


Last release 2.2.3 was issued on 2020-09-08.  There are currently 1 patches in 
the queue cut down this way:
- 1 MEDIUM, first one merged on 2020-09-11

Thus the computed ideal release date for 2.2.4 would be 2020-10-11, which is in 
four weeks or less.

Last release 2.1.8 was issued on 2020-07-31.  There are currently 28 patches in 
the queue cut down this way:
- 1 MAJOR, first one merged on 2020-09-07
- 9 MEDIUM, first one merged on 2020-08-05
- 18 MINOR, first one merged on 2020-08-11

Thus the computed ideal release date for 2.1.9 would be 2020-09-04, which was 
one week ago.

Last release 2.0.17 was issued on 2020-07-31.  There are currently 21 patches 
in the queue cut down this way:
- 1 MAJOR, first one merged on 2020-09-07
- 9 MEDIUM, first one merged on 2020-08-05
- 11 MINOR, first one merged on 2020-08-11

Thus the computed ideal release date for 2.0.18 would be 2020-10-04, which is 
in three weeks or less.

Last release 1.8.26 was issued on 2020-08-03.  There are currently 11 patches 
in the queue cut down this way:
- 4 MEDIUM, first one merged on 2020-08-05
- 7 MINOR, first one merged on 2020-08-03

Thus the computed ideal release date for 1.8.27 would be 2020-10-26, which is 
in six weeks or less.

The current list of patches in the queue is:
 - 2.0, 2.1  - MAJOR   : contrib/spoa-server: Fix unhandled 
python call leading to memory leak
 - 2.0, 2.1  - MEDIUM  : contrib/spoa-server: Fix ipv4_address 
used instead of ipv6_address
 - 2.0, 2.1  - MEDIUM  : doc: Fix replace-path action 
description
 - 2.1   - MEDIUM  : ssl: memory leak of ocsp data at 
SSL_CTX_free()
 - 2.0, 2.1  - MEDIUM  : mux-h1: Refresh H1 connection timeout 
after a synchronous send
 - 1.8, 2.0, 2.1 - MEDIUM  : map/lua: Return an error if a map is 
loaded during runtime
 - 2.0, 2.1  - MEDIUM  : htx: smp_prefetch_htx() must always 
validate the direction
 - 2.0, 2.1  - MEDIUM  : mux-h1: always apply the timeout on 
half-closed connections
 - 1.8, 2.0, 2.1, 2.2- MEDIUM  : pattern: Renew the pattern 
expression revision when it is pruned
 - 1.8, 2.0  - MEDIUM  : mux-h2: Don't fail if nothing is 
parsed for a legacy chunk response
 - 1.8, 2.0, 2.1 - MEDIUM  : ssl: check OCSP calloc in 
ssl_sock_load_ocsp()
 - 1.8, 2.0, 2.1 - MINOR   : lua: Check argument type to convert it 
to IP mask in arg validation
 - 2.0, 2.1  - MINOR   : contrib/spoa-server: Updating 
references to free in case of failure
 - 2.1   - MINOR   : ssl: fix memory leak at OCSP loading
 - 1.8, 2.0, 2.1 - MINOR   : lua: Check argument type to convert it 
to IPv4/IPv6 arg validation
 - 2.1   - MINOR   : http-rules: Replace path and 
query-string in "replace-path" action
 - 1.8, 2.0, 2.1 - MINOR   : reload: do not fail when no socket is 
sent
 - 1.8, 2.0, 2.1 - MINOR   : threads: work around a libgcc_s issue 
with chrooting
 - 2.0, 2.1  - MINOR   : snapshots: leak of snapshots on 
deinit()
 - 2.1   - MINOR   : converters: Store the sink in an arg 
pointer for debug() converter
 - 1.8   - MINOR   : dns: ignore trailing dot
 - 2.1   - MINOR   : lua: Duplicate lua strings in sample 
fetches/converters arg array
 - 1.8, 2.0, 2.1 - MINOR   : stats: use strncmp() instead of 
memcmp() on health states
 - 1.8, 2.0, 2.1 - MINOR   : startup: haproxy -s cause 100% cpu
 - 2.0, 2.1  - MINOR   : contrib/spoa-server: Ensure ip address 
references are freed
 - 2.0, 2.1  - MINOR   : auth: report valid crypto(3) support 
depending on build options
 - 2.1   - MINOR   : arg: Fix leaks during arguments 
validation for fetches/converters
 - 2.1   - MINOR   : lua: Duplicate map name to load it 
when a new Map object is created
 - 2.0, 2.1  - MINOR   : contrib/spoa-server: Do not free 
reference to NULL
 - 2.1   - MINOR   : http-rules: Replace path and 
query-string in "replace-path" action"

-- 
The haproxy stable-bot is freely provided by HAProxy Technologies to help 
improve the quality of each HAProxy release.  If you have any issue with these 
emails or if you want to suggest some improvements, please post them on the 
list so that the solutions suiting the most users can be found.



Re: TLS handshake error

2020-09-15 Thread vcjouni

Hi,

I tested for openssl-1.1.1g.tar.gz from openssl.org in Linux Mint 19.3:

$ patch -p1 < reorder-sigalgs.patch
patching file ssl/t1_lib.c

./config

make

make test

Test Summary Report
---
../test/recipes/80-test_ssl_new.t    (Wstat: 256 Tests: 29 
Failed: 1)

  Failed test:  20
  Non-zero exit status: 1
Files=155, Tests=1466, 76 wallclock secs ( 1.74 usr  0.10 sys + 75.96 
cusr  9.72 csys = 87.52 CPU)

Result: FAIL
Makefile:207: recipe for target '_tests' failed

What did you mean by thrown that test out? Now that test failed.

Br,

Jouni


On 9/15/20 4:36 PM, Bruno Henc wrote:

Hi,

Last time I saw this error it involved TLS decryption by firewalls that didn't 
support RSA-PSS. Why they blow up
when the new, more secure RSA-PSS signature algorithms are used beats me, but 
it's principally _on them_ for not supporting the latest IETF standards.

Attached is a patch that reorders the signature algorithms
in openssl 1.1.1 so that the pkcs1 ones are first. Also,
the test/recipes/80-test_ssl_new.t test needs to be thrown out for this to 
work. I would recommend trying to get this to work without docker and 
kubernetes first, and using the -d haproxy option to get more detailed OpenSSL 
logging.

It is also likely you will need to set ssl-default-bind-curves since the 
firewalls in question do not support curve x25519 either. This is a HAProxy 2.2 
option (https://www.haproxy.com/blog/announcing-haproxy-2-2/). 
ssl-default-bind-curves P-256 should/could the trick, although tuning this to 
include all supported curves should be done for production traffic.

As for implementing this in HAProxy, SSL_CTX_set1_sigalgs_list could be used in 
the part of the code that initializes the TLS connection, but it seems that 
somewhere in the handshake code the signature algorithms get renegotiated.
I haven't had any luck in identifying where exactly this renegotiation happens. Someone 
else might have more luck in writing up a patch that adds a "sigalgs" or 
similarly named option to adjust the signature algorithms.

Regards,

Bruno

‐‐‐ Original Message ‐‐‐
On Tuesday, September 15, 2020 1:45 PM, vcjouni  
wrote:


Hi!

We can not get haproxy-ingress to work with TLS authentication. Only
option to get this work is by using force-tlsv12 and then only Chrome
works. Problem is TLS handshake decrypt error when using RSA-PSS
signature algorithm, handshake fails every time. When we use
force-tlsv12, only Chrome will change signature back to pkcs1 and then
session is ok. Safari, Edge or IE does not work with any option and they
keep offering RSA-PSS signature.

This same CA bundle and cards has been used before with citrix netscaler
ingress controller without problems (TLSv1.2). Smart cards are
government provided FINEID cards, so we can't test them command line,
only by using those cards with browser.

I already discussed with haproxy-ingress builder jcmoraisjr and he
suggested us to ask from haproxy mailing list.

Docker image: image: quay.io/jcmoraisjr/haproxy-ingress

haproxy -vv



HA-Proxy version 2.0.17 2020/07/31 - https://haproxy.org/
Build options :
   TARGET  = linux-glibc
   CPU = generic
   CC  = gcc
   CFLAGS  = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement
-fwrapv -Wno-address-of-packed-member -Wno-unused-label
-Wno-sign-compare -Wno-unused-parameter -Wno-old-style-declaration
-Wno-ignored-qualifiers -Wno-clobbered -Wno-missing-field-initializers
-Wno-implicit-fallthrough -Wno-stringop-overflow -Wno-cast-function-type
-Wtype-limits -Wshift-negative-value -Wshift-overflow=2
-Wduplicated-cond -Wnull-dereference
   OPTIONS = USE_PCRE2=1 USE_PCRE2_JIT=1 USE_GETADDRINFO=1 USE_OPENSSL=1
USE_LUA=1 USE_ZLIB=1

Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER -PCRE
-PCRE_JIT +PCRE2 +PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD
-PTHREAD_PSHARED -REGPARM -STATIC_PCRE -STATIC_PCRE2 +TPROXY
+LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H -VSYSCALL +GETADDRINFO
+OPENSSL +LUA +FUTEX +ACCEPT4 -MY_ACCEPT4 +ZLIB -SLZ +CPU_AFFINITY +TFO
+NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER
+PRCTL +THREAD_DUMP -EVPORTS

Default settings :
   bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=2).
Built with OpenSSL version : OpenSSL 1.1.1g  21 Apr 2020
Running on OpenSSL version : OpenSSL 1.1.1g  21 Apr 2020
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.5
Built with network namespace support.
Built with transparent proxy support using: IP_TRANSPARENT
IPV6_TRANSPARENT IP_FREEBIND
Built with zlib version : 1.2.11
Running on zlib version : 1.2.11
Compression algorithms supported : identity("identity"),
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with PCRE2 version : 10.35 2020-05-09
PCRE2 library supports JIT : yes
Encrypted password 

Re: check successful reload using master cli

2020-09-15 Thread Jonathan Matthews
On Tue, 15 Sep 2020 at 16:42, Tim Düsterhus  wrote:

> Why not use the Tab (0x09) as separator and make the output a proper TSV?


I’m only 2% joking when I point out that ASCII already has single-byte
inter-field and inter-record delimiters defined, just waiting to be used
... :-)

https://ronaldduncan.wordpress.com/2009/10/31/text-file-formats-ascii-delimited-text-not-csv-or-tab-delimited-text/

http://html-codes.info/ascii/standard/What-is-the-ASCII-value-of-record%20separator_30


-- 
Jonathan Matthews
https://jpluscplusm.com


Re: check successful reload using master cli

2020-09-15 Thread Joao Morais



> Em 15 de set de 2020, à(s) 12:36, William Lallemand  
> escreveu:
> 
> Oh right... the space in "[was: ]" is troublesome for cutting the string,
> we must remove it.

It's not a problem at all when using chunks of fixed size, even if columns 
differ between them, and the lay out ([was: ...]) remains the same.

> Regarding the size of the fields, 16 bytes should be enough but there is
> the expection of the version field that can be much larger if a dev
> version with the commit ID in the version is used.

If changing the spec, please provide also a way to distinguish the haproxy 
version via master cli which is not in the same `show proc` command, something 
like a `show info` or `version`, or at most in a fixed position at the start of 
`show proc`, so I can use it to version the output parser.

~jm




Re: check successful reload using master cli

2020-09-15 Thread Tim Düsterhus
William,

Am 15.09.20 um 17:36 schrieb William Lallemand:
> Oh right... the space in "[was: ]" is troublesome for cutting the string,
> we must remove it.
> 

Why not use the Tab (0x09) as separator and make the output a proper TSV?

Best regards
Tim Düsterhus



Re: check successful reload using master cli

2020-09-15 Thread William Lallemand
On Tue, Sep 15, 2020 at 11:52:15AM -0300, Joao Morais wrote:
> 
> 
> > Em 14 de set de 2020, à(s) 19:14, William Lallemand 
> >  escreveu:
> > 
> > Hello,
> > 
> > On Mon, Sep 14, 2020 at 12:09:21PM -0300, Joao Morais wrote:
> >> Hello list, I'm working on an automation around haproxy process
> >> lifecycle in master-worker mode. It's working nice but I'm not
> >> confident that all premisses I used are correct. Please provide some
> >> guidance if I did any wrong assumption, RTFM link is welcome as usual.
> >> 
> > 
> > The documentation about the master CLI is probably light about how it's
> > implemented so I'll try to be as clear as possible.
> > 
> Cristal clear, thank you so much. Just a few more doubts below.
> 
> 
> >> First of all I figured out that master cli will "connection refused"
> >> after a reload cmd during the start of the new worker. I'm using a
> >> unix socket but I assume the behaviour would be the same using
> >> ip+port.
> > 
> > Indeed, during a reload all connections to the master CLI are closed
> > because the master process is executed again. There is currently no
> > mecanism to recover a previous connection from the previous process.
> > That's the same with the unix socket or an IP socket.
> > 
> Is there any chance to be fast enough to connect and receive a response to a 
> `show proc` command before the master started the reload process? Sleep 1ms 
> maybe? Or when master finishes the `reload` response it’ll immediately 
> shutdown its listener?
> 

Once it receive the 'reload' it closes the master CLI immediately, it
will be really difficult to be fast enough to do a 'show proc'.

> >> I also observed that the `show proc` output isn't that
> >> machine-friendly, anyway it's not a big deal, some regex does the job
> >> and I'm assuming the lay out is immutable. Maybe I missed some
> >> optional json output in the doc?
> >> 
> > It is supposed to be splitable easily with cut, but if that does not
> > work this is a clearly a bug. 
> 
> So may I assume fields have 16 bytes and relative pid is represented
> as "[was: <0-9...>]", otherwise we have a bug?
> 

Oh right... the space in "[was: ]" is troublesome for cutting the string,
we must remove it.

Regarding the size of the fields, 16 bytes should be enough but there is
the expection of the version field that can be much larger if a dev
version with the commit ID in the version is used.

-- 
William Lallemand



Re: check successful reload using master cli

2020-09-15 Thread Joao Morais



> Em 14 de set de 2020, à(s) 19:14, William Lallemand  
> escreveu:
> 
> Hello,
> 
> On Mon, Sep 14, 2020 at 12:09:21PM -0300, Joao Morais wrote:
>> Hello list, I'm working on an automation around haproxy process
>> lifecycle in master-worker mode. It's working nice but I'm not
>> confident that all premisses I used are correct. Please provide some
>> guidance if I did any wrong assumption, RTFM link is welcome as usual.
>> 
> 
> The documentation about the master CLI is probably light about how it's
> implemented so I'll try to be as clear as possible.
> 
Cristal clear, thank you so much. Just a few more doubts below.


>> First of all I figured out that master cli will "connection refused"
>> after a reload cmd during the start of the new worker. I'm using a
>> unix socket but I assume the behaviour would be the same using
>> ip+port.
> 
> Indeed, during a reload all connections to the master CLI are closed
> because the master process is executed again. There is currently no
> mecanism to recover a previous connection from the previous process.
> That's the same with the unix socket or an IP socket.
> 
Is there any chance to be fast enough to connect and receive a response to a 
`show proc` command before the master started the reload process? Sleep 1ms 
maybe? Or when master finishes the `reload` response it’ll immediately shutdown 
its listener?


>> I also observed that the `show proc` output isn't that
>> machine-friendly, anyway it's not a big deal, some regex does the job
>> and I'm assuming the lay out is immutable. Maybe I missed some
>> optional json output in the doc?
>> 
> It is supposed to be splitable easily with cut, but if that does not
> work this is a clearly a bug. 

So may I assume fields have 16 bytes and relative pid is represented as "[was: 
<0-9...>]", otherwise we have a bug?

~jm




Re: TLS handshake error

2020-09-15 Thread Bruno Henc
Hi,

Last time I saw this error it involved TLS decryption by firewalls that didn't 
support RSA-PSS. Why they blow up
when the new, more secure RSA-PSS signature algorithms are used beats me, but 
it's principally _on them_ for not supporting the latest IETF standards.

Attached is a patch that reorders the signature algorithms
in openssl 1.1.1 so that the pkcs1 ones are first. Also,
the test/recipes/80-test_ssl_new.t test needs to be thrown out for this to 
work. I would recommend trying to get this to work without docker and 
kubernetes first, and using the -d haproxy option to get more detailed OpenSSL 
logging.

It is also likely you will need to set ssl-default-bind-curves since the 
firewalls in question do not support curve x25519 either. This is a HAProxy 2.2 
option (https://www.haproxy.com/blog/announcing-haproxy-2-2/). 
ssl-default-bind-curves P-256 should/could the trick, although tuning this to 
include all supported curves should be done for production traffic.

As for implementing this in HAProxy, SSL_CTX_set1_sigalgs_list could be used in 
the part of the code that initializes the TLS connection, but it seems that 
somewhere in the handshake code the signature algorithms get renegotiated.
I haven't had any luck in identifying where exactly this renegotiation happens. 
Someone else might have more luck in writing up a patch that adds a "sigalgs" 
or similarly named option to adjust the signature algorithms.

Regards,

Bruno

‐‐‐ Original Message ‐‐‐
On Tuesday, September 15, 2020 1:45 PM, vcjouni  
wrote:

> Hi!
>
> We can not get haproxy-ingress to work with TLS authentication. Only
> option to get this work is by using force-tlsv12 and then only Chrome
> works. Problem is TLS handshake decrypt error when using RSA-PSS
> signature algorithm, handshake fails every time. When we use
> force-tlsv12, only Chrome will change signature back to pkcs1 and then
> session is ok. Safari, Edge or IE does not work with any option and they
> keep offering RSA-PSS signature.
>
> This same CA bundle and cards has been used before with citrix netscaler
> ingress controller without problems (TLSv1.2). Smart cards are
> government provided FINEID cards, so we can't test them command line,
> only by using those cards with browser.
>
> I already discussed with haproxy-ingress builder jcmoraisjr and he
> suggested us to ask from haproxy mailing list.
>
> Docker image: image: quay.io/jcmoraisjr/haproxy-ingress
>
> haproxy -vv
>
> 
>
> HA-Proxy version 2.0.17 2020/07/31 - https://haproxy.org/
> Build options :
>   TARGET  = linux-glibc
>   CPU = generic
>   CC  = gcc
>   CFLAGS  = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement
> -fwrapv -Wno-address-of-packed-member -Wno-unused-label
> -Wno-sign-compare -Wno-unused-parameter -Wno-old-style-declaration
> -Wno-ignored-qualifiers -Wno-clobbered -Wno-missing-field-initializers
> -Wno-implicit-fallthrough -Wno-stringop-overflow -Wno-cast-function-type
> -Wtype-limits -Wshift-negative-value -Wshift-overflow=2
> -Wduplicated-cond -Wnull-dereference
>   OPTIONS = USE_PCRE2=1 USE_PCRE2_JIT=1 USE_GETADDRINFO=1 USE_OPENSSL=1
> USE_LUA=1 USE_ZLIB=1
>
> Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER -PCRE
> -PCRE_JIT +PCRE2 +PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD
> -PTHREAD_PSHARED -REGPARM -STATIC_PCRE -STATIC_PCRE2 +TPROXY
> +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H -VSYSCALL +GETADDRINFO
> +OPENSSL +LUA +FUTEX +ACCEPT4 -MY_ACCEPT4 +ZLIB -SLZ +CPU_AFFINITY +TFO
> +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER
> +PRCTL +THREAD_DUMP -EVPORTS
>
> Default settings :
>   bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
>
> Built with multi-threading support (MAX_THREADS=64, default=2).
> Built with OpenSSL version : OpenSSL 1.1.1g  21 Apr 2020
> Running on OpenSSL version : OpenSSL 1.1.1g  21 Apr 2020
> OpenSSL library supports TLS extensions : yes
> OpenSSL library supports SNI : yes
> OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
> Built with Lua version : Lua 5.3.5
> Built with network namespace support.
> Built with transparent proxy support using: IP_TRANSPARENT
> IPV6_TRANSPARENT IP_FREEBIND
> Built with zlib version : 1.2.11
> Running on zlib version : 1.2.11
> Compression algorithms supported : identity("identity"),
> deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
> Built with PCRE2 version : 10.35 2020-05-09
> PCRE2 library supports JIT : yes
> Encrypted password support via crypt(3): yes
> Built with the Prometheus exporter as a service
>
> Available polling systems :
>   epoll : pref=300,  test result OK
>    poll : pref=200,  test result OK
>  select : pref=150,  test result OK
> Total: 3 (3 usable), will use epoll.
>
> Available multiplexer protocols :
> (protocols marked as  cannot be specified using 'proto' keyword)
> h2 : mode=HTX    side=FE|BE mux=H2
>   h2 : mode=HTTP   side=FE    mux=H2
>     : mode=HTX 

TLS handshake error

2020-09-15 Thread vcjouni

Hi!

We can not get haproxy-ingress to work with TLS authentication. Only 
option to get this work is by using force-tlsv12 and then only Chrome 
works. Problem is TLS handshake decrypt error when using RSA-PSS 
signature algorithm, handshake fails every time. When we use 
force-tlsv12, only Chrome will change signature back to pkcs1 and then 
session is ok. Safari, Edge or IE does not work with any option and they 
keep offering RSA-PSS signature.


This same CA bundle and cards has been used before with citrix netscaler 
ingress controller without problems (TLSv1.2). Smart cards are 
government provided FINEID cards, so we can't test them command line, 
only by using those cards with browser.


I already discussed with haproxy-ingress builder jcmoraisjr and he 
suggested us to ask from haproxy mailing list.


Docker image: image: quay.io/jcmoraisjr/haproxy-ingress

# haproxy -vv
HA-Proxy version 2.0.17 2020/07/31 - https://haproxy.org/
Build options :
  TARGET  = linux-glibc
  CPU = generic
  CC  = gcc
  CFLAGS  = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement 
-fwrapv -Wno-address-of-packed-member -Wno-unused-label 
-Wno-sign-compare -Wno-unused-parameter -Wno-old-style-declaration 
-Wno-ignored-qualifiers -Wno-clobbered -Wno-missing-field-initializers 
-Wno-implicit-fallthrough -Wno-stringop-overflow -Wno-cast-function-type 
-Wtype-limits -Wshift-negative-value -Wshift-overflow=2 
-Wduplicated-cond -Wnull-dereference
  OPTIONS = USE_PCRE2=1 USE_PCRE2_JIT=1 USE_GETADDRINFO=1 USE_OPENSSL=1 
USE_LUA=1 USE_ZLIB=1


Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER -PCRE 
-PCRE_JIT +PCRE2 +PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD 
-PTHREAD_PSHARED -REGPARM -STATIC_PCRE -STATIC_PCRE2 +TPROXY 
+LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H -VSYSCALL +GETADDRINFO 
+OPENSSL +LUA +FUTEX +ACCEPT4 -MY_ACCEPT4 +ZLIB -SLZ +CPU_AFFINITY +TFO 
+NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER 
+PRCTL +THREAD_DUMP -EVPORTS


Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=2).
Built with OpenSSL version : OpenSSL 1.1.1g  21 Apr 2020
Running on OpenSSL version : OpenSSL 1.1.1g  21 Apr 2020
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.5
Built with network namespace support.
Built with transparent proxy support using: IP_TRANSPARENT 
IPV6_TRANSPARENT IP_FREEBIND

Built with zlib version : 1.2.11
Running on zlib version : 1.2.11
Compression algorithms supported : identity("identity"), 
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")

Built with PCRE2 version : 10.35 2020-05-09
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with the Prometheus exporter as a service

Available polling systems :
  epoll : pref=300,  test result OK
   poll : pref=200,  test result OK
 select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as  cannot be specified using 'proto' keyword)
  h2 : mode=HTX    side=FE|BE mux=H2
  h2 : mode=HTTP   side=FE    mux=H2
    : mode=HTX    side=FE|BE mux=H1
    : mode=TCP|HTTP   side=FE|BE mux=PASS

Available services :
    prometheus-exporter

Available filters :
    [SPOE] spoe
    [COMP] compression
    [CACHE] cache
    [TRACE] trace

haproxy configmap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: haproxy-ingress
  namespace: ingress-controller
data:
  ssl-headers-prefix: HTTP_X_SSL
  max-connections: "1"
#  ssl-options: ssl-min-ver TLSv1.2 no-tls-tickets
#  ssl-options: force-tlsv12 no-tls-tickets
  ssl-options: no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
  ssl-cipher-suites: 
TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
  #  ssl-ciphers: 
ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384 

#  ssl-ciphers: 
RSA+SHA256:RSA+SHA512:RSA+SHA384:ECDSA+SHA256:ECDSA+SHA512:ECDSA+SHA384
  ssl-ciphers: 
ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384

  ssl-dh-default-max-size: "2048"
  syslog-endpoint: "10.32.121.202:514"
  syslog-length: "2048"
  http-log-format: "%ci:%cp\\ [%t]\\ %ft\\ %b/%s\\ 
%Tq/%Tw/%Tc/%Tr/%Tt\\ %ST\\ %B\\ %CC\\ \\ %CS\\ %tsc\\ 
%ac/%fc/%bc/%sc/%rc\\ %sq/%bq\\ %hr\\ %hs\\ %{+Q}r\\ ssl_version=%sslv\\ 
ssl_cypher=%sslc}\\ C_VERIFY=%{+Q}[ssl_c_verify]\\ 
C_ERR=%{+Q}[ssl_c_err]\\ C_S_DN=%{+Q}[ssl_c_s_dn]\\ 
C_i_DN=%{+Q}[ssl_c_i_dn]\\ 

Re: [PATCH] BUILD: makefile: Update feature flags for Solaris / FreeBSD / NetBSD / OpenBSD

2020-09-15 Thread Илья Шипицин
Brad, can you also review cirrus-ci job for freebsd?

On Tue, Sep 15, 2020, 1:06 PM Brad Smith  wrote:

> On 9/15/2020 3:45 AM, Lukas Tribus wrote:
> > On Tue, 15 Sep 2020 at 09:05, Brad Smith  wrote:
>  NetBSD 8.0 adds support for accept4() and closefrom(). Enable
> getaddrinfo().
> >>> We just had to disable threading on OpenBSD 6.7 for the build to
> succeed:
> >>>
> >>> https://github.com/haproxy/haproxy/issues/725
> >>>
> >>> Did you actually test all those combinations? Because otherwise it
> >>> does not seem like a good idea to commit this.
> >> I know. I saw that. That was wrong. The other diff I submitted switching
> >> the compiler default
> >> as well as this came from that bogus diff.
> >>
> >> 2 of the 4 targets are (FreeBSD / OpenBSD). The other 2 are just based
> >> on being aware of the API
> >> and what has been implemented and when. I'll split up my diff and send
> >> in the tested bits first.
> > Ok thanks (I was concerned about disabling threading on OpenBSD as well).
> >
> > Please add the information that your previous patch switching gcc to
> > cc is required for the OpenBSD change to work; it's obvious to me now,
> > but when looking at the single change we are not always aware of the
> > history (especially later during bisects).
>
> I will do so. Yes, those details can be important.
>
>


Re: [PATCH] BUILD: makefile: Update feature flags for Solaris / FreeBSD / NetBSD / OpenBSD

2020-09-15 Thread Brad Smith

On 9/15/2020 3:45 AM, Lukas Tribus wrote:

On Tue, 15 Sep 2020 at 09:05, Brad Smith  wrote:

NetBSD 8.0 adds support for accept4() and closefrom(). Enable getaddrinfo().

We just had to disable threading on OpenBSD 6.7 for the build to succeed:

https://github.com/haproxy/haproxy/issues/725

Did you actually test all those combinations? Because otherwise it
does not seem like a good idea to commit this.

I know. I saw that. That was wrong. The other diff I submitted switching
the compiler default
as well as this came from that bogus diff.

2 of the 4 targets are (FreeBSD / OpenBSD). The other 2 are just based
on being aware of the API
and what has been implemented and when. I'll split up my diff and send
in the tested bits first.

Ok thanks (I was concerned about disabling threading on OpenBSD as well).

Please add the information that your previous patch switching gcc to
cc is required for the OpenBSD change to work; it's obvious to me now,
but when looking at the single change we are not always aware of the
history (especially later during bisects).


I will do so. Yes, those details can be important.



Re: [PATCH] BUILD: makefile: Update feature flags for Solaris / FreeBSD / NetBSD / OpenBSD

2020-09-15 Thread Lukas Tribus
On Tue, 15 Sep 2020 at 09:05, Brad Smith  wrote:
> >> NetBSD 8.0 adds support for accept4() and closefrom(). Enable 
> >> getaddrinfo().
> > We just had to disable threading on OpenBSD 6.7 for the build to succeed:
> >
> > https://github.com/haproxy/haproxy/issues/725
> >
> > Did you actually test all those combinations? Because otherwise it
> > does not seem like a good idea to commit this.
>
> I know. I saw that. That was wrong. The other diff I submitted switching
> the compiler default
> as well as this came from that bogus diff.
>
> 2 of the 4 targets are (FreeBSD / OpenBSD). The other 2 are just based
> on being aware of the API
> and what has been implemented and when. I'll split up my diff and send
> in the tested bits first.

Ok thanks (I was concerned about disabling threading on OpenBSD as well).

Please add the information that your previous patch switching gcc to
cc is required for the OpenBSD change to work; it's obvious to me now,
but when looking at the single change we are not always aware of the
history (especially later during bisects).


Thanks,

Lukas



Re: Is adaptive circuit breaking in the roadmap for 2.3/2.4?

2020-09-15 Thread Willy Tarreau
On Tue, Sep 15, 2020 at 09:24:32AM +0200, Willy Tarreau wrote:
(...)
> There's no such ongoing work that I'm aware of but that has always
> been a subject of interest to me (I even wrote down the algorithm to
> compute weights by measured response times using a low-pass filter a
> decade ago but I lost my notes and never felt like doing work again).
> So if anyone is interested in this subject, we can continue this
> conversation till we reach something that looks like a possible design
> roadmap.

By the way there is another aspect that has always interested me around
this, which is that it would simplify the confuguration of a server's
maxconn when it's shared between multiple backends or even multiple LB
nodes. It will certainly lead to oscillations sometimes but that's
manageable by using randomness in increments so that multiple LB nodes
cannot enter into resonance.

Willy



Re: Is adaptive circuit breaking in the roadmap for 2.3/2.4?

2020-09-15 Thread Willy Tarreau
Hi Pavlos!

On Sat, Sep 12, 2020 at 11:45:12AM +0200, Pavlos Parissis wrote:
> Hi old friends!,
> 
> Is in the roadmap the addition of a circuit breaking which adapts its 
> settings using real-time data?
> I believe we discussed this in the last HAProxyConf with a group of people, 
> but I don't remember if there were
> , back then, concrete plans to work on it.
> 
> I know that something similar can  be accomplished by using agent check, but 
> IMHO it less ideal.
> - watch 
> https://www.youtube.com/watch?v=CQvmSXlnyeQ=PLj6h78yzYM2O1wlsM-Ma-RYhfT5LKq0XC=21
> 
> - read 
> https://www.envoyproxy.io/docs/envoy/v1.15.0/configuration/http/http_filters/adaptive_concurrency_filter

Yep as you say we can indeed already do it using agent checks. I know
it's not ideal, but what matters to me is that it indicates we already
have the required core functionalities to do something like this.

The proper way to implement such a mechanism is by using feedback
reaction, similar to how MPPT solar panel controllers work: you never
know if you're sending enough load to keep the servers busy, so you
constantly need to try to send a bit more to see if any service
degradation occurs or not. The degradation happens by response time
suddenly becoming affine to the number of concurrent requests. Then
the decision of where to stick needs to be made based on the choice
of lower latency (bottom of curve) or higher bandwidth (top of curve).
A simpler approach would consist in having a setting indicating how
much extra response time is acceptable compared to the measued optimal
one.

I also think that it remains important to let the user configure lower
and upper bounds that guarantee safe operations. And that's typically
what could be done using the minconn and maxconn values. Instead of
using the backend's fullconn setting we would rely on a response time
measurement.

Or maybe that could be the right opportunity for splitting connection
concurrency from request concurrency, keeping connections for the hard
limits and using request concurrency for softer limits.

There's no such ongoing work that I'm aware of but that has always
been a subject of interest to me (I even wrote down the algorithm to
compute weights by measured response times using a low-pass filter a
decade ago but I lost my notes and never felt like doing work again).
So if anyone is interested in this subject, we can continue this
conversation till we reach something that looks like a possible design
roadmap.

Cheers,
Willy



[PATCH] BUILD: makefile: Update feature flags for FreeBSD

2020-09-15 Thread Brad Smith
This updates the feature flags for FreeBSD.

FreeBSD 10 adds support for accept4().

Enable getaddrinfo().

>From the FreeBSD port / package.



diff --git a/Makefile b/Makefile
index 934ca1666..e69870595 100644
--- a/Makefile
+++ b/Makefile
@@ -363,11 +363,11 @@ ifeq ($(TARGET),solaris)
   TARGET_LDFLAGS = -lnsl -lsocket
 endif
 
-# FreeBSD 5 and above
+# FreeBSD 10 and above
 ifeq ($(TARGET),freebsd)
   set_target_defaults = $(call default_opts, \
 USE_POLL USE_TPROXY USE_LIBCRYPT USE_THREAD USE_CPU_AFFINITY USE_KQUEUE   \
-USE_CLOSEFROM)
+USE_ACCEPT4 USE_CLOSEFROM USE_GETADDRINFO)
 endif
 
 # Mac OS/X



Re: [PATCH] BUILD: makefile: Update feature flags for Solaris / FreeBSD / NetBSD / OpenBSD

2020-09-15 Thread Brad Smith

On 9/14/2020 5:39 AM, Lukas Tribus wrote:

Hello Brad,

On Sun, 13 Sep 2020 at 09:08, Brad Smith  wrote:

The following diff updates the feature flags for Solaris / FreeBSD / NetBSD / 
OpenBSD.

Bump the baseline Solaris to 9 which intruduced closefrom().

FreeBSD 10 is already EOL for support but its the new baseline. Introduces 
accept4().
Enable getaddrinfo(). The FreeBSD port enables these.

OpenBSD 6.3 is pretty old but it brings a compiler that supports TLS for the 
threads
support. 5.7 already supported closefrom(). Enable getaddrinfo().

NetBSD 8.0 adds support for accept4() and closefrom(). Enable getaddrinfo().

We just had to disable threading on OpenBSD 6.7 for the build to succeed:

https://github.com/haproxy/haproxy/issues/725

Did you actually test all those combinations? Because otherwise it
does not seem like a good idea to commit this.


I know. I saw that. That was wrong. The other diff I submitted switching 
the compiler default

as well as this came from that bogus diff.

2 of the 4 targets are (FreeBSD / OpenBSD). The other 2 are just based 
on being aware of the API
and what has been implemented and when. I'll split up my diff and send 
in the tested bits first.




Re: [PATCH] travis-ci: report asan findings in more intelligent way

2020-09-15 Thread Willy Tarreau
On Tue, Sep 15, 2020 at 11:46:02AM +0500,  ??? wrote:
> ping

Sorry, I missed it, now merged.

Thanks!
Willy



Re: [PATCH] travis-ci: report asan findings in more intelligent way

2020-09-15 Thread Илья Шипицин
ping

пт, 4 сент. 2020 г. в 11:53, Илья Шипицин :

> any attention to this ?
>
> чт, 27 авг. 2020 г. в 21:08, Илья Шипицин :
>
>> Hello,
>>
>> asan findings were mixed with reg-tests errors and it was hard to
>> investigate.
>>
>> Cheers,
>> Ilya Shipitcin
>>
>


Re: [PATCH] BUILD: makefile: change default value of CC from gcc to cc

2020-09-15 Thread Brad Smith

On 9/15/2020 1:31 AM, Willy Tarreau wrote:

On Mon, Sep 14, 2020 at 11:42:47PM -0400, Brad Smith wrote:

It's not about package maintainers as they know to do the right
thing. It's
about end users that don't know any better. In this case someone
submitted
a bogus patch which was commited based on using the wrong compiler.

why they do not just use package system ?

I don't know, but it's beside the point now. All kinds of people for various
reasons (re)build software outside of pre-built packages.

Well, after sleeping over it, I think your points above are valid. We
used to support exclusively gcc by extensively using GNU extensions,
but over time other compilers like icc, clang, and tcc have adopted a
lot of these GNU extensions and are compatible. And it makes sense not
to ask them to call themselves gcc of course. So I think we can do this,
and that 2.3 is the right version to do it, and it will leave us enough
time to see if any issues are reported. In practice I expect the vast
majority of systems where development packages are installed to have
at least one working "cc".

I'm going to take your patch, thanks!


Thanks.