[ANNOUNCE] haproxy-2.6.7

2022-12-02 Thread Christopher Faulet
verflow on qc_release_lost_pkts
  MINOR: ncbuf: complete doc for ncb_advance()
  BUG/MEDIUM: quic: fix unsuccessful handshakes on ncb_advance error
  BUG/MEDIUM: quic: fix memleak for out-of-order crypto data
  MINOR: quic: complete traces/debug for handshake
  DOC: configuration: fix quic prefix typo
  MINOR: quic: report error if force-retry without cluster-secret
  MINOR: global: generate random cluster.secret if not defined
  DOC: quic: add note on performance issue with listener contention
  BUG/MEDIUM: quic: fix datagram dropping on queueing failed
  CLEANUP: ncbuf: remove ncb_blk args by value
  CLEANUP: ncbuf: inline small functions
  CLEANUP: ncbuf: use standard BUG_ON with DEBUG_STRICT

Aurelien DARRAGON (12):
  BUG/MINOR: hlua: fixing hlua_http_msg_del_data behavior
  BUG/MINOR: hlua: fixing hlua_http_msg_insert_data behavior
  BUG/MINOR: hlua: _hlua_http_msg_delete incorrect behavior when offset is 
used
  BUG/MINOR: hlua: hlua_channel_insert_data() behavior conflicts with 
documentation
  DOC: configuration: missing 'if' in tcp-request content example
  BUG/MINOR: log: fixing bug in tcp syslog_io_handler Octet-Counting
  BUG/MEDIUM: wdt/clock: properly handle early task hangs
  BUG/MINOR: http_ana/txn: don't re-initialize txn and req var lists
  BUG/MINOR: cfgparse-listen: fix ebpt_next_dup pointer dereference on proxy 
"from" inheritance
  BUG/MINOR: log: fix parse_log_message rfc5424 size check
  DOC: configuration.txt: add default_value for table_idle signature
  DOC: configuration.txt: fix typo in table_idle signature

Christopher Faulet (31):
  REGTESTS: 4be_1srv_smtpchk_httpchk_layer47errors: Return valid SMTP 
replies
  BUG/MINOR: stream: Perform errors handling in right order in stream_new()
  BUG/MEDIUM: stconn: Reset SE descriptor when we fail to create a stream
  BUG/MEDIUM: resolvers: Remove aborted resolutions from query_ids tree
  DOC: config: Fix pgsql-check documentation to make user param mandatory
  BUG/MINOR: http-fetch: Update method after a prefetch in smp_fetch_meth()
  BUG/MINOR: mux-h1: Account consumed output data on synchronous connection 
error
  MINOR: smtpchk: Update expect rule to fully match replies to EHLO commands
  BUG/MEDIUM: httpclient: Don't set EOM flag on an empty HTX message
  MINOR: httpclient/lua: Don't set req_payload callback if body is empty
  BUG/MINOR: log: Preserve message facility when the log target is a ring 
buffer
  BUG/MINOR: ring: Properly parse connect timeout
  BUG/MEDIUM: compression: handle rewrite errors when updating response 
headers
  BUG/MINOR: sink: Only use backend capability for the sink proxies
  BUG/MINOR: sink: Set default connect/server timeout for implicit ring 
buffers
  BUG/MAJOR: stick-table: don't process store-response rules for applets
  BUG/MINOR: http-htx: Fix error handling during parsing http replies
  BUG/MINOR: resolvers: Don't wait periodic resolution on healthcheck 
failure
  BUG/MINOR: resolvers: Set port before IP address when processing SRV 
records
  BUG/MINOR: mux-fcgi: Be sure to send empty STDING record in case of 
zero-copy
  BUG/MEDIUM: mux-fcgi: Avoid value length overflow when it doesn't fit at 
once
  REG-TESTS: cache: Remove T-E header for 304-Not-Modified responses
  BUG/MEDIUM: listener: Fix race condition when updating the global mngmt 
task
  BUG/MEDIUM: raw-sock: Don't report connection error if something was 
received
  BUILD: peers: Remove unused variables
  BUG/MINOR: http-htx: Don't consider an URI as normalized after a set-uri 
action
  BUILD: http-htx: Silent build error about a possible NULL start-line
  BUILD: ssl-sock: Silent error about NULL deref in 
ssl_sock_bind_verifycbk()
  BUG/MINOR: mux-h1: Fix handling of 408-Request-Time-Out
  Revert "BUG/MINOR: http-htx: Don't consider an URI as normalized after a 
set-uri action"
  MINOR: mux-h1: add the expire task and its expiration date in "show fd"

Emeric Brun (1):
  BUG/MEDIUM: peers: messages about unkown tables not correctly ignored

Erwan Le Goas (1):
  BUG/MINOR: config: don't count trailing spaces as empty arg (v2)

Fatih Acar (1):
  BUG/MINOR: checks: update pgsql regex on auth packet

Frédéric Lécaille (11):
  MINOR: quic: New quic_cstream object implementation
  MINOR: quic: Extract CRYPTO frame parsing from qc_parse_pkt_frms()
  MINOR: quic: Use a non-contiguous buffer for RX CRYPTO data
  BUG/MINOR: quic: Stalled 0RTT connections with big ClientHello TLS message
  MINOR: quic: Split the secrets key allocation in two parts
  BUILD: ssl_sock: bind_conf uninitialized in ssl_sock_bind_verifycbk()
  BUILD: quic: QUIC mux build fix for 32-bit build
  BUILD: quic: Fix build for m68k cross-compilation
  BUG/MAJOR: quic: Crash upon retransmissi

Re: [PATCH] spelling fixes

2022-11-30 Thread Christopher Faulet

Le 11/30/22 à 12:24, Илья Шипицин a écrit :

hello,

yet another spelling patch.



Thanks, merged !

--
Christopher Faulet




Re: Multiple http-check in backend

2022-11-24 Thread Christopher Faulet

Le 11/23/22 à 06:59, Carlhens Baptiste a écrit :

Hi,

I am trying to figure out how to use multiple |http-check| in my backend. I 
can’t figure out the proper syntax. Any help is appreciated.


backend avax-mainnet
option httpchk
stick-table type ip size 1m expire 1h
stick on src
balance leastconn
http-check send meth POST uri /ext/info hdr Content-Type application/json body 
"{"jsonrpc":"2.0","method":"info.isBootstrapped","params":[{"chain": "C"}],"i>

http-check expect rstring "isBootstrapped":true
http-check send meth POST uri /ext/info hdr Content-Type application/json body 
"{"jsonrpc":"2.0","method":"info.isBootstrapped","params":[{"chain": "q2aTwKuy>

http-check expect rstring "isBootstrapped":true
http-check send meth POST uri /ext/info hdr Content-Type application/json body 
"{"jsonrpc":"2.0","method":"info.isBootstrapped","params":[{"chain": "2K33xS9A>

http-check expect rstring "isBootstrapped":true
default-server inter 5s fall 3 rise 2 on-marked-down shutdown-sessions
server mun2np001 10.0.2.10:9650 check



Hi,

To send several requests, you must add an "http-check connect" rule between each
request. It means the first request must be an "http-check connect" rule. The
first one is implicit when there is only one request. In this case, it must be
defined.

In addition, if you want to match the double quotes, you must escape them.
Double quotes must also be escape for the body string. Spaces too. But it is
probably easier to enclose all the body between simple quotes.

Finally, a "string" matching method is a better solution in your case because
your pattern is not a regex.

To sum up, here is how it should be written:

http-check connect
http-check send meth POST uri /ext/info hdr Content-Type application/json body 
'"{"jsonrpc":"2.0","method":"info.isBootstrapped","params":[{"chain": "C"}],"i>'
http-check expect string \"isBootstrapped\":true
httpc-check connect
http-check send meth POST uri /ext/info hdr Content-Type application/json body 
'"{"jsonrpc":"2.0","method":"info.isBootstrapped","params":[{"chain": 
"q2aTwKuy>'
http-check expect string \"isBootstrapped\":true
http-check connect
http-check send meth POST uri /ext/info hdr Content-Type application/json body 
'"{"jsonrpc":"2.0","method":"info.isBootstrapped","params":[{"chain": 
"2K33xS9A>'
http-check expect string \"isBootstrapped\":true

--
Christopher Faulet




Re: How to return 429 Status Code instead of 503

2022-11-17 Thread Christopher Faulet

Le 11/17/22 à 08:13, Jarno Huuskonen a écrit :

Hello,

On Tue, 2022-11-08 at 09:30 +0530, Chilaka Ramakrishna wrote:

On queue timeout, currently HAProxy throws 503, But i want to return 429,
I understand that 4xx means a client problem and client can't help here.
But due to back compatibility reasons, I want to return 429 instead of
503. Is this possible ?


errorfile 503 /path/to/429.http
(http://cbonte.github.io/haproxy-dconv/2.6/configuration.html#4-errorfile)

Or maybe it's possible with http-error
(http://cbonte.github.io/haproxy-dconv/2.6/configuration.html#http-error)



This will work, but all errors generating 503 responses will be replaced by 429 
responses. For now, there is no way to make difference between a queue timeout 
and connection error for instance.


--
Christopher Faulet




Re: change accept() behaviour under maxconn

2022-11-02 Thread Christopher Faulet

Le 10/20/22 à 07:47, William Dauchy a écrit :

Hello,

I was looking at maxconn behaviour on frontend side. I understood we
don't trigger accept() once maxconn is reached
https://git.haproxy.org/?p=haproxy.git;a=blob;f=src/listener.c;h=412af94a160662aa11802dd58512c6410c9b56da;hb=HEAD#l920
I wondered whether it would be an acceptable patch to add an option in
order to accept() and close() the connection right after during such
event. The context would be to "fail fast" on some environment where
it is preferable to let know the client as soon as possible we can't
accept the connection. In an environment where the haproxy instance is
the second level of proxies, we want to fail fast instead of waiting
for the end of tcp timeout to connect.

Thanks,


Hi William,

Sorry for the delay ! It is probably not so handy to configure but you may use a 
"tcp-request connection reject" with an ACL based on fe_conn sample fetch (or 
any other sample).


--
Christopher Faulet




Re: Haproxy :: compression issue for IPv6

2022-10-24 Thread Christopher Faulet

Le 10/21/22 à 11:12, Narendra Patel a écrit :

Hi Christopher,

This did help. The buffer seemed to have been exhausted.
Increasing it by 100 bytes to 1124 got the issue resolved.

Thank you so much for your response. It was really helpful :)



FYI, I pushed a fix in 2.7-DEV. It will be backported ASAP. Thanks !

--
Christopher Faulet




Re: Haproxy :: compression issue for IPv6

2022-10-21 Thread Christopher Faulet

Le 10/21/22 à 08:16, Narendra Patel a écrit :

Hi Team,

We are having one issue with routing traffic through HAProxy.

Have configured IPv6 for our Google load balancer backed by HAProxy. The 
architecture looks like below:


GLB --> L1 HAProxy --> Kong Gateway --> L2 HAProxy --> App node

L1 HAProxy has the logic for compression.

# zlib Compression
     compression algo gzip
     compression type text/html text/tab-separated-values text/plain text/xml 
text/csv text/css text/javascript application/xml application/x-javascript 
application/javascript


But while evaluating IPv6, we are seeing one peculiar error. For IPV6 request 
for a css file, we see that HAProxy injects a "Content-Encoding: gzip" Header, 
but does not really compress the file thus corrupting the response.


We tried doing a tcpdump on L1 HAProxy and Kong to check.
Besides the `x-forwarded-for` header, HAProxy sends the same request to Kong for 
both IPv4 and IPv6.
Also Kong responds with the same header values in response to HAProxy for both 
IPv4 and IPv6.
Content length also is the same in Kong's response to HAProxy for both IPv4 and 
IPv6. So Kong is not doing any compression.


HAProxy version is : HAProxy version 2.6.4-2a2078c 2022/08/22

Attaching the tcpdump details for HAProxy and Kong for both IPv4 and IPv6. Have 
masked a few values for compliance.


P.S: We are on GCP and the IPv6 setup looks like this : 
https://cloud.google.com/load-balancing/docs/ipv6#ipv6_termination_and_proxy 
<https://cloud.google.com/load-balancing/docs/ipv6#ipv6_termination_and_proxy>


Any directions on this would be really appreciated.


Hi,

In the HAProxy IPv6 tcpdump, there is no "Vary" header. It means HAProxy fails 
to add it. When this happens, an error is triggered and the compression is not 
performed. So here, there is an issue because "Content-Encoding" header is not 
removed while it should be. As a workaround, you may try to increase 
"tune.maxrewrite" value in the global section. It seems you add several headers 
in your L1 HAProxy, exceeding the default reserve (1024 bytes). If the response 
payload is received with the headers, the buffer may be full, preventing some 
headers additions.


Note that it may also happen on any headers added via http-response rules. In 
this case, this will generate an internal error, and a 500 response will be 
returned to the client. So, we should fix the compression related error. But, it 
remains a good idea to increase "tune.maxrewrite" value in your case.


Regards,
--
Christopher Faulet




Re: clarify close behaviour on http-request rules

2022-10-18 Thread Christopher Faulet

Le 10/18/22 à 18:24, William Dauchy a écrit :

On Tue, Oct 18, 2022 at 4:15 PM Christopher Faulet  wrote:

On all HTX versions, K/A and close modes are handled in the H1 multiplexer.
Thus, on these versions, http_reply_and_close() is only closing the stream. The
multiplexer is responsible to close the client connection or not.


ok, thanks for the details.


Note there is a case where the connection client may be closed. If the HAProxy
response is returned before the end of the request, the client connection is
closed. There is no (not yet) draining mode at the mux level.


That particular point is very interesting to me. Would you have a
pointer so that I can better understand where it is happening in the
code?



Of course: https://github.com/haproxy/haproxy/blob/master/src/mux_h1.c#L2172

--
Christopher Faulet




Re: clarify close behaviour on http-request rules

2022-10-18 Thread Christopher Faulet

Le 10/18/22 à 13:01, William Dauchy a écrit :

Hi Christopher,

Thanks for your reply.

On Tue, Oct 18, 2022 at 10:22 AM Christopher Faulet  wrote:

Since the 2.2, HAProxy responses don't close the client connections in K/A mode,
except for 400-Bad-Request and 408-Request-Time-out. It is the behavior with
default error messages. Of course, if a "Connection: close" header is found in a
custom reply, the client connection is closed.

Thus, in your case, if there is no "Connection: close" header in the response,
both http-request action (return and deny) should do the same and keep the
client connection alive.

So if you observe connection closing, it may be for something else. Maybe a
timeout. Or a bug...


ok I will investigate more.
To come back on the code path, was I wrong for the
`http_reply_and_close` difference?
Would you have a pointer so I may better understand where
`http_reply_and_close` would not close the connection in K/A mode?



On all HTX versions, K/A and close modes are handled in the H1 multiplexer. 
Thus, on these versions, http_reply_and_close() is only closing the stream. The 
multiplexer is responsible to close the client connection or not.


On pre-HTX versions, when http_reply_and_close() is used, the client connection 
is also closed. It is a limitation of of HAProxy versions using the legacy HTTP.


Note there is a case where the connection client may be closed. If the HAProxy 
response is returned before the end of the request, the client connection is 
closed. There is no (not yet) draining mode at the mux level.


--
Christopher Faulet




Re: clarify close behaviour on http-request rules

2022-10-18 Thread Christopher Faulet

Le 10/17/22 à 16:15, William Dauchy a écrit :

Hello,

I am trying to clarify in which case a tcp connection might be closed
following those rules:
- http-request return
- http-request deny
unless I missed something I have not been able to see the answer within the doc.

General context being, we are using `option http-keep-alive`; also our
haproxy is behind another L7 proxy; it means the later do not expect
the connection to be loosely closed.

[cloud L7 LB] <--> [haproxy] <--> backend

 From what I read in
https://git.haproxy.org/?p=haproxy.git;a=blob;f=src/http_ana.c;h=2b2cfdc56103f313d766143f9016d91200065092;hb=HEAD#l354
I got:
- `http-request return` gets `HTTP_RULE_RES_ABRT`; this flag leads to
`return_prx_cond`, which is not calling `http_reply_and_close`
- `http-request deny` gets `HTTP_RULE_RES_DENY`; this flag leads to
`deny`, which is calling `http_reply_and_close`

We are using haproxy v2.4.x but I think the behaviour did not change
in an earlier version.

Some followup questions:

- do we confirm `http-request return` does not close the connection
while `http-request deny` does?
meaning:
- http-request return status 403
- http-request deny
don't have the same behaviour?
- from what I understood, using http-keep-alive does not influence the
behaviour of `http_reply_and_close`, is that correct?
- while using `http-request deny` which might close the connection, we
see some requests on the cloud LB side being ended in error 502. We
suspect those are the next requests, where the cloud LB tried to use
the same previous connection. For now it is hard to understand why the
cloud LB would not detect the close. Is there a moment where haproxy
would loosely close the connection which might explain the behavior?

Thanks in advance,



Hi William,

Since the 2.2, HAProxy responses don't close the client connections in K/A mode, 
except for 400-Bad-Request and 408-Request-Time-out. It is the behavior with 
default error messages. Of course, if a "Connection: close" header is found in a 
custom reply, the client connection is closed.


Thus, in your case, if there is no "Connection: close" header in the response, 
both http-request action (return and deny) should do the same and keep the 
client connection alive.


So if you observe connection closing, it may be for something else. Maybe a 
timeout. Or a bug...


--
Christopher Faulet




Re: [PATCH] BUG/MINOR: checks: update pgsql regex on auth packet

2022-10-04 Thread Christopher Faulet

Le 9/26/22 à 17:27, Fatih Acar a écrit :

This patch adds support to the following authentication methods:

- AUTH_REQ_GSS (7)
- AUTH_REQ_SSPI (9)
- AUTH_REQ_SASL (10)

Note that since AUTH_REQ_SASL allows multiple authentication mechanisms
such as SCRAM-SHA-256 or SCRAM-SHA-256-PLUS, the auth payload length may
vary since the method is sent in plaintext. In order to allow this, the
regex now matches any payload length.

This partially fixes Github issue #1508 since user authentication is
still broken but should restore pre-2.2 behavior.

This should be backported up to 2.2.

Signed-off-by: Fatih Acar 


Merged, thanks !

--
Christopher Faulet




Re: [PATCH] BUG/MINOR: checks: update pgsql regex on auth packet

2022-09-30 Thread Christopher Faulet

Le 9/26/22 à 17:27, Fatih Acar a écrit :

This patch adds support to the following authentication methods:

- AUTH_REQ_GSS (7)
- AUTH_REQ_SSPI (9)
- AUTH_REQ_SASL (10)

Note that since AUTH_REQ_SASL allows multiple authentication mechanisms
such as SCRAM-SHA-256 or SCRAM-SHA-256-PLUS, the auth payload length may
vary since the method is sent in plaintext. In order to allow this, the
regex now matches any payload length.

This partially fixes Github issue #1508 since user authentication is
still broken but should restore pre-2.2 behavior.

This should be backported up to 2.2.



Thanks ! Sorry for the delay. We will review it and merge it soon.

--
Christopher Faulet




[ANNOUNCE] haproxy-2.4.19

2022-09-28 Thread Christopher Faulet
 queued messages, slowing down all processing.

  * Willy managed to trigger an error on reload where the old process died
saying "t->tid >= 0 && t->tid != tid". This is caused by the deinit code
that needs to stop stuff initialized on other threads, and as such it
violates some consistency checks. The check was relaxed to ignore the
stopping condition.

  * Characters escaping process in log messages was not correctly processing
strings coming from sample fetches truncating the output string.

  * Using HAProxy built with PCRE2_JIT with a lib built without would fail
to match. Now it will fall back to the regular match.

  * Agent-check could be delayed by ~200ms due to TCP QUICKACK being
disabled by default.

  * Reading from the rings could also occasionally freeze at high rate if
the reader had to stop due to a buffer full while the writer had already
stopped due to a ring full.

  * In Lua, it was possible to hand reading HTTP payload (by line or not)
from an HTTP applet because we relied on a transiant HTX flags to detect
the end of the message instead of relying on the channel flag.

  * A 60s delay could be experienced after stopping HAProxy. This was
happening when a signal was received before entering the poller and
without any activity on the process. In mworker mode, if a worker exited
and the SIGCHLD signal was delivered at the right time to the master,
this one could be stuck for 60s. The timeout is now set to 0 in this
specific case.

  * In master-worker mode, it was reported that HAProxy was consuming too
much memory because of a too high maxconn value. To limit memory
consumption, the master is now using a default maxconn value in
wait-mode.

  * In HTTP/1.1, the matching between the authority and the Host header value
for CONNECT requests was buggy. An exact match was performed ignoring any
normalization on the port. For CONNECT request the authority must contain
the port but it may be omitted from the host header value for default
ports (80 or 443). The matching was fixed to properly handle this case.

The following improvements were also backported:

  * Headers case adjustment in H1 is now available for TCP proxies. It was
an issue for HTTP health-checks on backend side or for TCP connections
upgraded to HTTP on frontend side.

  * The stats applet was reported paused frontends as OPEN. Now, these
frontends are reported as PAUSED.

  * Encrypted password in Userlists are now supported on NetBSD

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : https://www.haproxy.org/
   Documentation: https://docs.haproxy.org/
   Wiki : https://github.com/haproxy/wiki/wiki
   Discourse: https://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Sources  : https://www.haproxy.org/download/2.4/src/
   Git repository   : https://git.haproxy.org/git/haproxy-2.4.git/
   Git Web browsing : https://git.haproxy.org/?p=haproxy-2.4.git
   Changelog: https://www.haproxy.org/download/2.4/src/CHANGELOG
   Pending bugs : https://www.haproxy.org/l/pending-bugs
   Reviewed bugs: https://www.haproxy.org/l/reviewed-bugs
   Code reports : https://www.haproxy.org/l/code-reports
   Latest builds: https://www.haproxy.org/l/dev-packages


---
Complete changelog :
Aurelien DARRAGON (4):
  BUG/MEDIUM: proxy: ensure pause_proxy() and resume_proxy() own PROXY_LOCK
  MINOR: listener: small API change
  BUG/MINOR: stats: fixing stat shows disabled frontend status as 'OPEN'
  BUG/MINOR: log: improper behavior when escaping log data

Brad Smith (1):
  BUILD: makefile: enable crypt(3) for NetBSD

Christopher Faulet (25):
  MINOR: http: Add function to get port part of a host
  MINOR: http: Add function to detect default port
  BUG/MEDIUM: h1: Improve authority validation for CONNCET request
  MINOR: http-htx: Use new HTTP functions for the scheme based normalization
  MINOR: peers: Use a dedicated reconnect timeout when stopping the local 
peer
  BUG/MEDIUM: peers: limit reconnect attempts of the old process on reload
  BUG/MINOR: peers: Use right channel flag to consider the peer as connected
  BUG/MEDIUM: dns: Properly initialize new DNS session
  MINOR: server: Constify source server to copy its settings
  REORG: server: Export srv_settings_cpy() function
  BUG/MEDIUM: proxy: Perform a custom copy for default server settings
  BUG/MINOR: tcpcheck: Disable QUICKACK only if data should be sent after 
connect
  REGTESTS: Fix prometheus script to perform HTTP health-checks
  BUG/MEDIUM: spoe: Properly update streams waiting for a ACK in async mode
  BUG/MEDIUM: peers: Add connect and server timeut to peers proxy
  BUG/MEDI

[ANNOUNCE] haproxy-2.5.9

2022-09-23 Thread Christopher Faulet
me from
the CLI. In itself, it is not an issue but the server is created with no
address and an operation was not guarded against NULL addresses.

  * There was a bug in the SPOE. In sync or pipelining modes, an unhealthy
SPOA could led HAProxy to create a huge number of applets to process
queued messages, slowing down all processing.

  * Willy managed to trigger an error on reload where the old process died
saying "t->tid >= 0 && t->tid != tid". This is caused by the deinit code
that needs to stop stuff initialized on other threads, and as such it
violates some consistency checks. The check was relaxed to ignore the
stopping condition.

  * Characters escaping process in log messages was not correctly processing
strings coming from sample fetches truncating the output string.

  * Using HAProxy built with PCRE2_JIT with a lib built without would fail
to match. Now it will fall back to the regular match.

  * Agent-check could be delayed by ~200ms due to TCP QUICKACK being
disabled by default.

  * Reading from the rings could also occasionally freeze at high rate if
the reader had to stop due to a buffer full while the writer had already
stopped due to a ring full.

  * In Lua, it was possible to hand reading HTTP payload (by line or not)
from an HTTP applet because we relied on a transiant HTX flags to detect
the end of the message instead of relying on the channel flag.

  * Some ca-file elements could leak during "commit ssl ca-file".

  * On the CLI, no error was returned when an empty ca-file was added. This
could be a problem is the file was malformated and did not contain any
PEM header.

  * A 60s delay could be experienced after stopping HAProxy. This was
happening when a signal was received before entering the poller and
without any activity on the process. In mworker mode, if a worker exited
and the SIGCHLD signal was delivered at the right time to the master,
this one could be stuck for 60s. The timeout is now set to 0 in this
specific case.

The following improvements were also backported:

  * Headers case adjustment in H1 is now available for TCP proxies. It was
an issue for HTTP health-checks on backend side or for TCP connections
upgraded to HTTP on frontend side.

  * The stats applet was reported paused frontends as OPEN. Now, these
frontends are reported as PAUSED.

  * Encrypted password in Userlists are now supported on NetBSD

Thanks everyone for your help and your contributions! The 2.4.19 will be
released the next week.

Please find the usual URLs below :
   Site index   : https://www.haproxy.org/
   Documentation: https://docs.haproxy.org/
   Wiki : https://github.com/haproxy/wiki/wiki
   Discourse: https://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Sources  : https://www.haproxy.org/download/2.5/src/
   Git repository   : https://git.haproxy.org/git/haproxy-2.5.git/
   Git Web browsing : https://git.haproxy.org/?p=haproxy-2.5.git
   Changelog: https://www.haproxy.org/download/2.5/src/CHANGELOG
   Pending bugs : https://www.haproxy.org/l/pending-bugs
   Reviewed bugs: https://www.haproxy.org/l/reviewed-bugs
   Code reports : https://www.haproxy.org/l/code-reports
   Latest builds: https://www.haproxy.org/l/dev-packages


---
Complete changelog :
Aurelien DARRAGON (6):
  BUG/MEDIUM: proxy: ensure pause_proxy() and resume_proxy() own PROXY_LOCK
  MINOR: listener: small API change
  MINOR: proxy/listener: support for additional PAUSED state
  BUG/MINOR: stats: fixing stat shows disabled frontend status as 'OPEN'
  BUG/MEDIUM: server: segv when adding server with hostname from CLI
  BUG/MINOR: log: improper behavior when escaping log data

Brad Smith (1):
  BUILD: makefile: enable crypt(3) for NetBSD

Christopher Faulet (20):
  MINOR: peers: Use a dedicated reconnect timeout when stopping the local 
peer
  BUG/MEDIUM: peers: limit reconnect attempts of the old process on reload
  BUG/MINOR: peers: Use right channel flag to consider the peer as connected
  BUG/MEDIUM: dns: Properly initialize new DNS session
  MINOR: server: Constify source server to copy its settings
  REORG: server: Export srv_settings_cpy() function
  BUG/MEDIUM: proxy: Perform a custom copy for default server settings
  BUG/MINOR: tcpcheck: Disable QUICKACK only if data should be sent after 
connect
  REGTESTS: Fix prometheus script to perform HTTP health-checks
  BUG/MEDIUM: spoe: Properly update streams waiting for a ACK in async mode
  BUG/MEDIUM: peers: Add connect and server timeut to peers proxy
  BUG/MEDIUM: peers: Don't use resync timer when local resync is in progress
  BUG/MEDIUM: peers: Don't start resync on reload if local peer is not 
up

Re: [UK OFFICIAL] [PATCH] Issue 1812 - smtpchk option does not send a QUIT

2022-09-22 Thread Christopher Faulet

Le 9/21/22 à 16:13, Wright Loz a écrit :

Classification: UK OFFICIAL


Hi Christopher,

Many thanks - good point regarding EHLO reply and yes, if you could add the 
backports / reg-tests parts as well that would be brilliant. This is my first 
ever patch submission so am keen to learn all the necessary parts for next time!



Hi,

I merged your patch. Thanks !

--
Christopher Faulet




[ANNOUNCE] haproxy-2.6.6

2022-09-22 Thread Christopher Faulet
buf
  BUG/MEDIUM: mux-quic: properly trim HTX buffer on snd_buf reset

Aurelien DARRAGON (8):
  BUG/MEDIUM: proxy: ensure pause_proxy() and resume_proxy() own PROXY_LOCK
  MINOR: listener: small API change
  MINOR: proxy/listener: support for additional PAUSED state
  BUG/MINOR: stats: fixing stat shows disabled frontend status as 'OPEN'
  CLEANUP: listener: function comment typo in stop_listener()
  BUG/MINOR: listener: null pointer dereference suspected by coverity
  BUG/MEDIUM: server: segv when adding server with hostname from CLI
  BUG/MINOR: log: improper behavior when escaping log data

Brad Smith (2):
  MINOR: Revert part of clarifying samples support per os commit
  BUILD: makefile: enable crypt(3) for NetBSD

Christopher Faulet (4):
  BUG/MINOR: h1: Support headers case adjustment for TCP proxies
  BUG/MINOR: task: Fix detection of tasks profiling in 
tasklet_wakeup_after()
  BUG/MINOR: mux-h1: Increment open_streams counter when H1 stream is 
created
  REGTESTS: healthcheckmail: Relax matching on the healthcheck log message

Emeric Brun (1):
  BUG/MEDIUM: sink: bad init sequence on tcp sink from a ring.

Frédéric Lécaille (12):
  BUG/MINOR: quic: Retransmitted frames marked as acknowledged
  BUG/MINOR: quic: Possible crash with "tls-ticket-keys" on QUIC bind lines
  BUG/MINOR: quic: Possible crash when verifying certificates
  MINOR: quic: Add traces about sent or resent TX frames
  MINOR: quic: No TRACE_LEAVE() in retrieve_qc_conn_from_cid()
  BUG/MINOR: quic: Wrong connection ID to thread ID association
  BUG/MINOR: quic: Speed up the handshake completion only one time
  BUG/MINOR: quic: Trace fix about packet number space information.
  BUG/MINOR: h3: Crash when h3 trace verbosity is "minimal"
  MINOR: h3: Add the quic_conn object to h3 traces
  MINOR: h3: Missing connection argument for a TRACE_LEAVE() argument
  MINOR: h3: Send the h3 settings with others streams (requests)

Ilya Shipitsin (4):
  CI: cirrus-ci: bump FreeBSD image to 13-1
  REGTESTS: ssl: adopt tests to OpenSSL-3.0.N
  REGTESTS: ssl: adopt tests to OpenSSL-3.0.N
  REGTESTS: ssl: fix grep invocation to use extended regex in 
ssl_generate_certificate.vtc

Mathias Weiersmueller (1):
  DOC: fix TOC in starter guide for subsection 3.3.8. Statistics

Matthias Wirth (1):
  BUG/MINOR: signals/poller: ensure wakeup from signals

William Lallemand (11):
  BUILD: quic: add some ifdef around the SSL_ERROR_* for libressl
  BUILD: ssl: fix ssl_sock_switchtx_cbk when no client_hello_cb
  BUILD: quic: temporarly ignore chacha20_poly1305 for libressl
  BUILD: quic: enable early data only with >= openssl 1.1.1
  BUILD: ssl: fix the ifdef mess in ssl_sock_initial_ctx
  BUILD: quic: fix the #ifdef in ssl_quic_initial_ctx()
  MINOR: quic: add QUIC support when no client_hello_cb
  BUG/MINOR: signals/poller: set the poller timeout to 0 when there are 
signals
  REGTESTS: log: test the log-forward feature
  REGTESTS: ssl/log: test the log-forward with SSL
  MEDIUM: quic: separate path for rx and tx with set_encryption_secrets

Willy Tarreau (14):
  MEDIUM: peers: limit the number of updates sent at once
  BUG/MINOR: task: always reset a new tasklet's call date
  BUG/MINOR: task: make task_instant_wakeup() work on a task not a tasklet
  MINOR: task: permanently enable latency measurement on tasklets
  CLEANUP: task: rename ->call_date to ->wake_date
  BUG/MINOR: sched: properly account for the CPU time of dying tasks
  MINOR: sched: store the current profile entry in the thread context
  BUG/MINOR: stream/sched: take into account CPU profiling for the last call
  DEV: flags: fix usage message to reflect available options
  DEV: flags: add missing CO_FL_FDLESS connection flag
  CLEANUP: pollers: remove dead code in the polling loop
  BUG/MEDIUM: captures: free() an error capture out of the proxy lock
  BUILD: fd: fix a build warning on the DWCAS
  SCRIPTS: announce-release: update some URLs to https

cui fliter (1):
  CLEANUP: quic,ssl: fix tiny typos in C comments

--
Christopher Faulet



Re: [UK OFFICIAL] [PATCH] Issue 1812 - smtpchk option does not send a QUIT

2022-09-21 Thread Christopher Faulet

Le 9/21/22 à 11:42, Wright Loz a écrit :

Classification: UK OFFICIAL


Hi,

Reference Github issue 1812 ( https://github.com/haproxy/haproxy/issues/1812 
<https://github.com/haproxy/haproxy/issues/1812> ) relating to unclean shutdown 
of “smtpchk” health checking, I attach a patch which implements an SMTP QUIT 
(and then a wait for successful confirmation) prior to closing the TCP 
connection on each SMTP service check.


Hopefully relatively straightforward and obviously happy to discuss!



It seems to be ok at first glance.

However, I must fix the regex to properly match the EHLO reply first. It is a 
multi-line reply and, for now, the regex is only matching on the first line. I 
will fix it before merging your patch.


In addition and if it is ok for you, I will amend you patch to add information 
about backports and to update reg-tests accordingly 
(reg-tests/checks/4be_1srv_smtpchk_httpchk_layer47errors.vtc and 
reg-tests/checks/smtp-check.vtc)


Thanks !
--
Christopher Faulet




Re: Health Checks and DNS lookups in stopping processes

2022-09-20 Thread Christopher Faulet

Le 9/19/22 à 21:04, Tim Düsterhus a écrit :

Christopher,

On 9/19/22 20:22, Christopher Faulet wrote:

It is not exactly how it works. When HAProxy is reloaded, it stops to accept new
connections and it closes idle HTTP connections on the server side and also on
the client side. However, on client sides, a connection is idle if at least one
request was processed. So totally inactive clients are not closed. These clients
may still perform a request. Connection to servers are not blocked on old 
workers.

It is important when "idle-close-on-response" option is enabled. In this case,
idle client connections are not closed and may try to perform a last request.
This means all backend mechanisms must still be running (load-balancing,
redispatch, l7-retry, health-checks, dns resolution ...).




In other words: If I do not use 'idle-close-on-response' or if all
existing connections have been used at least once, the health/DNS logic
could technically be disabled?

In that case I would file a feature request for that, because
'hard-stop-after' and 'mworker-max-reloads' both are "not great" for our
use case. I want to keep the connections alive for as long at possible,
but without adding unnecessary load on our backends or resolvers.




I would say yes, it is possible. But I don't know if it is easy to achieve.


--
Christopher




Re: Health Checks and DNS lookups in stopping processes

2022-09-19 Thread Christopher Faulet

Le 9/19/22 à 17:44, Tim Düsterhus a écrit :

Hi

recently our HAProxy nodes started handling long-running HTTP
connections (similar to WebSockets). This causes old workers to stay
around for several days after a reload.

This isn't too bad from a memory perspective, we have sufficient RAM to
keep around the old processes until the connections die naturally. It's
much worse from a CPU perspective: The old workers appear to still
perform health checks and DNS lookups and this takes away precious
resources from the active workers.

My understanding is that the old workers will only handle existing
connections and thus will never need to connect to a backend (server)
again. Thus it should not be necessary to waste CPU on DNS lookups and
health checks for a stopping worker.

Am I missing something here?



It is not exactly how it works. When HAProxy is reloaded, it stops to accept new 
connections and it closes idle HTTP connections on the server side and also on 
the client side. However, on client sides, a connection is idle if at least one 
request was processed. So totally inactive clients are not closed. These clients 
may still perform a request. Connection to servers are not blocked on old workers.


It is important when "idle-close-on-response" option is enabled. In this case, 
idle client connections are not closed and may try to perform a last request. 
This means all backend mechanisms must still be running (load-balancing, 
redispatch, l7-retry, health-checks, dns resolution ...).



--
Christopher Faulet



Re: Lua hangs with get line at end of request body

2022-08-29 Thread Christopher Faulet

Le 8/29/22 à 12:30, Christopher Faulet a écrit :


There is a bug. I'm able to reproduce it with your lua script. I will fix it 
soon.



FYI, I pushed a fix[1] to 2.7-DEV. It will be backported soon.


[1] https://github.com/haproxy/haproxy/commit/4a20972a

--
Christopher Faulet



Re: Lua hangs with get line at end of request body

2022-08-29 Thread Christopher Faulet

Le 8/26/22 à 18:57, Robert Newson a écrit :

Hi All,

I'm upgrading some HAproxy nodes here (from 2.0.29 to 2.6.4) and everything is 
going well except some Lua code I have. I've distilled the code and config to 
this;

haproxy.cfg;

global
lua-load httpbug.lua

defaults
mode http
timeout client 15
timeout connect 5000
timeout queue 5000
timeout server 36

listen proxy
bind 127.0.0.1:10001
http-request use-service lua.httpbug

httpbug.lua:

core.register_service("httpbug", "http", function(applet)
repeat
local line = applet:getline()
core.log(core.crit, line)
until line == ''
applet:set_status(200)
applet:start_response()
applet:send("success")
end)

--

With a request like 'curl localhost: 10001 -XPOST -T body' and file called 
'body' as follows;

line1
line2
line3

(where each line is newline-terminated)

With 2.0.29 I get a response (of 200 with body "success"), and line1, line2, 
line3 are logged to screen (haproxy -d -f haproxy.cfg)

With 2.6.4 I see the logs of line1, line2, line3, but no response.

It appears applet:getline() blocks forever at the end of the request body, 
rather than returning an empty string to indicate this like it used to (and is 
documented to do).

I searched the mailing list (and SO, etc) for other examples of reading the 
http request body line by line or for a report of this bug from someone else 
but found nothing.

What did I do wrong?



Hi,

There is a bug. I'm able to reproduce it with your lua script. I will fix it 
soon.

Thanks !

--
Christopher Faulet



Re: Lua: processing expired on timeout when using core.msleep

2022-08-25 Thread Christopher Faulet

Le 8/23/22 à 20:08, Bren a écrit :

--- Original Message ---
On Tuesday, August 23rd, 2022 at 4:26 AM, Christopher Faulet 
 wrote:


It could be good to share your config, at least the part calling your lua 
script.


I think these are the only relevant bits:

tcp-request inspect-delay 10s
http-request lua.delay_request 15000 3

I'm delaying requests a random number of ms between 15000 and 3.


But this error can be triggered when the inspect-delay for tcp rules
evaluation is expires.


Perhaps this is what is happening?



You have an inspect delay for tcp-request rules but your script is called during 
http-request rules eval. So it is not related. Here I guess you have enable 
abortonclose option and a client abort is caught. In this case, the script 
execution is interrupted. Or a read error occurred on the client side.


However, I must admit the warning is not accurate in this case...

--
Christopher Faulet



Re: Lua: processing expired on timeout when using core.msleep

2022-08-23 Thread Christopher Faulet

Le 8/22/22 à 18:53, Bren a écrit :

Greetings,

This is a minor issue I've been meaning to ask about. I have a fairly simple 
Lua script that simply runs core.msleep on certain requests for a random number 
of ms to slow them down. I've noticed this in the logs:

[ALERT]    (3650884) : Lua function 'delay_request': aborting Lua processing on 
expired timeout.

I've always been under the impression that a sleep shouldn't cause any 
timeouts. Both tune.lua.session-timeout and tune.lua.service-timeout says:

If the Lua does a sleep, the sleep is not taken in account.

Am I missing something?

Bren



Hi,

It could be good to share your config, at least the part calling your lua 
script. But this error can be triggered when the inspect-delay for tcp rules 
evaluation is expires. It can also happen when a read error is detected or an 
abort with abortonclose option.


--
Christopher Faulet



Re: Possible problem with custom error pages -- backend server returns 503, haproxy logs 503, but the browser gets 403

2022-08-23 Thread Christopher Faulet

Le 8/22/22 à 16:37, Shawn Heisey a écrit :



The same problem also happens with 2.6.4, built with the same options as
the dev version.

HAProxy version 2.6.4 2022/08/22 - https://haproxy.org/

I have documentation for the problem details in another project's bug
tracker:

https://issues.apache.org/jira/browse/SOLR-16327?focusedCommentId=17582990=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17582990

It appears so far as if haproxy is getting a 503 from the backend,
logging a 503, but actually sending a 403.  Here is the config snippet
when it works correctly:

A top-level config section:
http-errors myerrors
      errorfile 404 /etc/haproxy/errors/404.http
      errorfile 403 /etc/haproxy/errors/403.http
      errorfile 500 /etc/haproxy/errors/500.http
      errorfile 502 /etc/haproxy/errors/50x.http
      errorfile 503 /etc/haproxy/errors/50x.http
      errorfile 504 /etc/haproxy/errors/50x.http


In the frontend:
      errorfiles myerrors
      http-response return status 404 default-errorfiles if
!real_errors { status 404 }
      http-response return status 403 default-errorfiles if
!real_errors { status 403 }
      http-response return status 500 default-errorfiles if
!real_errors { status 500 }
      http-response return status 502 default-errorfiles if
!real_errors { status 502 }
      http-response return status 503 default-errorfiles if
!real_errors { status 503 }
      http-response return status 504 default-errorfiles if
!real_errors { status 504 }

Removing the "!real_errors" part and restarting haproxy is when the
problem occurs.  I created and used the real_errors acl as a working
bandaid for the issue -- turn off the custom error pages for the solr
hostname.



Hi,

It could be good to share your configuration and not only a snippet. However I'm 
puzzled because in your case, the status code must be the one returned by the 
server if no return rule matches or the one specified by the applied return rule.


There is also something I don't understand. In your bug report on solr project, 
HAProxy logs report HTTP/3.0 requests but the screenshots show HTTP/2.0 
requests. And the payload for the 403 response is talking about 50x errors. What 
is the 50x.http error file content ?


--
Christopher Faulet



Re: Server timeouts since HAProxy 2.2

2022-08-04 Thread Christopher Faulet

Le 8/3/22 à 16:23, William Edwards a écrit :

Hi,

Two days ago, I upgraded my first production system from HAProxy 1.8.19
to 2.2.9. Since then, many HTTP requests are hitting the server timeout.

Before upgrade:

  root@lb0-0:~# zgrep 'sD--' /var/log/haproxy.log.5.gz | wc -l
  0
  root@lb0-0:~# zgrep 'sD--' /var/log/haproxy.log.4.gz | wc -l
  0
  root@lb0-0:~# zgrep 'sD--' /var/log/haproxy.log.3.gz | wc -l
  0

After upgrade:

  # Day of upgrade
  root@lb0-0:~# zgrep 'sD--' /var/log/haproxy.log.2.gz | wc -l
  3798
  # Yesterday
  root@lb0-0:~# grep 'sD--' /var/log/haproxy.log.1 | wc -l
  127176
  # Today, so far
  root@lb0-0:~# grep 'sD--' /var/log/haproxy.log | wc -l
  85063

For this specific request, Ta ("total active time for the HTTP request")
is 3, and Tt ("total TCP session duration time, between the moment the
proxy accepted it and the moment both ends were closed") is 34 (5
minutes, the server timeout):

  Aug  3 00:31:05 lb0-0 haproxy[16884]: $ip:62223
[03/Aug/2022:00:26:05.337] fr_other~
bk_http.lyr_http-lyr02.cf.ha.cyberfusion.cloud/http-lyr02.cf.ha.cyberfusion.cloud
0/0/0/3/34 200 27992 - - sD-- 616/602/226/226/0 0/0 "GET
https://$domain/wp-content/uploads/2022/07/20220712_155022-300x300.jpg
HTTP/2.0"

The backend server indeed served the request within Ta:

  $domain $ip - - [03/Aug/2022:00:26:05 +0200] "GET
/wp-content/uploads/2022/07/20220712_155022-300x300.jpg HTTP/1.1" 200
28008 "https://$domain/stoffen/; "Mozilla/5.0 (Windows NT 10.0; Win64;
x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0
Safari/537.36"

The timeouts only occur with 5 out of 13 backends. There is no clear
pattern, i.e. the timeouts don't come in bursts, and they aren't caused
by fixed clients.

Does anyone know why the TCP session is kept open, and the HTTP request
is not responded to by HAProxy after the backend server responded to the
HTTP request, but only after the server timeout is reached?



Hi,

The 2.2.9 is pretty old. It is affected by 369 known bugs 
(http://www.haproxy.org/bugs/bugs-2.2.9.html). You must update it to 2.2.25 first.


Regards,
--
Christopher Faulet



Re: "Success" logs in HTTP frontends

2022-07-29 Thread Christopher Faulet

Le 7/29/22 à 11:21, Tim Düsterhus a écrit :

Hi

On 7/29/22 11:10, Christopher Faulet wrote:

What is your version ? At first glance, I can't find such log message in the
code. It could come from a lua module.


I'm seeing the same for both 2.4.x and 2.6.x. Christian and I had a
short chat about this in IRC.


In fact, I found something. It is probably because an "embryonic" session is
killed with no connection/ssl error. For instance, an SSL connection rejected
because of a "tcp-request session" rule (so after the SSL handshake). The same
may happen with a listener using the PROXY protocol.


On one of the machines I'm seeing it, we neither have 'tcp-' rules, nor
do we use PROXY protocol:



Well, it may have several reason to kill an embryonic session with no error. a 
reject at the session level is one of them. Probably the most common. It may 
also be an error when we try to install the client mux. The configuration may 
help in this case. I don't know if it is possible to have an handshake failure 
without setting any error. However, at first glance, the error is always set in 
this case.


Of course, it may be a bug. If not, such messages can be removed by setting 
"dontlognull" option.


--
Christopher Faulet



Re: "Success" logs in HTTP frontends

2022-07-29 Thread Christopher Faulet

Le 7/29/22 à 10:13, Christian Ruppert a écrit :

Hi list,

so I noticed on my private HAProxy I have 2 of those logs within the
past ~1-2 months:
haproxy[28669]: 1.2.3.4:48596 [17/Jun/2022:13:55:18.530] public/HTTPSv4:
Success

So that's nothing so far but still no idea what that means.
At work, of 250 mio log entries per day, there are about 600k of those
"Success" ones.
haproxy[27892]: 192.168.70.102:7904 [29/May/2022:00:13:37.316]
genfrontend_35310-foobar/3: Success

I'm not sure what it means by "3". Is it the third bind?

I couldn't trigger those "Success" logs by either restarting or
reloading. What is it for / where does it come from?



Hi Christian,

What is your version ? At first glance, I can't find such log message in the 
code. It could come from a lua module.


In fact, I found something. It is probably because an "embryonic" session is 
killed with no connection/ssl error. For instance, an SSL connection rejected 
because of a "tcp-request session" rule (so after the SSL handshake). The same 
may happen with a listener using the PROXY protocol.


Regards,
--
Christopher Faulet



[ANNOUNCE] haproxy-2.2.25

2022-07-28 Thread Christopher Faulet
www.haproxy.org/download/2.2/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.2.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.2.git
   Changelog: http://www.haproxy.org/download/2.2/src/CHANGELOG
   Pending bugs : http://www.haproxy.org/l/pending-bugs
   Reviewed bugs: http://www.haproxy.org/l/reviewed-bugs
   Code reports : http://www.haproxy.org/l/code-reports
   Latest builds: http://www.haproxy.org/l/dev-packages


---
Complete changelog :
Christopher Faulet (29):
  MEDIUM: http-ana: Add a proxy option to restrict chars in request header 
names
  REGTESTS: abortonclose: Fix some race conditions
  BUG/MEDIUM: config: Reset outline buffer size on realloc error in 
readcfgfile()
  BUG/MINOR: check: Reinit the buffer wait list at the end of a check
  BUG/MEDIUM: resolvers: Don't defer resolutions release in deinit function
  BUG/MEDIUM: dns: Keep the right count of active nameservers for a resolver
  BUG/MINOR: ssl_ckch: Free error msg if commit changes on a cert entry 
fails
  BUG/MEDIUM: ssl_ckch: Don't delete a cert entry if it is being modified
  BUG/MINOR: ssl_ckch: Don't duplicate path when replacing a cert entry
  BUG/MEDIUM: ssl_ckch: Rework 'commit ssl cert' to handle full buffer cases
  BUG/MEDIUM: ssl/crt-list: Rework 'add ssl crt-list' to handle full buffer 
cases
  MEDIUM: http-ana: Always report rewrite failures as PRXCOND in logs
  REGTESTS: abortonclose: Add a barrier to not mix up log messages
  REGTESTS: http_request_buffer: Increase client timeout to wait "slow" 
clients
  BUG/MINOR: ssl_ckch: Dump cert transaction only once if show command yield
  BUG/MINOR: ssl_ckch: Fix possible uninitialized value in show_cert I/O 
handler
  REGTESTS: restrict_req_hdr_names: Extend supported versions
  BUG/MEDIUM: mailers: Set the object type for check attached to an email 
alert
  REGTESTS: healthcheckmail: Update the test to be functionnal again
  REGTESTS: healthcheckmail: Relax health-check failure condition
  BUG/MINOR: tcp-rules: Make action call final on read error and delay 
expiration
  BUG/MINOR: http-ana: Set method to HTTP_METH_OTHER when an HTTP txn is 
created
  BUG/MINOR: http-fetch: Use integer value when possible in "method" sample 
fetch
  BUG/MINOR: http-check: Preserve headers if not redefined by an implicit 
rule
  BUG/MINOR: http-act: Properly generate 103 responses when several rules 
are used
  BUG/MEDIUM: http-ana: Don't wait to have an empty buf to switch in TUNNEL 
state
  BUG/MEDIUM: mux-h1: Handle connection error after a synchronous send
  REGTESTS: Fix some scripts to be compatible with 2.4 and prior
  BUG/MINOR: backend: Fallback on RR algo if balance on source is impossible

David Carlier (1):
  BUILD: fix build warning on solaris based systems with __maybe_unused.

Emeric Brun (4):
  BUG/MEDIUM: peers: fix segfault using multiple bind on peers sections
  BUG/MEDIUM: peers: prevent unitialized multiple listeners on peers section
  DOC: peers: clarify when entry expiration date is renewed.
  DOC: peers: fix port number and addresses on new peers section format

Ilya Shipitsin (3):
  CI: determine actual LibreSSL version dynamically
  CI: determine actual OpenSSL version dynamically
  CI: re-enable gcc asan builds

Remi Tricot-Le Breton (2):
  BUG/MINOR: ssl: Fix crash when no private key is found in pem
  BUG/MINOR: ssl: Do not look for key in extra files if already in pem

Thayne McCombs (1):
  BUG/MEDIUM: sample: Fix adjusting size in word converter

Tim Duesterhus (2):
  BUG/MEDIUM: http: Properly reject non-HTTP/1.x protocols
  REGTESTS: Do not use REQUIRE_VERSION for HAProxy 2.5+ (2)

William Lallemand (3):
  BUG/MEDIUM: ssl/cli: crash when crt inserted into a crt-list
  BUG/MINOR: peers: fix possible NULL dereferences at config parsing
  BUG/MINOR: sockpair: wrong return value for fd_send_uxst()

Willy Tarreau (12):
  BUG/MINOR: cfgparse: abort earlier in case of allocation error
  BUG/MINOR: peers: fix error reporting of "bind" lines
  SCRIPTS: add make-releases-json to recreate a releases.json file in 
download dirs
  SCRIPTS: make publish-release try to launch make-releases-json
  DOC: peers: indicate that some server settings are not usable
  BUG/MINOR: conn_stream: do not confirm a connection from the frontend path
  BUILD: compiler: implement unreachable for older compilers too
  BUG/MINOR: cli/stats: add missing trailing LF after JSON outputs
  BUG/MINOR: server: do not enable DNS resolution on disabled proxies
  BUG/MINOR: cli/stats: add missing trailing LF after "show info json"
  MEDIUM: mux-h2: try to coalesce outgoing WINDOW_UPDATE frames
  BUG/MINOR: peers/config: always fill the bind_conf's argument

--
Christopher Faulet



[ANNOUNCE] haproxy-2.4.18

2022-07-27 Thread Christopher Faulet
us list of headers is replaced by a new one only if it is not
   empty.

 * It was possible to crash HAProxy by defining multiple bind lines in a
   peers section. An error is now reported during configuration parsing.

 * A warning is now reported when some unsupported keywords are used in
   peers section instead of silently ignoring them. init_addr, resolvers,
   check, agent-check are concerned.

 * The DNS resolution is now ignored for disabled proxies preventing some
   crashes.

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Documentation: http://docs.haproxy.org/
   Wiki : https://github.com/haproxy/wiki/wiki
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Sources  : http://www.haproxy.org/download/2.4/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.4.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.4.git
   Changelog: http://www.haproxy.org/download/2.4/src/CHANGELOG
   Pending bugs : http://www.haproxy.org/l/pending-bugs
   Reviewed bugs: http://www.haproxy.org/l/reviewed-bugs
   Code reports : http://www.haproxy.org/l/code-reports
   Latest builds: http://www.haproxy.org/l/dev-packages


---
Complete changelog :
Brad Smith (1):
  BUILD: makefile: Fix install(1) handling for OpenBSD/NetBSD/Solaris/AIX

Christian Ruppert (1):
  BUILD: Makefile: Add Lua 5.4 autodetect

Christopher Faulet (35):
  MEDIUM: http-ana: Add a proxy option to restrict chars in request header 
names
  REGTESTS: abortonclose: Fix some race conditions
  BUG/MEDIUM: config: Reset outline buffer size on realloc error in 
readcfgfile()
  BUG/MINOR: check: Reinit the buffer wait list at the end of a check
  BUG/MEDIUM: resolvers: Don't defer resolutions release in deinit function
  BUG/MINOR: ssl_ckch: Free error msg if commit changes on a cert entry 
fails
  BUG/MEDIUM: ssl_ckch: Don't delete a cert entry if it is being modified
  BUG/MINOR: ssl_ckch: Don't duplicate path when replacing a cert entry
  BUG/MEDIUM: ssl_ckch: Rework 'commit ssl cert' to handle full buffer cases
  BUG/MEDIUM: ssl/crt-list: Rework 'add ssl crt-list' to handle full buffer 
cases
  MEDIUM: http-ana: Always report rewrite failures as PRXCOND in logs
  REGTESTS: abortonclose: Add a barrier to not mix up log messages
  REGTESTS: http_request_buffer: Increase client timeout to wait "slow" 
clients
  BUG/MINOR: ssl_ckch: Dump cert transaction only once if show command yield
  BUG/MINOR: ssl_ckch: Fix possible uninitialized value in show_cert I/O 
handler
  REGTESTS: http_abortonclose: Extend supported versions
  REGTESTS: restrict_req_hdr_names: Extend supported versions
  BUG/MEDIUM: mailers: Set the object type for check attached to an email 
alert
  BUG/MINOR: trace: Test server existence for health-checks to get proxy
  BUG/MINOR: checks: Properly handle email alerts in trace messages
  REGTESTS: healthcheckmail: Update the test to be functionnal again
  REGTESTS: healthcheckmail: Relax health-check failure condition
  BUG/MINOR: tcp-rules: Make action call final on read error and delay 
expiration
  BUG/MINOR: http-ana: Set method to HTTP_METH_OTHER when an HTTP txn is 
created
  BUG/MINOR: http-fetch: Use integer value when possible in "method" sample 
fetch
  BUG/MINOR: http-check: Preserve headers if not redefined by an implicit 
rule
  BUG/MINOR: http-act: Properly generate 103 responses when several rules 
are used
  BUG/MINOR: http-htx: Fix scheme based normalization for URIs wih userinfo
  BUG/MEDIUM: http-fetch: Don't fetch the method if there is no stream
  REGTEESTS: filters: Fix CONNECT request in random-forwarding script
  BUG/MINOR: mux-h1: Be sure to commit htx changes in the demux buffer
  BUG/MEDIUM: http-ana: Don't wait to have an empty buf to switch in TUNNEL 
state
  BUG/MEDIUM: mux-h1: Handle connection error after a synchronous send
  REGTESTS: Fix some scripts to be compatible with 2.4 and prior
  BUG/MINOR: backend: Fallback on RR algo if balance on source is impossible

David CARLIER (1):
  BUILD/MINOR: cpuset fix build for FreeBSD 13.1

David Carlier (2):
  BUILD: fix build warning on solaris based systems with __maybe_unused.
  MINOR: tools: add get_exec_path implementation for solaris based systems.

Emeric Brun (7):
  BUG/MEDIUM: peers: fix segfault using multiple bind on peers sections
  BUG/MEDIUM: peers: prevent unitialized multiple listeners on peers section
  DOC: peers: clarify when entry expiration date is renewed.
  DOC: peers: fix port number and addresses on new peers section format
  MINOR: fd: add a new FD_DISOWN flag to prevent from c

[ANNOUNCE] haproxy-2.5.8

2022-07-25 Thread Christopher Faulet
 the it is removed from
   the fdtab array.

 * Crashes could be experienced during hot-upgrade from 2.4 to 2.6 because
   old worker was still identified as a running worker.

 * An internal error was reported when loadbalancing on source IP address
   was impossible. It could happens with SPOE applets or with clients
   connected to HAPRoxy via a unix socket. Now, when this happens, a
   fallback to round-robin is performed.

 * The HTTP scheme based normalization did not properly handle the URIs with
   userinfo. They were not preserved after the normalization process.

 * Duplicate certificates in ca-file directories were not properly handled
   because of an OpenSSL error. The error is now ignored.

 * Lookup for a private key in extra files was not ignored when it was
   already found in the pem file, while it should.

 * HAProxy could crash on old Glibc on dlsym() function call if it is
   statically built. Now, we avoid to call it in static builds.

 * Depending on the declaration order of "http-check send" and "option
   httpchk" directives, the configured headers could be ignored. Now a
   previous list of headers is replaced by a new one only if it is not
   empty.

 * Mailers healthchecks were causing a crash since the refactoring of the
   internal HAProxy connection stack introduced in 2.6. This is now fixed.

 * It was possible to crash HAProxy by defining multiple bind lines in a
   peers section. An error is now reported during configuration parsing.

 * A warning is now reported when some unsupported keywords are used in
   peers section instead of silently ignoring them. init_addr, resolvers,
   check, agent-check are concerned.

 * The DNS resolution is now ignored for disabled proxies preventing some
   crashes.

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Documentation: http://docs.haproxy.org/
   Wiki : https://github.com/haproxy/wiki/wiki
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Sources  : http://www.haproxy.org/download/2.5/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.5.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.5.git
   Changelog: http://www.haproxy.org/download/2.5/src/CHANGELOG
   Pending bugs : http://www.haproxy.org/l/pending-bugs
   Reviewed bugs: http://www.haproxy.org/l/reviewed-bugs
   Code reports : http://www.haproxy.org/l/code-reports
   Latest builds: http://www.haproxy.org/l/dev-packages


---
Complete changelog :
Brad Smith (1):
  BUILD: makefile: Fix install(1) handling for OpenBSD/NetBSD/Solaris/AIX

Christian Ruppert (1):
  BUILD: Makefile: Add Lua 5.4 autodetect

Christopher Faulet (51):
  MEDIUM: http-ana: Add a proxy option to restrict chars in request header 
names
  REGTESTS: abortonclose: Fix some race conditions
  BUG/MEDIUM: config: Reset outline buffer size on realloc error in 
readcfgfile()
  BUG/MINOR: check: Reinit the buffer wait list at the end of a check
  BUG/MEDIUM: resolvers: Don't defer resolutions release in deinit function
  BUG/MINOR: ssl_ckch: Free error msg if commit changes on a cert entry 
fails
  BUG/MINOR: ssl_ckch: Free error msg if commit changes on a CA/CRL entry 
fails
  BUG/MEDIUM: ssl_ckch: Don't delete a cert entry if it is being modified
  BUG/MEDIUM: ssl_ckch: Don't delete CA/CRL entry if it is being modified
  BUG/MINOR: ssl_ckch: Don't duplicate path when replacing a cert entry
  BUG/MINOR: ssl_ckch: Don't duplicate path when replacing a CA/CRL entry
  BUG/MEDIUM: ssl_ckch: Rework 'commit ssl cert' to handle full buffer cases
  BUG/MEDIUM: ssl_ckch: Rework 'commit ssl ca-file' to handle full buffer 
cases
  BUG/MEDIUM: ssl/crt-list: Rework 'add ssl crt-list' to handle full buffer 
cases
  BUG/MEDIUM: httpclient: Don't remove HTX header blocks before duplicating 
them
  BUG/MEDIUM: httpclient: Rework CLI I/O handler to handle full buffer cases
  MEDIUM: http-ana: Always report rewrite failures as PRXCOND in logs
  MEDIUM: httpclient: Don't close CLI applet at the end of a response
  REGTESTS: abortonclose: Add a barrier to not mix up log messages
  REGTESTS: http_request_buffer: Increase client timeout to wait "slow" 
clients
  BUG/MINOR: ssl_ckch: Dump CRL transaction only once if show command yield
  BUG/MINOR: ssl_ckch: Dump CA transaction only once if show command yield
  BUG/MINOR: ssl_ckch: Dump cert transaction only once if show command yield
  BUG/MINOR: ssl_ckch: Init right field when parsing "commit ssl crl-file" 
cmd
  BUG/MINOR: ssl_ckch: Fix possible uninitialized value in show_cert I/O 
handler
  BUG/MINOR: ssl_ckch: Fix possible uninitialized value in show_ca

[ANNOUNCE] haproxy-2.6.2

2022-07-22 Thread Christopher Faulet
 MINOR: h3: handle errors on HEADERS parsing/QPACK decoding
  MINOR: qpack: properly handle invalid dynamic table references
  BUG/MEDIUM: mux-quic: fix server chunked encoding response
  BUG/MINOR: quic: fix closing state on NO_ERROR code sent
  BUG/MINOR: quic: do not send CONNECTION_CLOSE_APP in initial/handshake

Benoit DOLEZ (1):
  BUILD: quic: fix anonymous union for gcc-4.4

Brad Smith (1):
  BUILD: makefile: Fix install(1) handling for OpenBSD/NetBSD/Solaris/AIX

Christian Ruppert (1):
  BUILD: Makefile: Add Lua 5.4 autodetect

Christopher Faulet (16):
  BUG/MINOR: http-ana: Set method to HTTP_METH_OTHER when an HTTP txn is 
created
  BUG/MINOR: http-fetch: Use integer value when possible in "method" sample 
fetch
  BUG/MINOR: http-check: Preserve headers if not redefined by an implicit 
rule
  BUG/MINOR: http-act: Properly generate 103 responses when several rules 
are used
  BUG/MINOR: http-htx: Fix scheme based normalization for URIs wih userinfo
  MINOR: http: Add function to get port part of a host
  MINOR: http: Add function to detect default port
  BUG/MEDIUM: h1: Improve authority validation for CONNCET request
  MINOR: http-htx: Use new HTTP functions for the scheme based normalization
  BUG/MEDIUM: http-fetch: Don't fetch the method if there is no stream
  REGTEESTS: filters: Fix CONNECT request in random-forwarding script
  BUG/MINOR: mux-h1: Be sure to commit htx changes in the demux buffer
  BUG/MEDIUM: http-ana: Don't wait to have an empty buf to switch in TUNNEL 
state
  BUG/MEDIUM: mux-h1: Handle connection error after a synchronous send
  BUG/MEDIUM: stconn: Only reset connect expiration when processing backend 
side
  BUG/MINOR: backend: Fallback on RR algo if balance on source is impossible

Emeric Brun (3):
  MINOR: fd: add a new FD_DISOWN flag to prevent from closing a deleted FD
  BUG/MEDIUM: ssl/fd: unexpected fd close using async engine
  MINOR: fd: Add BUG_ON checks on fd_insert()

Frédéric Lécaille (13):
  BUG/MINOR: quic: Missing acknowledgments for trailing packets
  BUG/MINOR: quic: Wrong reuse of fulfilled dgram RX buffer
  BUG/MAJOR: quic: Big RX dgrams leak when fulfilling a buffer
  BUG/MAJOR: quic: Big RX dgrams leak with POST requests
  BUILD: quic+h3: 32-bit compilation errors fixes
  BUG/MINOR: quic: Dropped packets not counted (with RX buffers full)
  MINOR: quic: Add new stats counter to diagnose RX buffer overrun
  MINOR: quic: Duplicated QUIC_RX_BUFSZ definition
  MINOR: task: Add tasklet_wakeup_after()
  MINOR: quic: Improvements for the datagrams receipt
  MINOR: quic: Increase the QUIC connections RX buffer size (upto 64Kb)
  CLEANUP: h2: Typo fix in h2_unsubcribe() traces
  BUG/MAJOR: mux_quic: fix invalid PROTOCOL_VIOLATION on POST data overlap

Ilya Shipitsin (1):
  CI: re-enable gcc asan builds

Remi Tricot-Le Breton (1):
  BUG/MINOR: ssl: Do not look for key in extra files if already in pem

William Lallemand (7):
  BUG/MINOR: peers: fix possible NULL dereferences at config parsing
  MEDIUM: mworker: set the iocb of the socketpair without using fd_insert()
  MINOR: resolvers: resolvers_destroy() deinit and free a resolver
  BUG/MINOR: resolvers: shut off the warning for the default resolvers
  BUG/MINOR: ssl: allow duplicate certificates in ca-file directories
  BUG/MINOR: mworker/cli: relative pid prefix not validated anymore
  BUG/MEDIUM: mworker: proc_self incorrectly set crashes upon reload

Willy Tarreau (8):
  MEDIUM: mux-h2: try to coalesce outgoing WINDOW_UPDATE frames
  BUG/MINOR: peers/config: always fill the bind_conf's argument
  BUG/MEDIUM: cli/threads: make "show threads" more robust on applets
  BUG/MINOR: debug: enter ha_panic() only once
  BUG/MEDIUM: tools: avoid calling dlsym() in static builds
  BUG/MEDIUM: tools: avoid calling dlsym() in static builds (try 2)
  BUG/MINOR: tools: fix statistical_prng_range()'s output range
  BUILD: add detection for unsupported compiler models

--
Christopher Faulet



Re: Issue - consecutive early-hint defined with "if" condition

2022-07-06 Thread Christopher Faulet

Le 7/4/22 à 11:19, Christopher Faulet a écrit :

Le 6/30/22 à 11:47, Krzysztof Kozłowski a écrit :

Hi,

I’m struggling with consecutive early-hint statements defined with if condition.

Considering below configuration:

http-request early-hint Link "<https://example.com/style1.css
<https://example.com/style1.css>>; rel="preload"; as="style"" if { path -i -m
beg /test1 }
http-request early-hint Link "<https://example.com/style2.css
<https://example.com/style2.css>>; rel="preload"; as="style"" if { path -i -m
beg /test2 }

I’m not getting HTTP 103 response to 2nd statement.
Response I’m getting with above config:

example.com/test1 <http://example.com/test1> response: HTTP/2 103 + HTTP/2 200
example.com/test2 <http://example.com/test2> response: HTTP/2 200

However if I add some dummy statement in between those early-hints like this:

http-request early-hint Link "<https://example.com/style1.css
<https://example.com/style1.css>>; rel="preload"; as="style"" if { path -i -m
beg /test1 }
http-request set-header X-Test "test-request"
http-request early-hint Link "<https://example.com/style2.css
<https://example.com/style2.css>>; rel="preload"; as="style"" if { path -i -m
beg /test2 }

I’m receiving HTTP 103 response to both statements as expected.
Response I’m getting with above config:

example.com/test1 <http://example.com/test1> response: HTTP/2 103 + HTTP/2 200
example.com/test2 <http://example.com/test2> response: HTTP/2 103 + HTTP/2 200

It looks like an issue with parsing early-hint statements which are defined with
"if” statement.

It’s my first message sent to this mailing list - so please excuse me if I miss
any information required to properly identify the problem. If I’m missing
anything - please do let me know.
I would appreciate any help here.

I’m using HAProxy version 2.4.15-7782e23



Thanks ! Indeed, I can confirm the issue. I'll try to fix it soon.



Hi,

It should be fixed now in 2.7-DEV but not backported yet. The commit:

  http://git.haproxy.org/?p=haproxy.git;a=commit;h=4c3d3d2

Thanks !
--
Christopher Faulet



Re: Issue - consecutive early-hint defined with "if" condition

2022-07-04 Thread Christopher Faulet

Le 6/30/22 à 11:47, Krzysztof Kozłowski a écrit :

Hi,

I’m struggling with consecutive early-hint statements defined with if condition.

Considering below configuration:

http-request early-hint Link "<https://example.com/style1.css 
<https://example.com/style1.css>>; rel="preload"; as="style"" if { path -i -m 
beg /test1 }
http-request early-hint Link "<https://example.com/style2.css 
<https://example.com/style2.css>>; rel="preload"; as="style"" if { path -i -m 
beg /test2 }


I’m not getting HTTP 103 response to 2nd statement.
Response I’m getting with above config:

example.com/test1 <http://example.com/test1> response: HTTP/2 103 + HTTP/2 200
example.com/test2 <http://example.com/test2> response: HTTP/2 200

However if I add some dummy statement in between those early-hints like this:

http-request early-hint Link "<https://example.com/style1.css 
<https://example.com/style1.css>>; rel="preload"; as="style"" if { path -i -m 
beg /test1 }

http-request set-header X-Test "test-request"
http-request early-hint Link "<https://example.com/style2.css 
<https://example.com/style2.css>>; rel="preload"; as="style"" if { path -i -m 
beg /test2 }


I’m receiving HTTP 103 response to both statements as expected.
Response I’m getting with above config:

example.com/test1 <http://example.com/test1> response: HTTP/2 103 + HTTP/2 200
example.com/test2 <http://example.com/test2> response: HTTP/2 103 + HTTP/2 200

It looks like an issue with parsing early-hint statements which are defined with 
"if” statement.


It’s my first message sent to this mailing list - so please excuse me if I miss 
any information required to properly identify the problem. If I’m missing 
anything - please do let me know.

I would appreciate any help here.

I’m using HAProxy version 2.4.15-7782e23



Thanks ! Indeed, I can confirm the issue. I'll try to fix it soon.

--
Christopher Faulet



Re: Haproxy [2.4.0] cvs stats return " * L7OK,0 " in check_status and check_code fields

2022-06-24 Thread Christopher Faulet

Le 6/23/22 à 22:51, Rashid B. a écrit :

Hi all,

Maybe someone have already encountered the same issue:

Sometimes I get " * L7OK" and "0" in check_status and check_code fields 
respectively in stats in cvs format response.


The full response looks as follows:

'write_read,hostname1,0,0,1,2,500,866,238067,642557,,0,,0,0,0,0,UP,1,1,0,0,0,97441,0,,1,2,2,,866,,2,0,,1,** 
L7OK,0*,3,,,0,0,49,,,0,0,0,112047Layer7 check 
passed,,2,3,4,,tcp0,866,0,,,0,,0,7,0,540337,0,0,0,0,1,1,-,0,0,0,,\n'


My configuration is as follows:
listen write_read
   bind *:5000
   option httpchk
   http-check expect status 200
   default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions
   server hostname1 hostname1:1234 maxconn 500 check port 0910 inter 1s

HAProxy version 2.4.0-6cbbecf 2021/05/14
CentOS Linux release 7.9.2009

Any help is appreciated. Thanks in advance.


Hi,

When the health-check is running, the status reported is the last known status 
and it is prefixed with an asterisks. In this case the code is set to 0 in the 
CSV output. So it is expected. The same is done for the agent-check.


--
Christopher Faulet



Re: Segfault on 2.6.0 with TCP switching to HTTP/2

2022-06-17 Thread Christopher Faulet

Le 6/16/22 à 05:12, David Leadbeater a écrit :

I tried upgrading to 2.6.0 (from 2.5.6) and I'm seeing a segfault when
making HTTP/2 requests. I'm using a frontend in TCP mode and then
switching it to HTTP/2.

I've made a minimal config that exhibits the segfault, below. Simply
doing curl -vk https://ip is enough to trigger it for me.

Thread 1 "haproxy" received signal SIGSEGV, Segmentation fault.
0x555d1d07 in h2s_close (h2s=0x55a60b70) at src/mux_h2.c:1497
1497 HA_ATOMIC_DEC(>h2c->px_counters->open_streams);
(gdb) bt
#0  0x555d1d07 in h2s_close (h2s=0x55a60b70) at src/mux_h2.c:1497
#1  h2s_destroy (h2s=0x55a60b70) at src/mux_h2.c:1515
#2  0x555d3463 in h2_detach (sd=) at src/mux_h2.c:4432

The exact backtrace varies but always in h2s_destroy.

(In case you're wondering what on earth I'm doing, there's a write-up
of it at https://dgl.cx/2022/04/showing-you-your-actual-http-request)



I pushed a fix in 2.7-dev :

  https://github.com/haproxy/haproxy/commit/82af3c68

It will be backported to 2.6 soon. A 2.6.1 should be released the next week.

Thanks !

--
Christopher Faulet



Re: Segfault on 2.6.0 with TCP switching to HTTP/2

2022-06-16 Thread Christopher Faulet

Le 6/16/22 à 05:12, David Leadbeater a écrit :

I tried upgrading to 2.6.0 (from 2.5.6) and I'm seeing a segfault when
making HTTP/2 requests. I'm using a frontend in TCP mode and then
switching it to HTTP/2.

I've made a minimal config that exhibits the segfault, below. Simply
doing curl -vk https://ip is enough to trigger it for me.

Thread 1 "haproxy" received signal SIGSEGV, Segmentation fault.
0x555d1d07 in h2s_close (h2s=0x55a60b70) at src/mux_h2.c:1497
1497 HA_ATOMIC_DEC(>h2c->px_counters->open_streams);
(gdb) bt
#0  0x555d1d07 in h2s_close (h2s=0x55a60b70) at src/mux_h2.c:1497
#1  h2s_destroy (h2s=0x55a60b70) at src/mux_h2.c:1515
#2  0x555d3463 in h2_detach (sd=) at src/mux_h2.c:4432

The exact backtrace varies but always in h2s_destroy.

(In case you're wondering what on earth I'm doing, there's a write-up
of it at https://dgl.cx/2022/04/showing-you-your-actual-http-request)

David

---
global
   ssl-default-bind-options no-sslv3 no-tlsv10
   user nobody

defaults
   timeout connect 10s
   timeout client 30s
   timeout server 2m

frontend tcp-https
   mode tcp
   bind [::]:443 v4v6 ssl crt /etc/haproxy/ssl/bodge.cloud.pem alpn h2,http/1.1
   acl ipwtf hdr(Host),lower,field(1,:),word(-1,.,2) ip.wtf
   default_backend ipwtf
   tcp-request inspect-delay 10s
   tcp-request content switch-mode http if !ipwtf
   use_backend cloud-regions.bodge.cloud if !ipwtf

backend ipwtf
   mode tcp
   server ipwtf localhost:8080

backend cloud-regions.bodge.cloud
   mode http
   server cr localhost:8080



Hi,

Thanks ! I'm able to reproduce the segfault. I'm on it.

--
Christopher Faulet



Re: Ability to set response in lua via HTTP class

2022-06-09 Thread Christopher Faulet

Le 6/8/22 à 15:22, Yuriy Ivkin a écrit :

Greetings!

  Is the TXN.done(txn[, reply]) function in haproxy 2.6 what I am looking for ?

08.06.2022 12:53, Yuriy Ivkin пишет:


Greetings!

 In the lua HTTP class did not find a method allows to override response body. 
May be it is possible via some other methods ?


 If not, are there any reasons should prevent me to write it ?



Hi,

It depends on what you want to achieve. If you want to return a custom response, 
you can indeed register a lua action and call it in an http-response rule. You 
can create a Reply object with some original response headers, if necessary, and 
with a custom body. In this case, TXN.done() should be used.


However, this solution is limited to one buffer. You cannot stream the response 
payload. In the same way, you must wait for the response body to rewrite it. But 
only if the whole response fits in a buffer. With this solution, it is not 
possible to rewrite a  too big response body.


If you want to rewrite a large response, you must write a lua filter. It is more 
difficult. The API is a bit rough. But it is more versatile.


--
Christopher Faulet



Re: [PATCH] CLEANUP: Re-apply xalloc_size.cocci (2)

2022-06-02 Thread Christopher Faulet

Le 6/1/22 à 21:58, Tim Duesterhus a écrit :

This reapplies the xalloc_size.cocci patch across the whole `src/` tree.



Merged. Thanks !

--
Christopher Faulet



Re: SV: Traffic from HAproxy get error 401 and 500

2022-06-02 Thread Christopher Faulet

Le 6/1/22 à 23:39, Henning Svane a écrit :

Hi

I have tried haproxy -d and here I saw 401 and 500.

But I have also seen this, but I have and Error I do not how to fix:

odin@haproxy01:~$ sudo haproxy -d -f /home/odin/haproxy07e.cfg

Available polling systems :

   epoll : pref=300,  test result OK

    poll : pref=200,  test result OK

  select : pref=150,  test result FAILED

Total: 3 (2 usable), will use epoll.

Available filters :

     [SPOE] spoe

     [CACHE] cache

     [FCGI] fcgi-app

     [COMP] compression

     [TRACE] trace

Using epoll() as the polling mechanism.

But to your question

I have attached the file debug.txt which are the output from haproxy -d whenI 
try to open Outlook.


There are some errors but I do not what they mean.




Hi,

You should specify your HAProxy version and share your redacted configuration. 
Otherwise it will be hard to help you. Log messages may also help.


Note that NTLM authenticates a connection not a request. Thus you must be sure 
to keep connections alive. This may explain your 401 responses. About the 500 
errors, it may be because of header rewrite failures. You may check 'wrew' 
counter in "show stat" command output or in the csv export of the HTTP state page.


Regards,
--
Christopher Faulet



Re: Peers using heavily single cpu core

2022-05-17 Thread Christopher Faulet

Le 4/20/22 à 14:51, Maciej Zdeb a écrit :

Hi Willy,
I saw Christopher changes are now merged. I was wondering how to proceed with my 
issue. Right now in stream_new() I'm able to get cs_endpoint and appctx (if 
endpoint is applet), so I can get thread_mask of appctx to create a stream task 
on the same thread. Is this approach correct?




Hi Maciej,

I've finally finish my applet refactoring. Now it is possible to choose where to 
start an applet. I've also merged your patches. Peer applets must now be 
balanced across threads. So, you may give it a try to be sure it solves your 
issue and also validate everything works fine.


--
Christopher Faulet



[ANNOUNCE] haproxy-2.0.29

2022-05-13 Thread Christopher Faulet

Hi,

HAProxy 2.0.29 was released on 2022/05/13. It added 41 new commits
after version 2.0.28.

Here are the issues fixed by this release:

 * An internal issue leading to truncated messages was fixed. When data were
   mixed with an error report, connection errors could be handled too early
   by the stream-interface. Now connection errors are only considered by the
   stream-interface during the connection establishment. After that, it
   relies on the conn-stream to be notified of any error.

 * An issue in the pass-through multiplexer, exposed by the previous fix,
   and that may lead to a loop at 100% CPU was fixed. Connection error was
   not properly reported to the conn-stream on the sending path.

 * Still on the pass-through multiplexer, a fix of the previous release was
   reverted because it introduced a regression in legacy HTTP mode. A crash
   could be experienced when a keep-alive backend connection was
   reused. While the fix is valid for higher versions, it is not applicable
   for this one.

 * A major issue in the DNS part was fixed. A concurrency issue that could
   lead to a crash when a DNS request was failing. Because of some missing
   locks on dgram structure, it was possible to set the UDP socket FD to -1
   on a thread while it as used to access to fdtab array on another thread.

 * Two issues in the H1 multiplexer were fixed. First, Connection error was
   reported to early, when there were still pending data for the stream.
   Because of this bug, last pending data could be truncated. Now the
   connection error is reported only if there is no pending data. The second
   issue is a problem about full buffer detection during the trailers
   parsing. Because of this bug, it was possible to block the message
   parsing till the timeout expiration. The same bug was fixed about
   processing of EOM block.

 * Some issues in the H2 multiplexers were fixed. First the GOAWAY frame is
   no longer sent if SETTINGS were not sent. Then, as announced, the
   "timeout http-keep-alive" and "timeout http-request" are now respected
   and work as documented, so that it will finally be possible to force such
   connections to be closed when no request comes even if they're seeing
   control traffic such as PING frames. This can typically happen in some
   server-to-server communications whereby the client application makes use
   of PING frames to make sure the connection is still alive.

 * A crash of HAproxy was fixed. It happened when HAproxy was compiled
   without the PCRE/PCRE2 support if it tried to replace part of the uri
   while the path is invalid or not specified.

 * Some bugs in CLI commands were fixed. "show resolvers" and "show cli
   sockets" commands were not properly yielding and some locks were missing
   in "show map" command. It is very unlikely to have ever hit one of these
   bugs, but not impossible though. In addition, expired entries were
   displayed in "show cache" output. These entries are now evicted instead
   of being listed.

 * The watchdog could be erroneously triggered because an unitialized value
   was not tested. It was possible to encounter this issue in the master
   just after loading the configuration.

 * It was reported the maximum line length on the server-state file was too
   small. It was increased to 2kB.

 * Some shared pools were not properly released on exit.

 * An improvement which is not related to the code, with the precious help
   of Tim and Cyril, we could finally set up an automatic generation of the
   HTML documentation. It's performed daily and published on github pages at
   http://docs.haproxy.org.

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Documentation: http://docs.haproxy.org/
   Wiki : https://github.com/haproxy/wiki/wiki
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Sources  : http://www.haproxy.org/download/2.0/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.0.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.0.git
   Changelog: http://www.haproxy.org/download/2.0/src/CHANGELOG
   Pending bugs : http://www.haproxy.org/l/pending-bugs
   Reviewed bugs: http://www.haproxy.org/l/reviewed-bugs
   Code reports : http://www.haproxy.org/l/code-reports
   Latest builds: http://www.haproxy.org/l/dev-packages


---
Complete changelog :
Christopher Faulet (7):
  Revert "BUG/MAJOR: mux-pt: Always destroy the backend connection on 
detach"
  BUG/MEDIUM: http-act: Don't replace URI if path is not found or invalid
  BUG/MEDIUM: mux-h1: Don't request more room on partial trailers
  BUG/MEDIUM: compression: Don't forget to update htx_sl and http_msg flags
  B

[ANNOUNCE] haproxy-2.2.24

2022-05-13 Thread Christopher Faulet

Hi,

HAProxy 2.2.24 was released on 2022/05/13. It added 11 new commits
after version 2.2.23.

Here are the issues fixed by this release:

 * A major issue in the DNS part was fixed. A concurrency issue that could
   lead to a crash when a DNS request was failing. Because of some missing
   locks on dgram structure, it was possible to set the UDP socket FD to -1
   on a thread while it as used to access to fdtab array on another thread.

 * The watchdog could be erroneously triggered because an uninitialized value
   was not tested. It was possible to encounter this issue in the master
   just after loading the configuration.

 * H2 streams were marked as open after processing it instead of before. It
   could be an issue when a client didn't respect the H2
   MAX_CONCURRENT_STREAMS setting because the max_id was only updated on the
   success path. Thus, under some circumstances an connection error was
   reported instead of a stream error.

 * It was reported the maximum line length on the server-state file was too
   small. It was increased to 2kB.

 * Some bugs in CLI commands were fixed. "show resolvers" and "show cli
   sockets" commands were not properly yielding and some locks were missing
   in "show map" command. It is very unlikely to have ever hit one of these
   bugs, but not impossible though.

Thanks everyone for your help and your contributions !

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Documentation: http://docs.haproxy.org/
   Wiki : https://github.com/haproxy/wiki/wiki
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Sources  : http://www.haproxy.org/download/2.2/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.2.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.2.git
   Changelog: http://www.haproxy.org/download/2.2/src/CHANGELOG
   Pending bugs : http://www.haproxy.org/l/pending-bugs
   Reviewed bugs: http://www.haproxy.org/l/reviewed-bugs
   Code reports : http://www.haproxy.org/l/code-reports
   Latest builds: http://www.haproxy.org/l/dev-packages


---
Complete changelog :
Christopher Faulet (2):
  DOC: config: Update doc for PR/PH session states to warn about rewrite 
failures
  CLEANUP: mux-h1: Fix comments and error messages for global options

Emeric Brun (1):
  BUG/MAJOR: dns: multi-thread concurrency issue on UDP socket

William Lallemand (2):
  BUG/MINOR: tcp/http: release the expr of set-{src,dst}[-port]
  BUG/MEDIUM: wdt: don't trigger the watchdog when p is unitialized

Willy Tarreau (5):
  BUG/MINOR: mux-h2: mark the stream as open before processing it not after
  BUG/MEDIUM: cli: make "show cli sockets" really yield
  BUG/MINOR: map/cli: protect the backref list during "show map" errors
  BUG/MINOR: map/cli: make sure patterns don't vanish under "show map"'s 
init
  DOC: fix typo "ant" for "and" in INSTALL

vigneshsp (1):
  BUG/MINOR: server: Make SRV_STATE_LINE_MAXLEN value from 512 to 2kB (2000 
bytes).

--
Christopher Faulet



[ANNOUNCE] haproxy-2.4.17

2022-05-13 Thread Christopher Faulet

Hi,

HAProxy 2.4.17 was released on 2022/05/13. It added 24 new commits
after version 2.4.16.

Here are the issues fixed by this release:

 * A regression in the H1 multiplexer was fixed. If an H2 message announced
   the payload size with a Content-Length header and contained trailers, an
   internal error was triggered during forwarding on the other side, in the
   H1 multiplexer.

 * A major issue in the DNS part was fixed. A concurrency issue that could
   lead to a crash when a DNS request was failing. Because of some missing
   locks on dgram structure, it was possible to set the UDP socket FD to -1
   on a thread while it as used to access to fdtab array on another thread.

 * A server abort or a server timeout could be experienced with FCGI backend
   connections when the END_REQUEST record was delayed for responses with no
   content-length.

 * A timing issue could lead to some delay in the server-side connection
   establishment. It was a tricky issue, but sometimes the server-side
   connection attempts were only validated after the "timeout connect"
   value, and only with H2 clients.

 * H2 streams were marked as open after processing it instead of before. It
   could be an issue when a client didn't respect the H2
   MAX_CONCURRENT_STREAMS setting because the max_id was only updated on the
   success path. Thus, under some circumstances a connection error was
   reported instead of a stream error.

 * The watchdog could be erroneously triggered because an uninitialized value
   was not tested. It was possible to encounter this issue in the master
   just after loading the configuration.

 * It was reported the maximum line length on the server-state file was too
   small. It was increased to 2kB.

 * Some bugs in CLI commands were fixed. "show resolvers" and "show cli
   sockets" commands were not properly yielding and some locks were missing
   in "show map" command. It is very unlikely to have ever hit one of these
   bugs, but not impossible though.

Thanks everyone for your help and your contributions !

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Documentation: http://docs.haproxy.org/
   Wiki : https://github.com/haproxy/wiki/wiki
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Sources  : http://www.haproxy.org/download/2.4/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.4.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.4.git
   Changelog: http://www.haproxy.org/download/2.4/src/CHANGELOG
   Pending bugs : http://www.haproxy.org/l/pending-bugs
   Reviewed bugs: http://www.haproxy.org/l/reviewed-bugs
   Code reports : http://www.haproxy.org/l/code-reports
   Latest builds: http://www.haproxy.org/l/dev-packages


---
Complete changelog :
Christopher Faulet (4):
  BUG/MEDIUM: mux-fcgi: Be sure to never set EOM flag on an empty HTX 
message
  BUG/MEDIUM: mux-h1: Be able to handle trailers when C-L header was 
specified
  DOC: config: Update doc for PR/PH session states to warn about rewrite 
failures
  CLEANUP: mux-h1: Fix comments and error messages for global options

Emeric Brun (1):
  BUG/MAJOR: dns: multi-thread concurrency issue on UDP socket

Ilya Shipitsin (2):
  CI: github actions: update LibreSSL to 3.5.2
  CI: dynamically determine actual version of h2spec

William Lallemand (2):
  BUG/MINOR: tcp/http: release the expr of set-{src,dst}[-port]
  BUG/MEDIUM: wdt: don't trigger the watchdog when p is unitialized

Willy Tarreau (14):
  SCRIPTS: announce-release: add URL of dev packages
  BUG/MINOR: mux-h2: mark the stream as open before processing it not after
  MINOR: mux-h2: report a trace event when failing to create a new stream
  BUG/MEDIUM: resolvers: make "show resolvers" properly yield
  BUG/MEDIUM: cli: make "show cli sockets" really yield
  BUG/MINOR: map/cli: protect the backref list during "show map" errors
  BUG/MINOR: map/cli: make sure patterns don't vanish under "show map"'s 
init
  DOC: fix typo "ant" for "and" in INSTALL
  BUILD: ssl: work around bogus warning in gcc 12's -Wformat-truncation
  BUILD: debug: work around gcc-12 excessive -Warray-bounds warnings
  BUILD: listener: shut report of possible null-deref in listener_accept()
  BUG/MEDIUM: ssl: fix the gcc-12 broken fix :-(
  DOC: install: update gcc version requirements
  BUG/MINOR: conn_stream: do not confirm a connection from the frontend path

vigneshsp (1):
  BUG/MINOR: server: Make SRV_STATE_LINE_MAXLEN value from 512 to 2kB (2000 
bytes).

--
Christopher Faulet



[ANNOUNCE] haproxy-2.5.7

2022-05-13 Thread Christopher Faulet
: http://www.haproxy.org/l/pending-bugs
   Reviewed bugs: http://www.haproxy.org/l/reviewed-bugs
   Code reports : http://www.haproxy.org/l/code-reports
   Latest builds: http://www.haproxy.org/l/dev-packages


---
Complete changelog :
Boyang Li (2):
  BUG/MEDIUM: lua: fix argument handling in data removal functions
  DOC/MINOR: fix typos in the lua-api document

Christopher Faulet (7):
  BUG/MEDIUM: http-ana: Fix memleak in redirect rules with ignore-empty 
option
  BUG/MEDIUM: httpclient: Fix loop consuming HTX blocks from the response 
channel
  BUG/MEDIUM: mux-fcgi: Be sure to never set EOM flag on an empty HTX 
message
  BUG/MEDIUM: mux-h1: Be able to handle trailers when C-L header was 
specified
  DOC: config: Update doc for PR/PH session states to warn about rewrite 
failures
  MINOR: mux-h1: Add global option accpet payload for any HTTP/1.0 requests
  CLEANUP: mux-h1: Fix comments and error messages for global options

Emeric Brun (1):
  BUG/MAJOR: dns: multi-thread concurrency issue on UDP socket

Ilya Shipitsin (2):
  CI: github actions: update LibreSSL to 3.5.2
  CI: dynamically determine actual version of h2spec

Remi Tricot-Le Breton (2):
  MINOR: connection: Add way to disable active connection closing during 
soft-stop
  BUG/MINOR: ssl: Fix typos in crl-file related CLI commands

Thomas Prückl (1):
  MINOR: ssl: add a new global option "tune.ssl.hard-maxrecord"

Tim Duesterhus (1):
  BUG/MINOR: resolvers: Fix memory leak in resolvers_deinit()

William Lallemand (4):
  BUG/MINOR: tcp/http: release the expr of set-{src,dst}[-port]
  BUG/MINOR: startup: usage() when no -cc arguments
  BUG/MEDIUM: ssl/cli: fix yielding in show_cafile_detail
  BUG/MEDIUM: wdt: don't trigger the watchdog when p is unitialized

Willy Tarreau (22):
  BUILD: compiler: properly distinguish weak and global symbols
  BUG/MINOR: pools: make sure to also destroy shared pools in 
pool_destroy_all()
  SCRIPTS: announce-release: add URL of dev packages
  BUG/MINOR: mux-h2: mark the stream as open before processing it not after
  MINOR: mux-h2: report a trace event when failing to create a new stream
  BUG/MEDIUM: resolvers: make "show resolvers" properly yield
  BUG/MEDIUM: cli: make "show cli sockets" really yield
  BUG/MINOR: proxy/cli: don't enumerate internal proxies on "show backend"
  BUG/MINOR: map/cli: protect the backref list during "show map" errors
  BUG/MINOR: map/cli: make sure patterns don't vanish under "show map"'s 
init
  BUG/MINOR: ssl/cli: fix "show ssl ca-file/crl-file" not to mix cli+ssl 
contexts
  BUG/MINOR: ssl/cli: fix "show ssl ca-file " not to mix cli+ssl 
contexts
  BUG/MINOR: ssl/cli: fix "show ssl crl-file" not to mix cli+ssl contexts
  BUG/MINOR: ssl/cli: fix "show ssl cert" not to mix cli+ssl contexts
  DOC: fix typo "ant" for "and" in INSTALL
  BUILD: ssl: work around bogus warning in gcc 12's -Wformat-truncation
  BUILD: debug: work around gcc-12 excessive -Warray-bounds warnings
  BUILD: listener: shut report of possible null-deref in listener_accept()
  BUG/MEDIUM: ssl: fix the gcc-12 broken fix :-(
  DOC: install: update gcc version requirements
  BUG/MINOR: conn_stream: do not confirm a connection from the frontend path
  CLEANUP: applet: make appctx_new() initialize the whole appctx

vigneshsp (1):
  BUG/MINOR: server: Make SRV_STATE_LINE_MAXLEN value from 512 to 2kB (2000 
bytes).

--
Christopher Faulet



Re: [PATCH] BUG/MEDIUM: lua: fix argument handling in data removal functions

2022-05-13 Thread Christopher Faulet

Le 5/10/22 à 19:47, Boyang Li a écrit :

Lua API Channel.remove() and HTTPMessage.remove() expects 1 to 3
arguments (counting the manipulated object), with offset and length
being the 2nd and 3rd argument, respectively.

hlua_{channel,http_msg}_del_data() incorrectly gets the 3rd argument as
offset, and 4th (nonexistent) as length. hlua_http_msg_del_data() also
improperly checks arguments. This patch fixes argument handling in both.

Must be backported to 2.5.


Thanks ! Both patches were merged.

--
Christopher Faulet



[ANNOUNCE] haproxy-2.2.23

2022-05-06 Thread Christopher Faulet

Hi,

HAProxy 2.2.23 was released on 2022/05/06. It added 40 new commits
after version 2.2.22.

This release contains more or less the same fixes than the 2.3.20:

  * An internal issue leading to truncated messages was fixed. When data
were mixed with an error report, connection errors could be handled too
early by the stream-interface. Now connection errors are only considered
by the stream-interface during the connection establishment. After that,
it relies on the conn-stream to be notified of any error.

  * An issue in the pass-through multiplexer, exposed by the previous fix,
and that may lead to a loop at 100% CPU was fixed. Connection error was
not properly reported to the conn-stream on the sending path.

  * An issue with the FCGI multiplexer when the response is compressed was
fixed. The FCGI application was rewriting the response headers modifying
HTX flags while the compression filter was doing so by modifying the
HTTP message flags. Thus some modification performed on a side were not
detected by the other, leading to produce invalid responses. Now, the
flags of both structures are systematically updated.

  * An issue with responses to HEAD requests sent to FCGI servers was fixed.
A "Content-Length: 0" header was erroneously added on the bodyless
responses while it should not. Indeed, if the expected payload size is
not specified by the server, HAProxy must not add this header because it
cannot know it. In addition, still in the FCGI multiplexer, the parsing
of headers and trailers was fixed to properly handle parsing errors.

  * Two issues in the H1 multiplexer were fixed. First, Connection error was
reported to early, when there were still pending data for the
stream. Because of this bug, last pending data could be truncated. Now
the connection error is reported only if there is no pending data. The
second issue is a problem about full buffer detection during the
trailers parsing. Because of this bug, it was possible to block the
message parsing till the timeout expiration. The same bug was fixed
about processing of EOM block.

  * Some issues in the H2 multiplexers were fixed. First the GOAWAY frame is
no longer sent if SETTINGS were not sent. Then, as announced, the
"timeout http-keep-alive" and "timeout http-request" are now respected
and work as documented, so that it will finally be possible to force
such connections to be closed when no request comes even if they're
seeing control traffic such as PING frames. This can typically happen in
some server-to-server communications whereby the client application
makes use of PING frames to make sure the connection is still alive.

  * A crash of HAproxy was fixed. It happened when HAproxy was compiled
without the PCRE/PCRE2 support if it tried to replace part of the uri
while the path is invalid or not specified.

  * An issue with url_enc() converter was fixed. It was able to crush HTTP
headers. It is now fixed.

  * Expired entries were displayed in "show cache" output. These entries are
now evicted instead of being listed.

Thanks everyone for your help and your contributions !

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Documentation: http://docs.haproxy.org/
   Wiki : https://github.com/haproxy/wiki/wiki
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Sources  : http://www.haproxy.org/download/2.2/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.2.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.2.git
   Changelog: http://www.haproxy.org/download/2.2/src/CHANGELOG
   Pending bugs : http://www.haproxy.org/l/pending-bugs
   Reviewed bugs: http://www.haproxy.org/l/reviewed-bugs
   Code reports : http://www.haproxy.org/l/code-reports
   Latest builds: http://www.haproxy.org/l/dev-packages


---
Complete changelog :
Christopher Faulet (12):
  BUG/MEDIUM: mux-fcgi: Properly handle return value of headers/trailers 
parsing
  BUG/MEDIUM: mux-h1: Properly detect full buffer cases during message 
parsing
  BUG/MEDIUM: mux-h1: Properly detect full buffer cases when adding EOM 
block
  BUG/MINOR: fcgi-app: Don't add C-L header on response to HEAD requests
  BUG/MEDIUM: http-conv: Fix url_enc() to not crush const samples
  BUG/MEDIUM: http-act: Don't replace URI if path is not found or invalid
  BUG/MEDIUM: mux-h1: Don't request more room on partial trailers
  BUG/MEDIUM: fcgi-app: Use http_msg flags to know if C-L header can be 
added
  BUG/MEDIUM: compression: Don't forget to update htx_sl and http_msg flags
  BUG/MINOR: cache: Disable cache if applet creation fails
  BUG/MINOR: rules: Fix check_

[ANNOUNCE] haproxy-2.3.20

2022-04-29 Thread Christopher Faulet

Hi,

HAProxy 2.3.20 was released on 2022/04/29. It added 41 new commits
after version 2.3.19.

The 2.3 branch was planned to be EOL last quarter. There are no longer bug
reports for this specific branch. Thus, it is probably the last 2.3
release. Except if there are critical bugs in next few weeks, no further
release should be expected. You should have no reason to deploy it anymore
in a production environment. Use the 2.4 instead. No specific support should
no longer be expected on the 2.3.

Here are main changes for this release, cut-pasted from 2.4.16 announce:

 * An internal issue leading to truncated messages was fixed. When data were
   mixed with an error report, connection errors could be handled too early
   by the stream-interface. Now connection errors are only considered by the
   stream-interface during the connection establishment. After that, it
   relies on the conn-stream to be notified of any error.

 * An issue in the pass-through multiplexer, exposed by the previous fix,
   and that may lead to a loop at 100% CPU was fixed. Connection error was
   not properly reported to the conn-stream on the sending path.

 * An issue with the FCGI multiplexer when the response is compressed was
   fixed. The FCGI application was rewriting the response headers modifying
   HTX flags while the compression filter was doing so by modifying the HTTP
   message flags. Thus some modification performed on a side were not
   detected by the other, leading to produce invalid responses. Now, the
   flags of both structures are systematically updated.

 * An issue with responses to HEAD requests sent to FCGI servers was fixed.
   A "Content-Length: 0" header was erroneously added on the bodyless
   responses while it should not. Indeed, if the expected payload size is
   not specified by the server, HAProxy must not add this header because it
   cannot know it. In addition, still in the FCGI multiplexer, the parsing
   of headers and trailers was fixed to properly handle parsing errors.

 * Two issues in the H1 multiplexer were fixed. First, Connection error was
   reported to early, when there were still pending data for the
   stream. Because of this bug, last pending data could be truncated. Now
   the connection error is reported only if there is no pending data. The
   second issue is a problem about full buffer detection during the trailers
   parsing. Because of this bug, it was possible to block the message
   parsing till the timeout expiration. The same bug was fixed about
   processing of EOM block.

 * Some issues in the H2 multiplexers were fixed. First the GOAWAY frame is
   no longer sent if SETTINGS were not sent. Then, as announced, the
   "timeout http-keep-alive" and "timeout http-request" are now respected
   and work as documented, so that it will finally be possible to force such
   connections to be closed when no request comes even if they're seeing
   control traffic such as PING frames. This can typically happen in some
   server-to-server communications whereby the client application makes use
   of PING frames to make sure the connection is still alive.

 * A crash of HAproxy was fixed. It happened when HAproxy was compiled
   without the PCRE/PCRE2 support if it tried to replace part of the uri
   while the path is invalid or not specified.

 * An issue with url_enc() converter was fixed. It was able to crush HTTP
   headers. It is now fixed.

 * Expired entries were displayed in "show cache" output. These entries are
   now evicted instead of being listed.

Thanks everyone for your help and your contributions !

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Documentation: http://docs.haproxy.org/
   Wiki : https://github.com/haproxy/wiki/wiki
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Sources  : http://www.haproxy.org/download/2.3/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.3.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.3.git
   Changelog: http://www.haproxy.org/download/2.3/src/CHANGELOG
   Pending bugs : http://www.haproxy.org/l/pending-bugs
   Reviewed bugs: http://www.haproxy.org/l/reviewed-bugs
   Code reports : http://www.haproxy.org/l/code-reports


---
Complete changelog :
Christopher Faulet (14):
  BUG/MEDIUM: mux-fcgi: Properly handle return value of headers/trailers 
parsing
  BUG/MEDIUM: mux-h1: Properly detect full buffer cases during message 
parsing
  BUG/MEDIUM: mux-h1: Properly detect full buffer cases when adding EOM 
block
  BUG/MINOR: fcgi-app: Don't add C-L header on response to HEAD requests
  BUG/MEDIUM: http-conv: Fix url_enc() to not crush const samples
  BUG/MEDIUM: http-act: Don't replace URI if path is not found or invalid
  BUG/MEDIUM: mu

[ANNOUNCE] haproxy-2.4.16

2022-04-29 Thread Christopher Faulet
ite index   : http://www.haproxy.org/
   Documentation: http://docs.haproxy.org/
   Wiki : https://github.com/haproxy/wiki/wiki
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Sources  : http://www.haproxy.org/download/2.4/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.4.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.4.git
   Changelog: http://www.haproxy.org/download/2.4/src/CHANGELOG
   Pending bugs : http://www.haproxy.org/l/pending-bugs
   Reviewed bugs: http://www.haproxy.org/l/reviewed-bugs
   Code reports : http://www.haproxy.org/l/code-reports


---
Complete changelog :
Christopher Faulet (19):
  DOC: config: Explictly add supported MQTT versions
  BUG/MEDIUM: mux-fcgi: Properly handle return value of headers/trailers 
parsing
  BUG/MEDIUM: mux-h1: Properly detect full buffer cases during message 
parsing
  BUG/MINOR: fcgi-app: Don't add C-L header on response to HEAD requests
  BUG/MEDIUM: stats: Be sure to never set EOM flag on an empty HTX message
  BUG/MEDIUM: hlua: Don't set EOM flag on an empty HTX message in HTTP 
applet
  BUG/MEDIUM: promex: Be sure to never set EOM flag on an empty HTX message
  BUG/MEDIUM: mux-h1: Set outgoing message to DONE when payload length is 
reached
  BUG/MEDIUM: http-conv: Fix url_enc() to not crush const samples
  BUG/MEDIUM: http-act: Don't replace URI if path is not found or invalid
  BUG/MEDIUM: mux-h1: Don't request more room on partial trailers
  BUG/MEDIUM: fcgi-app: Use http_msg flags to know if C-L header can be 
added
  BUG/MEDIUM: compression: Don't forget to update htx_sl and http_msg flags
  BUG/MINOR: cache: Disable cache if applet creation fails
  BUG/MAJOR: connection: Never remove connection from idle lists outside 
the lock
  BUG/MINOR: rules: Fix check_capture() function to use the right rule 
arguments
  REGTESTS: fix the race conditions in be2dec.vtc ad field.vtc
  CLEANUP: acl: Remove unused variable when releasing an acl expression
  BUILD: opentracing: Fix OT build due to misuse of var_clear()

Dhruv Jain (1):
  MEDIUM: mqtt: support mqtt_is_valid and mqtt_field_value converters for 
MQTTv3.1

Ilya Shipitsin (3):
  CI: github actions: switch to LibreSSL-3.5.1
  CI: github actions: update OpenSSL to 3.0.2
  CI: cirrus: switch to FreeBSD-13.0

Lukas Tribus (1):
  DOC: reflect H2 timeout changes

Miroslav Zagorac (8):
  BUG/MINOR: opentracing: setting the return value in function 
flt_ot_var_set()
  EXAMPLES: opentracing: refined shell scripts for testing filter 
performance
  DOC: opentracing: corrected comments in function descriptions
  CLEANUP: opentracing: removed unused function flt_ot_var_unset()
  CLEANUP: opentracing: removed unused function flt_ot_var_get()
  CLEANUP: opentracing: added flt_ot_smp_init() function
  CLEANUP: opentracing: added variable to store variable length
  DEBUG: opentracing: show return values of all functions in the debug 
output

Tim Duesterhus (3):
  CI: Update to actions/checkout@v3
  CI: Update to actions/cache@v3
  BUG/MINOR: resolvers: Fix memory leak in resolvers_deinit()

William Lallemand (3):
  BUG/MINOR: tools: fix url2sa return value with IPv4
  BUG/MINOR: server/ssl: free the SNI sample expression
  BUG/MINOR: tools: url2sa reads too far when no port nor path

Willy Tarreau (27):
  BUG/MEDIUM: mux-h1: only turn CO_FL_ERROR to CS_FL_ERROR with empty ibuf
  BUG/MEDIUM: trace: avoid race condition when retrieving session from 
conn->owner
  BUG/MEDIUM: stream-int: do not rely on the connection error once 
established
  MEDIUM: mux-h2: slightly relax timeout management rules
  BUG/MEDIUM: mux-h2: make use of http-request and keep-alive timeouts
  BUG/MINOR: samples: add missing context names for sample fetch functions
  BUG/MINOR: cli/stream: fix "shutdown session" to iterate over all threads
  BUG/MAJOR: mux_pt: always report the connection error to the conn_stream
  BUG/MINOR: mux-h2: do not send GOAWAY if SETTINGS were not sent
  BUG/MINOR: cache: do not display expired entries in "show cache"
  BUILD: debug: mark the __start_mem_stats/__stop_mem_stats symbols as weak
  BUG/MINOR: mux-h2: do not use timeout http-keep-alive on backend side
  BUG/MINOR: mux-h2: use timeout http-request as a fallback for 
http-keep-alive
  BUILD: sched: workaround crazy and dangerous warning in Clang 14
  BUILD: compiler: use a more portable set of asm(".weak") statements
  BUG/MEDIUM: stream: do not abort connection setup too early
  SCRIPTS: announce-release: update the doc's URL
  DOC: lua: update a few doc URLs
  SCRIPTS: announce-release: add shortened links to

[ANNOUNCE] haproxy-2.5.6

2022-04-26 Thread Christopher Faulet
d connection closing mechanism on reload that was
   brought into 2.5 caused some concerns to some users, leading to an option
   to disable it. Now there's a new global setting, "close-spread-time",
   which can be used to indicate that the closure of idle connections should
   be randomly spread over that interval, so that reconnecting clients don't
   all rush at the same time on the new process. This applies both to
   passive close ("connection: close" on responses), and to active close of
   idle connections. For best efficiency, the interval should obviously be
   shorter than the one used in "hard-stop-after" if any. We'll also see how
   to extend the mechanism to allow never to close at all as there's also
   some demand for this.

 * Opentracing was updated. In 2.5 we had to disable the use of variables
   between the plugin and the haproxy core because the code was relying on
   an original misfeature of the variables which was that they would never
   disappear after being created, and this misfeature was fixed in 2.5,
   breaking that part of Opentracing. Miroslav finally found the time to
   address this and rework it in an elegant way so that the module is fully
   functional again.

 * Support for MQTT 3.1 was added.

 * Another improvement which is not related to the code, with the precious
   help of Tim and Cyril, we could finally set up an automatic generation of
   the HTML documentation. It's performed daily and published on github
   pages at http://docs.haproxy.org.

As usual, people using the 2.5 branch are encouraged to migrate to this
version. Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Documentation: http://docs.haproxy.org/
   Wiki : https://github.com/haproxy/wiki/wiki
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Sources  : http://www.haproxy.org/download/2.5/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.5.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.5.git
   Changelog: http://www.haproxy.org/download/2.5/src/CHANGELOG
   Pending bugs : http://www.haproxy.org/l/pending-bugs
   Reviewed bugs: http://www.haproxy.org/l/reviewed-bugs
   Code reports : http://www.haproxy.org/l/code-reports


---
Complete changelog :
Aleksandar Lazic (1):
  DOC: remove double blanks in configuration.txt

Christopher Faulet (21):
  BUG/MINOR: rules: Initialize the list element when allocating a new rule
  DOC: config: Explictly add supported MQTT versions
  BUG/MEDIUM: mux-fcgi: Properly handle return value of headers/trailers 
parsing
  BUG/MEDIUM: mux-h1: Properly detect full buffer cases during message 
parsing
  BUG/MINOR: fcgi-app: Don't add C-L header on response to HEAD requests
  BUG/MEDIUM: stats: Be sure to never set EOM flag on an empty HTX message
  BUG/MEDIUM: hlua: Don't set EOM flag on an empty HTX message in HTTP 
applet
  BUG/MEDIUM: promex: Be sure to never set EOM flag on an empty HTX message
  BUG/MEDIUM: mux-h1: Set outgoing message to DONE when payload length is 
reached
  BUG/MEDIUM: http-conv: Fix url_enc() to not crush const samples
  BUG/MEDIUM: http-act: Don't replace URI if path is not found or invalid
  BUG/MEDIUM: mux-h1: Don't request more room on partial trailers
  BUG/MEDIUM: fcgi-app: Use http_msg flags to know if C-L header can be 
added
  BUG/MEDIUM: compression: Don't forget to update htx_sl and http_msg flags
  BUG/MINOR: cache: Disable cache if applet creation fails
  BUG/MAJOR: connection: Never remove connection from idle lists outside 
the lock
  BUG/MINOR: rules: Forbid captures in defaults section if used by a backend
  BUG/MEDIUM: rules: Be able to use captures defined in defaults section
  BUG/MINOR: rules: Fix check_capture() function to use the right rule 
arguments
  Revert "CI: github actions: disable -Wno-deprecated"
  REGTESTS: fix the race conditions in be2dec.vtc ad field.vtc

Dhruv Jain (1):
  MEDIUM: mqtt: support mqtt_is_valid and mqtt_field_value converters for 
MQTTv3.1

Ilya Shipitsin (4):
  CI: github actions: switch to LibreSSL-3.5.1
  REGTESTS: ssl: use X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY for cert 
check
  CI: github actions: update OpenSSL to 3.0.2
  CI: cirrus: switch to FreeBSD-13.0

Lukas Tribus (1):
  DOC: reflect H2 timeout changes

Miroslav Zagorac (16):
  BUG/MINOR: opentracing: setting the return value in function 
flt_ot_var_set()
  BUG/BUILD: opentracing: fixed OT_DEFINE variable setting
  EXAMPLES: opentracing: refined shell scripts for testing filter 
performance
  DOC: opentracing: corrected comments in function descriptions
  CLEANUP: opentracing: remov

Re: Check interval rise and fall behaviour

2022-03-30 Thread Christopher Faulet

Le 3/29/22 à 18:02, Lais, Alexander a écrit :

Dear all,

We are using the backend health checks to disable flapping backends.

The default values for rise and fall are 2 subsequent succeeded and 3 
subsequent failed checks.

Our check interval is at 1000ms (a little frequent, potentially part of the 
problem).

Here is what we observed, using HAProxy 2.4.4:

1. Falling

It started with the backend being up and then going down (fall).


2022-03-23T21:31:54.942ZHealth check for server 
http-routers-http1/node4 failed, reason: Layer4 timeout, check duration: 
1000ms, status: 2/3 UP.
2022-03-23T21:31:56.920ZHealth check for server 
http-routers-http1/node4 failed, reason: Layer4 timeout, check duration: 
1001ms, status: 1/3 UP.
2022-03-23T21:31:57.931ZHealth check for server 
http-routers-http1/node4 succeeded, reason: Layer7 check passed, code: 200, 
check duration: 1ms, status: 3/3 UP.
2022-03-24T10:03:27.223ZHealth check for server http-routers-http1/node4 failed, 
reason: Layer7 wrong status, code: 503, info: "Service Unavailable", check 
duration: 1ms, status: 2/3 UP.
2022-03-24T10:03:28.234ZHealth check for server http-routers-http1/node4 failed, 
reason: Layer7 wrong status, code: 503, info: "Service Unavailable", check 
duration: 1ms, status: 1/3 UP.
2022-03-24T10:03:29.237ZHealth check for server http-routers-http1/node4 failed, 
reason: Layer7 wrong status, code: 503, info: "Service Unavailable", check 
duration: 1ms, status: 0/2 DOWN.


We go down from 3/3 to 2/3, 1/3 and back up again to 3/3. My assumption is that 
it then measured 2/3, but only needs 2 for rising, i.e. 2/2, which is bumped to 
3/3 as the backend is now considered up.

The backend stays up for a while and then goes down with my expected health 
checks, i.e. 3/3, 2/3, 1/3, 0/3 -> 0/2 (as we need 2 for rise).

2. Rising


2022-03-24T10:12:26.846ZHealth check for server 
http-routers-http1/node4 failed, reason: Layer4 timeout, check duration: 
1000ms, status: 0/2 DOWN.
2022-03-24T10:12:29.843ZHealth check for server http-routers-http1/node4 failed, 
reason: Layer4 connection problem, info: "Connection refused", check duration: 
1ms, status: 0/2 DOWN.
2022-03-24T10:13:43.902ZHealth check for server http-routers-http1/node4 failed, 
reason: Layer7 wrong status, code: 503, info: "Service Unavailable", check 
duration: 2ms, status: 0/2 DOWN.
2022-03-24T10:14:03.039ZHealth check for server 
http-routers-http1/node4 succeeded, reason: Layer7 check passed, code: 200, 
check duration: 1ms, status: 1/2 DOWN.
2022-03-24T10:14:04.079ZHealth check for server 
http-routers-http1/node4 succeeded, reason: Layer7 check passed, code: 200, 
check duration: 1ms, status: 3/3 UP.


So coming up (rise), it goes from 0/2 probes to 1/2 to 3/3. My assumption that 
it goes to 2/2, is considered up and is bumped to 3/3 because for fall we now 
need 3 failed probes.


The documentation describes rise / fall as “number of subsequent probes that 
succeeded / failed.
 From my observations it looks like it is a sliding window of the last n being 
successful, i.e. when the number of fall is larger than rise, it is easier to 
rise back up with a single successful probe.

Maybe I’m misreading the log outputs or drawing the wrong conclusions.

If someone knows by heart how it’s supposed to work based on the code that 
would be great. Otherwise we can dig some more ourselves.



Hi,

Rise and fall values are the number of consecutive successful/unsuccessful 
health checks. When a server is DOWN, we count the number of consecutive 
successful health checks. If the counter reaches the rise value, the server is 
considered as UP. Otherwise, on each failure, the counter is reset. The same is 
done when the server is UP. we count the number of consecutive unsuccessful 
health checks. If the counter reaches the fall value, the server is considered 
as DOWN. Otherwise, on each success, the counter is reset.


Internally it is a bit more complex but the idea is the same.

In logs, the rise value is reported when the server is DOWN (X/rise) and the 
counter is incremented on each success (so from 0 to rise-1). And the fall value 
is reported when the server is UP (Y/fall) and the counter is decremented on 
each failure (from fall to 1). So when the server is set to DOWN state, you will 
never see "0/3 UP" in logs but "0/2 DOWN" instead. The same is true when the 
server is set to UP state, "2/2 UP" is never reported because "0/3 DOWN" is 
reported.


And you're right, with a rise value lower than the fall value it is quicker to 
consider a DOWN server as UP than the opposite. But with a rise to 2, we need 2 
successful health checks to set a server UP.


--
Christopher Faulet



[ANNOUNCE] haproxy-2.0.28

2022-03-14 Thread Christopher Faulet
ate
the channel response buffer while a connect or an analysis timeout
occurred, the stream was woken up in loop because its task was requeued
with an expired date. Now an error is reported when this happens and the
stream processing is interrupted.

Note there is a mechanism to deal with errors on buffers allocation.
Unfortunately, since the 1.7, this mechanism is broken. And it is even
worse now with the multiplexers. All this part must be refactored. But
for now, HAProxy may be partially frozen if too many entities are
waiting for a buffer.

  * An issue with multi-line ESMTP response in the mailer code.

  * An issue in the resolvers code with domain names with a trailing
dot. The trailing dot was not ignored as expected and a junk character
was added at the end of the encoded part of the domain name.

  * An issue in the master CLI. When a command was sent to a worker, the
errors, especially write errors, during the response processing were not
properly handled. The session could remain stuck if a client quickly
closed the connection before the response was fully sent. The maxconn
value of the master CLI is set 10. Thus, it could quickly be
unresponsive if this happened several times.

  * An issue with all HTX applets. The end of a message was only reported at
the HTX level. The channel's flags were not updated accordingly. The
only known visible effect of this bug was some server aborts erroneously
reported in the stats counters.

  * Proxy mode (tcp, http, cli...) is not properly reported when
displayed. Missing "syslog" and "peers" mode can now be reported.

  * The anti-loop protection in process_stream() was improved to only count
the no-progress calls.

Some other issues of lower level of importance were also fixed, such as the
CLI being extremely slow to parse pipelined requests because it was looking
for the line feed first, hence the larger the buffer, the slower it was with
batch updates like ACL/map updates; a possibly truncated pidfile in master
mode; an inconsistency with the parsing of IPv4 addresses.

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.0/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.0.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.0.git
   Changelog: http://www.haproxy.org/download/2.0/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Alex (1):
  DOC: use the req.ssl_sni in examples

Andrew McDermott (1):
  BUG/MAJOR: http/htx: prevent unbounded loop in 
http_manage_server_side_cookies

Christian Ruppert (1):
  DOC: Fix usage/examples of deprecated ACLs

Christopher Faulet (11):
  BUG/MEDIUM: resolvers: Really ignore trailing dot in domain names
  BUG/MEDIUM: mux-h1: Don't wake h1s if mux is blocked on lack of output 
buffer
  BUG/MAJOR: mux-h2: Be sure to always report HTX parsing error to the app 
layer
  BUG/MEDIUM: stream: Abort processing if response buffer allocation fails
  BUG/MINOR: hlua: Set conn-stream/channel EOI flags at the end of request
  BUG/MINOR: stats: Set conn-stream/channel EOI flags at the end of request
  BUG/MINOR: cache: Set conn-stream/channel EOI flags at the end of request
  BUG/MINOR: promex: Set conn-stream/channel EOI flags at the end of request
  DEBUG: cache: Update underlying buffer when loading HTX message in cache 
applet
  BUG/MEDIUM: mcli: Properly handle errors and timeouts during reponse 
processing
  BUG/MAJOR: mux-pt: Always destroy the backend connection on detach

Ilya Shipitsin (1):
  CI: github actions: use cache for SSL libs

Lukas Tribus (2):
  BUG/MINOR: mailers: negotiate SMTP, not ESMTP
  DOC: ssl: req_ssl_sni needs implicit TLS

William Lallemand (4):
  BUG/MINOR: mworker: does not erase the pidfile upon reload
  BUG/MINOR: mworker: fix a FD leak of a sockpair upon a failed reload
  BUG/MINOR: tools: url2sa reads ipv4 too far
  BUG/MINOR: cli: shows correct mode in "show sess"

Willy Tarreau (16):
  MEDIUM: cli: yield between each pipelined command
  MINOR: channel: add new function co_getdelim() to support multiple 
delimiters
  BUG/MINOR: cli: avoid O(bufsize) parsing cost on pipelined commands
  BUG/MEDIUM: mcli: do not try to parse empty buffers
  BUG/MEDIUM: mcli: always realign wrapping buffers before parsing them
  BUG/MEDIUM: mworker: don't lose the stats socket on failed reload
  BUG/MAJOR: spoe: properly detach all agents when releasing t

[ANNOUNCE] haproxy-2.2.22

2022-03-14 Thread Christopher Faulet

Hi,

HAProxy 2.2.22 was released on 2022/03/14. It added 13 new commits
after version 2.2.21.

This one contains the same fixes than the 2.3.19. So, I'm not going to be
really original:

  * An issue in the pass-through multiplexer leading to a connection leak on
the server side when timeout occurred during the connection
establishment. In this case, the server connection was detached from the
application stream but not closed. At this stage the connection could
only be closed by the server, if it was finally accepted, or by the
kernel, after all SYN retries. All versions as far as 2.3 are affected
by this bug.

  * An issue in the master CLI. When a command was sent to a worker, the
errors, especially write errors, during the response processing were not
properly handled. The session could remain stuck if a client quickly
closed the connection before the response was fully sent. The maxconn
value of the master CLI is set 10. Thus, it could quickly be
unresponsive if this happened several times.

  * An issue with all HTX applets. The end of a message was only reported at
the HTX level. The channel's flags were not updated accordingly. The
only known visible effect of this bug was some server aborts erroneously
reported in the stats counters.

  * Proxy mode (tcp, http, cli...) is not properly reported when
displayed. Missing "syslog" and "peers" mode can now be reported.

  * The anti-loop protection in process_stream() was improved to only count
the no-progress calls.

This release cycle was performed to be able to finally release the
2.0.28. It was announced few weeks ago. Twice. But it was delayed because of
lack of time. This time, it must be released tomorrow morning !

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.2/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.2.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.2.git
   Changelog: http://www.haproxy.org/download/2.2/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Christian Ruppert (1):
  DOC: Fix usage/examples of deprecated ACLs

Christopher Faulet (9):
  BUG/MINOR: hlua: Set conn-stream/channel EOI flags at the end of request
  BUG/MINOR: stats: Set conn-stream/channel EOI flags at the end of request
  BUG/MINOR: cache: Set conn-stream/channel EOI flags at the end of request
  BUG/MINOR: promex: Set conn-stream/channel EOI flags at the end of request
  DEBUG: cache: Update underlying buffer when loading HTX message in cache 
applet
  BUG/MEDIUM: mcli: Properly handle errors and timeouts during reponse 
processing
  DEBUG: stream: Add the missing descriptions for stream trace events
  DEBUG: stream: Fix stream trace message to print response buffer state
  BUG/MAJOR: mux-pt: Always destroy the backend connection on detach

William Lallemand (1):
  BUG/MINOR: cli: shows correct mode in "show sess"

Willy Tarreau (2):
  BUG/MINOR: stream: make the call_rate only count the no-progress calls
  BUILD: tree-wide: mark a few numeric constants as explicitly long long

--
Christopher Faulet



[ANNOUNCE] haproxy-2.3.19

2022-03-14 Thread Christopher Faulet

Hi,

HAProxy 2.3.19 was released on 2022/03/14. It added 14 new commits
after version 2.3.18.

All fixes included in this release were already described in the 2.4.14
announcement. Here is a cut-paste of relevant parts:

  * An issue in the pass-through multiplexer leading to a connection leak on
the server side when timeout occurred during the connection
establishment. In this case, the server connection was detached from the
application stream but not closed. At this stage the connection could
only be closed by the server, if it was finally accepted, or by the
kernel, after all SYN retries. All versions as far as 2.3 are affected
by this bug.

  * An issue in the master CLI. When a command was sent to a worker, the
errors, especially write errors, during the response processing were not
properly handled. The session could remain stuck if a client quickly
closed the connection before the response was fully sent. The maxconn
value of the master CLI is set 10. Thus, it could quickly be
unresponsive if this happened several times.

  * An issue with all HTX applets. The end of a message was only reported at
the HTX level. The channel's flags were not updated accordingly. The
only known visible effect of this bug was some server aborts erroneously
reported in the stats counters.

  * Proxy mode (tcp, http, cli...) is not properly reported when
displayed. Missing "syslog" and "peers" mode can now be reported.

  * The anti-loop protection in process_stream() was improved to only count
the no-progress calls.

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.3/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.3.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.3.git
   Changelog: http://www.haproxy.org/download/2.3/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Christian Ruppert (1):
  DOC: Fix usage/examples of deprecated ACLs

Christopher Faulet (9):
  BUG/MINOR: hlua: Set conn-stream/channel EOI flags at the end of request
  BUG/MINOR: stats: Set conn-stream/channel EOI flags at the end of request
  BUG/MINOR: cache: Set conn-stream/channel EOI flags at the end of request
  BUG/MINOR: promex: Set conn-stream/channel EOI flags at the end of request
  DEBUG: cache: Update underlying buffer when loading HTX message in cache 
applet
  BUG/MEDIUM: mcli: Properly handle errors and timeouts during reponse 
processing
  DEBUG: stream: Add the missing descriptions for stream trace events
  DEBUG: stream: Fix stream trace message to print response buffer state
  BUG/MAJOR: mux-pt: Always destroy the backend connection on detach

William Lallemand (2):
  BUG/MINOR: add missing modes in proxy_mode_str()
  BUG/MINOR: cli: shows correct mode in "show sess"

Willy Tarreau (2):
  BUG/MINOR: stream: make the call_rate only count the no-progress calls
  BUILD: tree-wide: mark a few numeric constants as explicitly long long

--
Christopher Faulet



[ANNOUNCE] haproxy-2.4.15

2022-03-14 Thread Christopher Faulet

Hi,

HAProxy 2.4.15 was released on 2022/03/14. It added 26 new commits
after version 2.4.14.

This one contains more or less the same fixes than the 2.5.5, except
2.5-specific ones :

  * An issue in the pass-through multiplexer leading to a connection leak on
the server side when timeout occurred during the connection
establishment. In this case, the server connection was detached from the
application stream but not closed. At this stage the connection could
only be closed by the server, if it was finally accepted, or by the
kernel, after all SYN retries. All versions as far as 2.3 are affected
by this bug.

  * An issue in the master CLI. When a command was sent to a worker, the
errors, especially write errors, during the response processing were not
properly handled. The session could remain stuck if a client quickly
closed the connection before the response was fully sent. The maxconn
value of the master CLI is set 10. Thus, it could quickly be
unresponsive if this happened several times.

  * A possible null deref in the htx_xfer_blks() function, when headers or
trailers were partially transferred. Concretely, it was only possible
when H2 trailers were copied from the mux to the channel buffer.

  * An issue with all HTX applets. The end of a message was only reported at
the HTX level. The channel's flags were not updated accordingly. The
only known visible effect of this bug was some server aborts erroneously
reported in the stats counters.

  * A theoretical risk of memleak in session_accept_fd() because of a wrong
goto label on the error path.

  * An alignment issue with pool_head structure.

  * Proxy mode (tcp, http, cli...) is not properly reported when
displayed. Missing "syslog" and "peers" mode can now be reported.

  * "no-memory-trimming" global option was added to disable call to
malloc_trim(). Some users with very large numbers of connections have
been facing extremely long malloc_trim() calls on reload that managed to
trigger the watchdog! That's a bit counter-productive. It's even
possible that some implementations are not perfectly reliable or that
their trimming time grows quadratically with the memory used. With this
option, it is possible to disable this mechanism.

  * The anti-loop protection in process_stream() was improved to only count
the no-progress calls.

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.4/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.4.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.4.git
   Changelog: http://www.haproxy.org/download/2.4/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Christian Ruppert (1):
  DOC: Fix usage/examples of deprecated ACLs

Christopher Faulet (12):
  BUG/MEDIUM: htx: Fix a possible null derefs in htx_xfer_blks()
  REGTESTS: fix the race conditions in normalize_uri.vtc
  REGTESTS: fix the race conditions in secure_memcmp.vtc
  BUG/MINOR: hlua: Set conn-stream/channel EOI flags at the end of request
  BUG/MINOR: stats: Set conn-stream/channel EOI flags at the end of request
  BUG/MINOR: cache: Set conn-stream/channel EOI flags at the end of request
  BUG/MINOR: promex: Set conn-stream/channel EOI flags at the end of request
  DEBUG: cache: Update underlying buffer when loading HTX message in cache 
applet
  BUG/MEDIUM: mcli: Properly handle errors and timeouts during reponse 
processing
  DEBUG: stream: Add the missing descriptions for stream trace events
  DEBUG: stream: Fix stream trace message to print response buffer state
  BUG/MAJOR: mux-pt: Always destroy the backend connection on detach

Ilya Shipitsin (3):
  CI: github actions: add OpenTracing builds
  CI: github actions: use cache for OpenTracing
  CI: github actions: use cache for SSL libs

William Lallemand (2):
  BUG/MINOR: add missing modes in proxy_mode_str()
  BUG/MINOR: cli: shows correct mode in "show sess"

Willy Tarreau (8):
  CI: github actions: add the output of $CC -dM -E-
  BUG/MINOR: pool: always align pool_heads to 64 bytes
  BUG/MEDIUM: pools: fix ha_free() on area in the process of being freed
  MINOR: pools: add a new global option "no-memory-trimming"
  BUILD: pools: fix backport of no-memory-trimming on non-linux OS
  BUG/MINOR: session: fix theoretical risk of memleak in session_accept_fd()
  BUG/MINOR: stream: make the call_rate only count the no

[ANNOUNCE] haproxy-2.5.5

2022-03-14 Thread Christopher Faulet

Hi,

HAProxy 2.5.5 was released on 2022/03/14. It added 39 new commits
after version 2.5.4.

The main issues fixed in this version are:

  * An issue in the pass-through multiplexer leading to a connection leak on
the server side when timeout occurred during the connection
establishment. In this case, the server connection was detached from the
application stream but not closed. At this stage the connection could
only be closed by the server, if it was finally accepted, or by the
kernel, after all SYN retries. All versions as far as 2.3 are affected
by this bug.

  * Two issues in the HTTP client applet. First it was possible to trigger
an infinite loop when the same HTTP client lua instance was used to send
several POST requests. A counter was not reset between the requests.
Then, the applet was unexpectedly able to consume the response before
its analysis by the application stream. To hit the bug, the applet's I/O
handler had to be scheduled before the stream one. The result was a
crash because of a NULL dereferenced pointer.

  * An issue in the master CLI. When a command was sent to a worker, the
errors, especially write errors, during the response processing were not
properly handled. The session could remain stuck if a client quickly
closed the connection before the response was fully sent. The maxconn
value of the master CLI is set 10. Thus, it could quickly be
unresponsive if this happened several times.

  * A possible null deref in the htx_xfer_blks() function, when headers or
trailers were partially transferred. Concretely, it was only possible
when H2 trailers were copied from the mux to the channel buffer.

  * A crash with the FCGI health-checks. When the multi-level source and
destination addresses were introduced, a bug was also introduced. The
FCGI multiplexer was relying on the server stream-interface to set some
parameters (REMOTE_ADDR/REMOTE_PORT and SERVER_NAME/SERVER_PORT). But
there is no stream-interface with the health-check because there is no
stream. Now, the server connection is used instead of the
stream-interface when the origin is a health-check.

  * A design issue for listener-less streams. When a stream was created from
a session without listener, the request analyzers were not properly
set. Concretely, it is only an issue for client applets, more
specifically the HTTP ones. Thus only the HTTP client was affected by
this bug. However, there was no visible effect.

  * An issue with all HTX applets. The end of a message was only reported at
the HTX level. The channel's flags were not updated accordingly. The
only known visible effect of this bug was some server aborts erroneously
reported in the stats counters.

  * A theoretical risk of memleak in session_accept_fd() because of a wrong
goto label on the error path.

  * An alignment issue with pool_head structure.

  * Some build issues were fixed. kFreeBSD is now a distinct target, the old
HA_ATOMIC_LOAD() macro now supports const pointers, few numeric
constants are explicitly marked as long long,

In addition, it adds some improvements:

  * Proxy mode (tcp, http, cli...) is not properly reported when
displayed. Missing "syslog" and "peers" mode can now be reported.

  * "no-memory-trimming" global option was added to disable call to
malloc_trim(). Some users with very large numbers of connections have
been facing extremely long malloc_trim() calls on reload that managed to
trigger the watchdog! That's a bit counter-productive. It's even
possible that some implementations are not perfectly reliable or that
their trimming time grows quadratically with the memory used. With this
option, it is possible to disable this mechanism.

  * The dark mode support of the stat page was updated to be applied on
socket rows.

As usual, people using the 2.5 branch are encouraged to migrate to this
version. Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.5/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.5.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.5.git
   Changelog: http://www.haproxy.org/download/2.5/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Christopher Faulet (16):
  BUG/MEDIUM: mux-fcgi: Don't rely on SI src/dst addresses for FCGI 
health-checks
  BUG/MEDIUM: htx: Fix a possible null derefs in htx_xfer_blks()
  REGTESTS: fix the race conditions in normali

Re: server check inter and timeout check relation

2022-03-14 Thread Christopher Faulet

Le 3/14/22 à 10:53, Artur a écrit :

Hello,

I'd like to know how checks behaves depending on the "inter" and
"timeout check" settings.

Let's try this simplified setup :

backend back
   mode tcp
   timeout check 5s
   server s1 1.2.3.4:80 check inter 2s
   server s2 1.2.3.5:80 check inter 2s

"inter 2s" is the default setup. We should have there one check every 2s
if everything is optimal.
"timeout check 5s" specify that the server check can take up to 5s (once
the connection established).

In this configuration, what happens if the check takes more than 2 seconds ?
Does haproxy wait (up to 5 seconds) for this check to finish before
launching another check or it's still launching checks every 2s anyway ?



Hi,

For a given server, inter/fastinter/downinter timeouts are used to define the 
delay between the end of a health-check and the beginning of the following one. 
This is independent on the evaluation time. Thus in your example, a health-check 
will still run 2s after the end of the previous one, independently on its duration.


--
Christopher Faulet



Re: SV: Incompatible with 'frontend http-request header rule'

2022-03-04 Thread Christopher Faulet

Le 3/4/22 à 00:42, Henning Svane a écrit :

Hi Christopher

I tried your rule and it did not compile, but I am trying to understand it.
/haproxy02.cfg:20] : error detected while parsing an 'http-request tarpit' 
condition : no such ACL : 'http-response'
I placed the rule in the frontend, but was thinking if it should be in the 
backend, as it is here server is called and hereby produce the return code.

I understand the idea in your rule, but at the same time, I do not understand 
the order of execution.
It looks like it has to be executed from the right with the " if { capture.req.uri 
-m beg /login } { status 401 }" first.
But then what?

If I understand correctly
1) You save the request url in a table with capture.req.uri.
2) Then server try to execute the url
3) Based on the server return the http-response (this part I have not fully 
understand yet)
4) If the response is 401 then " http-request tarpit deny_status 429"

I will try to work a little more with you suggestion and see if I can get to 
work.

Regards
Henning


haproxy02.cfg:20] : error detected while parsing an 'http-request tarpit' 
condition : no such ACL : 'http-response'.



Your email client seems to have mangled my reply. Or there is a formatting issue 
on my side. Anyway, it is not one rule with everything on one line, but 2 rules. 
An http-request one to deny the request on its own line and an http-response one 
to track login failures, on another line.


Both can be specified in the frontend, the backend or split. It depends on your 
other rules, if any. With your config snippet, it doesn't matter.



--
Christopher Faulet



[ANNOUNCE] haproxy-2.3.18

2022-03-02 Thread Christopher Faulet
ess couldn't
happen.

  - some issues about errors on buffers allocation. First, in the H1
multiplexer. If we failed to send data because we failed to allocate the
H1 output buffer, the H1 stream was erroneously woken up. This led to a
wakeup loop to send more data while it is not possible because there is
no output buffer. Then, in process_stream(), if we failed to allocate
the channel response buffer while a connect or an analysis timeout
occurred, the stream was woken up in loop because its task was requeued
with an expired date. Now an error is reported when this happens and the
stream processing is interrupted.

Note there is a mechanism to deal with errors on buffers allocation.
Unfortunately, since the 1.7, this mechanism is broken. And it is even
worse now with the multiplexers. All this part must be refactored. But
for now, HAProxy may be partially frozen if too many entities are
waiting for a buffer.

  - some alignment problems that were found when using gcc-11 + RHEL8,
resulting in instant crashes on startup.

  - an issue with multi-line ESMTP response in the mailer code.

  - an issue in the resolvers code with domain names with a trailing
dot. The trailing dot was not ignored as expected and a junk character
was added at the end of the encoded part of the domain name.

  - there were still a number of other issues of lower level of importance,
such as the CLI being extremely slow to parse pipelined requests because
it was looking for the line feed first, hence the larger the buffer, the
slower it was with batch updates like ACL/map updates; a possibly
truncated pidfile in master mode; a bug with the data transfer in the
HTX layer for large data block; an inconsistency with the parsing of
IPv4 addresses.

Note that the EOL of the 2.3 is planned for the end of this quarter. So this
release is probably one of the last 2.3 releases. For everyone running a
2.3, it could be a good idea to migrate to the 2.4.

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.3/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.3.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.3.git
   Changelog: http://www.haproxy.org/download/2.3/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Andrew McDermott (1):
  BUG/MAJOR: http/htx: prevent unbounded loop in 
http_manage_server_side_cookies

Christopher Faulet (9):
  BUG/MEDIUM: htx: Adjust length to add DATA block in an empty HTX buffer
  BUG/MEDIUM: cli: Never wait for more data on client shutdown
  BUG/MINOR: sink: Use the right field in appctx context in release callback
  BUG/MEDIUM: resolvers: Really ignore trailing dot in domain names
  BUG/MEDIUM: htx: Be sure to have a buffer to perform a raw copy of a 
message
  BUG/MEDIUM: mux-h1: Don't wake h1s if mux is blocked on lack of output 
buffer
  BUG/MAJOR: mux-h2: Be sure to always report HTX parsing error to the app 
layer
  BUG/MEDIUM: stream: Abort processing if response buffer allocation fails
  REGTESTS: fix the race conditions in secure_memcmp.vtc

David Carlier (1):
  BUILD/MINOR: fix solaris build with clang.

Ilya Shipitsin (5):
  BUILD: adopt script/build-ssl.sh for OpenSSL-3.0.0beta2
  CI: github actions: add OpenSSL-3.0.0 builds
  CI: github actions: relax OpenSSL-3.0.0 version comparision
  CI: github actions: update OpenSSL to 3.0.1
  CI: github actions: use cache for SSL libs

Lukas Tribus (1):
  BUG/MINOR: mailers: negotiate SMTP, not ESMTP

William Lallemand (6):
  BUG/MINOR: mworker: does not erase the pidfile upon reload
  BUG/MINOR: mworker: fix a FD leak of a sockpair upon a failed reload
  BUILD: fix compilation for OpenSSL-3.0.0-alpha17
  CI: github actions: -Wno-deprecated-declarations with OpenSSL 3.0.0
  CI: github: switch to OpenSSL 3.0.0
  BUG/MINOR: tools: url2sa reads ipv4 too far

Willy Tarreau (20):
  MEDIUM: cli: yield between each pipelined command
  MINOR: channel: add new function co_getdelim() to support multiple 
delimiters
  BUG/MINOR: cli: avoid O(bufsize) parsing cost on pipelined commands
  BUG/MEDIUM: mcli: do not try to parse empty buffers
  BUG/MEDIUM: mcli: always realign wrapping buffers before parsing them
  BUG/MEDIUM: mworker: don't lose the stats socket on failed reload
  MINOR: listener: replace the listener's spinlock with an rwlock
  BUG/MEDIUM: listener: read-lock the listener during accept()
  BUG/MA

Re: Incompatible with 'frontend http-request header rule'

2022-03-02 Thread Christopher Faulet

Le 3/1/22 à 22:00, Henning Svane a écrit :
http-request track-sc0 src table table_login_limiter if { url_beg /login } { 
status 401 }


http-request tarpit deny_status 429 if { sc_http_req_rate(0) gt 10 } { url_beg 
/login }




Hi,

You cannot match on the response status in a request rule. At this stage, the
response is not received yet. So, you should rely on an http-response rule
instead. But, at this stage, url_beg is no longer available because the request
was already sent. You must use capture.req.uri instead.

In addition, because the tracking will be performed during the response
evaluation, you must use table_http_req_rate() converter to look up in your
stick-table. (Note that in your tarpit rule, you must explicitly specify the
table name)

You can try the following rules :

http-request tarpit deny_status 429 if { 
src,table_http_req_rate(table_login_limiter) gt 10 } { url_beg /login }
http-response track-sc0 src table table_login_limiter if { capture.req.uri -m 
beg /login } { status 401 }

You can also match on the url in an http-request rule to set a variable and use
it in the http-response rule.

Regards,
--
Christopher Faulet



[ANNOUNCE] haproxy-2.4.14

2022-02-25 Thread Christopher Faulet

Hi,

HAProxy 2.4.14 was released on 2022/02/25. It added 26 new commits
after version 2.4.13.

The main issues fixed in this version are:

  - A major issue in the H2 multiplexer. An error during the response
processing, after the HEADERS frame parsing, led to a wakeup loop
consuming all the CPU because the error was not properly reported to the
upper layer. For instance, this happened if an invalid header value, an
invalid status code or a forbidden header was found in the
response. Note that only HAProxy >= 2.4 are affected by this issue.

  - A FD leak on reload failures. When the master process is reloaded on a
new config, it will try to connect to the previous process' socket to
retrieve all known listening FDs to be reused by the new listeners. If
listeners were removed, their unused FDs are simply closed. However
there's a catch. In case a socket fails to bind, the master will cancel
its startup and switch to wait mode for a new operation to happen. In
this case it didn't close the possibly remaining FDs that were left
unused.

  - A FD leak of a sockpair upon a failed reload.  When starting HAProxy in
master-worker, the master pre-allocate a struct mworker_proc and do a
socketpair() before the configuration parsing. If the configuration
loading failed, the FD was never closed because they aren't part of
listener, they are not even in the fdtab.

  - Some issues about errors on buffers allocation. First, in the H1
multiplexer. If we failed to send data because we failed to allocate the
H1 output buffer, the H1 stream was erroneously woken up. This led to a
wakeup loop to send more data while it is not possible because there is
no output buffer. Then, in process_stream(), if we failed to allocate
the channel response buffer while a connect or an analysis timeout
occurred, the stream was woken up in loop because its task was requeued
with an expired date. Now an error is reported when this happens and the
stream processing is interrupted.

Note there is a mechanism to deal with errors on buffers allocation.
Unfortunately, since the 1.7, this mechanism is broken. And it is even
worse now with the multiplexers. All this part must be refactored. But
for now, HAProxy may be partially frozen if too many entities are
waiting for a buffer.

  - Some alignment problems that were found when using gcc-11 + RHEL8,
resulting in instant crashes on startup.

  - An issue with multi-line ESMTP response in the mailer code.

  - An issue in the resolvers code with domain names with a trailing dot. The
trailing dot was not ignored as expected and a junk character was added
at the end of the encoded part of the domain name.

The remaining is the usual bunch of fixes and improvements. As usual, people
using the 2.4 branch are encouraged to migrate to this version.

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.4/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.4.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.4.git
   Changelog: http://www.haproxy.org/download/2.4/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Christopher Faulet (6):
  BUG/MINOR: sink: Use the right field in appctx context in release callback
  BUG/MEDIUM: resolvers: Really ignore trailing dot in domain names
  BUG/MEDIUM: htx: Be sure to have a buffer to perform a raw copy of a 
message
  BUG/MEDIUM: mux-h1: Don't wake h1s if mux is blocked on lack of output 
buffer
  BUG/MAJOR: mux-h2: Be sure to always report HTX parsing error to the app 
layer
  BUG/MEDIUM: stream: Abort processing if response buffer allocation fails

Ilya Shipitsin (4):
  BUILD: adopt script/build-ssl.sh for OpenSSL-3.0.0beta2
  CI: github actions: add OpenSSL-3.0.0 builds
  CI: github actions: relax OpenSSL-3.0.0 version comparision
  CI: github actions: update OpenSSL to 3.0.1

Lukas Tribus (1):
  BUG/MINOR: mailers: negotiate SMTP, not ESMTP

William Lallemand (5):
  BUG/MINOR: mworker: fix a FD leak of a sockpair upon a failed reload
  BUILD: fix compilation for OpenSSL-3.0.0-alpha17
  CI: github actions: -Wno-deprecated-declarations with OpenSSL 3.0.0
  CI: github: switch to OpenSSL 3.0.0
  BUG/MINOR: tools: url2sa reads ipv4 too far

Willy Tarreau (10):
  MINOR: sock: move the unused socket cleaning code into its own function
  BUG/MEDIUM: mworker: close unused transferred FDs on load failure
  BUG/MEDIUM: fd: alw

[ANNOUNCE] haproxy-2.5.4

2022-02-25 Thread Christopher Faulet

Hi,

HAProxy 2.5.4 was released on 2022/02/25. It added 8 new commits
after version 2.5.3.

A major issue in the H2 multiplexer was fixed in this release. An error
during the response processing, after the HEADERS frame parsing, led to a
wakeup loop consuming all the CPU because the error was not properly
reported to the upper layer. For instance, this happened if an invalid
header value, an invalid status code or a forbidden header was found in the
response. Note that only HAProxy >= 2.4 are affected by this issue.

In addition, some issues about errors on buffers allocation were
fixed. First, in the H1 multiplexer. If we failed to send data because we
failed to allocate the H1 output buffer, the H1 stream was erroneously woken
up. This led to a wakeup loop to send more data while it is not possible
because there is no output buffer. Then, in process_stream(), if we failed
to allocate the channel response buffer while a connect or an analysis
timeout occurred, the stream was woken up in loop because its task was
requeued with an expired date. Now an error is reported when this happens
and the stream processing is interrupted.

Note there is a mechanism to deal with errors on buffers allocation.
Unfortunately, since the 1.7, this mechanism is broken. And it is even worse
now with the multiplexers. All this part must be refactored. But for now,
HAProxy may be partially Frozen if too many entities are waiting for a
buffer.

Other commits are doc improvements and small fixes here and there. As usual,
people using the 2.5 branch are encouraged to migrate to this version.

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.5/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.5.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.5.git
   Changelog: http://www.haproxy.org/download/2.5/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Christian Ruppert (1):
  DOC: Fix usage/examples of deprecated ACLs

Christopher Faulet (4):
  BUG/MEDIUM: htx: Be sure to have a buffer to perform a raw copy of a 
message
  BUG/MEDIUM: mux-h1: Don't wake h1s if mux is blocked on lack of output 
buffer
  BUG/MAJOR: mux-h2: Be sure to always report HTX parsing error to the app 
layer
  BUG/MEDIUM: stream: Abort processing if response buffer allocation fails

Willy Tarreau (3):
  BUG/MINOR: proxy: preset the error message pointer to NULL in 
parse_new_proxy()
  REGTESTS: fix the race conditions in 40be_2srv_odd_health_checks
  CI: github: enable pool debugging by default

--
Christopher Faulet



Re: DOC issue with http-request return

2022-02-02 Thread Christopher Faulet

Le 2/1/22 à 18:31, Jay Wren (jawren) a écrit :

I think I've found an error in the docs. The entry for `http-request return` shows 
the last component as [ {if | unless}  ] but if I leave that off in 
a backend I get [ALERT] parsing... 'http-request return' expects either 'if' or 
'unless' followed by a condition but found ‘…'.


Hi,

The ACL part in the "http-request return" rule is optional. Otherwise it means 
there is a bug in the parser. But, first of all, you should share the failing 
rule and the HAProxy version you use.


Thanks,
--
Christopher Faulet



Re: invalid request

2022-01-13 Thread Christopher Faulet

Le 1/13/22 à 02:57, Aleksandar Lazic a écrit :


On 12.01.22 21:52, Andrew Anderson wrote:


On Wed, Jan 12, 2022 at 11:58 AM Aleksandar Lazic mailto:al-hapr...@none.at>> wrote:

 Well, looks like you want a forward proxy like squid not a reverse proxy 
like haproxy.


The application being load balanced is a proxy, so http_proxy is not a good fit 
(and as you mention on the
deprecation list), but haproxy as a load balancer is a much better at 
front-ending this environment than
any other solution available.

We upgraded to 2.4 recently, and a Java application that uses these proxy 
servers is what exposed this
issue for us.  Even if we were to use squid, we would still run into this, as I 
would want to ensure that
squid was highly available for the environment, and we would hit the same code 
path when going through
haproxy to connect to squid.

The only option currently available in 2.4 that I am aware of is to setup 
internal-only frontend/backend
paths with accept-invalid-http-request configured on those paths exclusively 
for Java clients to use. This
is effectively how we have worked around this for now:

listen proxy
      bind :8080
      mode http
      option httplog
      server proxy1 192.0.2.1:8080
      server proxy2 192.0.2.2:8080

listen proxy-internal
      bind :8081
      mode http
      option httplog
      option accept-invalid-http-request
      server proxy1 192.0.2.1:8080 track proxy/proxy1
      server proxy2 192.0.2.2:8080 track proxy/proxy2

This is a viable workaround for us in the short term, but this would not be a 
solution that would work for
everyone.  If the uri parser patches I found in the 2.5/2.6 branches are the 
right ones to make haproxy
more permissive on matching the authority with the host in CONNECT requests, 
that will remove the need for
the parallel frontend/backends without validation enabled.  I hope to be able 
to have time to test a 2.4
build with those patches included over the next few days.


By design is HAProxy a reverse proxy to a origin server not to a forwarding 
proxy which is the reason why the
CONNECT method is a invalid method.

Because of that fact I would not use "mode http" for the squid backend/servers 
because of the issues you
described.
Why not "mode tcp" with proxy protocol 
http://www.squid-cache.org/Doc/config/proxy_protocol_access/ if you
need the client ip.


Hi,

CONNECT method is valid and HAproxy supports it. It doesn't handle it by itself 
however. It forwards it to the server.


Since the 2.4, a scheme-based normalization is performed on the requests after 
the parsing but with some limitation. The target-uri must be in absolute-form. 
Thus CONNECT requests are not normalized. But as said, the normalization is 
performed after the H1 parsing. Idea is to sanitize the request before sending 
it to the server and to simplify host-based routing configurations.


However, during H1 parsing, the authority found in the URI is validated against 
the Host header. At this stage, both must be identical. Otherwise an error is 
reported. "accept-invalid-http-request" option is a valid workaround in this case.


So now the question is to know if a pre-normalization must be performed during 
H1 parsing or not (in addition to the one performed during the request 
analysis). And if it should be extended to CONNECT requests. In practice, it is 
only an issue for CONNECT requests because the absolue-form is not the common 
form for URIs in H1.


--
Christopher Faulet



Re: [PATCH] BUG/MEDIUM: server: avoid changing healthcheck ctx with set server ssl

2022-01-12 Thread Christopher Faulet

Le 1/11/22 à 22:47, William Dauchy a écrit :

Agree. But, if possible, a warning may be added in the documentation to warn
about implicit changes.


 From the discussion, I would be tempted to say the opposite, as I feel
like keeping implicit things for this command is worse.


I don't know what is the expected behavior on the stable releases for users. The 
actual state is buggy because health-check are only updated when ssl is 
disabled. When SSL is enabled on a server, there is no implicit change on the 
check. when SSL is disabled, there is an implicit change. So we must first state 
for the expected behavior on stable releases. But, keep in mind there is no way 
to dynamically enable/disable SSL on health-checks. So if a user configures the 
SSL on its server but disables it on startup, when he enables ssl, he probably 
want to do so on the health-check, explicitly or not.




as mentioned above I am not sure I am aligned here; I would rather
remove the side effect of changing s->check.


Honestly, I've misread you patch and kept in mind you alternative solution... 
But as said, if there is no implicit change (and I'm fine with this solution), a 
new command or an option to current ones must be introduced to be able to change 
SSL setting for health-check too. Otherwise, it will be unusable.



Note I don't know if all this stuff works properly with the server-state file...


I am always scared to look at the server state functionality...


I truly understand :( ...

--
Christopher Faulet



Re: [PATCH] BUG/MEDIUM: server: avoid changing healthcheck ctx with set server ssl

2022-01-11 Thread Christopher Faulet

Le 1/10/22 à 23:19, Willy Tarreau a écrit :

At this point I'm starting to think that we should probably avoid as
much as possible to use implicit settings for whatever is dynamic.
Originally a lot of settings were implicit because we don't want to
have huge config lines to enforce lots of obvious settings. On the CLI
it's less of a problem and for example if "ssl" only deals with the
traffic without manipulating the checks, I'm personally not shocked,
because these are normally sent by bots and we don't care if they
have to set a few more settings to avoid multiple implicit changes
that may not always be desired. This is where the CLI (or any future
API) might differ a bit from the config, and an agent writing some
config might have to explicitly state certain things like "no-check-ssl"
for example to stay safe and avoid such implicit dependencies.



I agree with Willy on this point. Especially because, depending the order of 
commands, the result can be different because of implicit changes and it is 
pretty hard to predict how it will behave in all cases.


For instance, to fix William's bug about "set server ssl" command, in a way or 
another, we must stop to dynamically update the health-check if its port or its 
address is explicitly specified. With this change, the result of following set 
of commands will be different:


  $ set server BK/SRV ssl on
  $ set server BK/SRV check-port XXX

==> SSL is enabled for the server and the health-check

  $ set server BK/SRV check-port XXX
  $ set server BK/SRV ssl on

==> because the check's port was updated first, the SSL is only enabled for the 
server.



Note that such a brainstorming doesn't apply to your fix and should
not hold it from being merged in any way, I'm just speaking in the
general sense, as I don't feel comfortable with keep all these special
cases in a newly introduced API, that are only there for historical
reasons.


Agree. But, if possible, a warning may be added in the documentation to warn 
about implicit changes.


About the fix, I checked the code, and, at first glance, there is no reason to 
change "srv->check.use_ssl" value when the health-check uses the server 
settings. Thus, we may fix "set server ssl" command this way:


diff --git a/src/check.c b/src/check.c
index f0ae81504..8cf8a1c5b 100644
--- a/src/check.c
+++ b/src/check.c
@@ -1565,10 +1565,8 @@ int init_srv_check(struct server *srv)
 * default, unless one is specified.
 */
if (!srv->check.port && !is_addr(>check.addr)) {
-   if (!srv->check.use_ssl && srv->use_ssl != -1) {
-   srv->check.use_ssl = srv->use_ssl;
-   srv->check.xprt= srv->xprt;
-   }
+   if (!srv->check.use_ssl && srv->use_ssl != -1)
+   srv->check.xprt = srv->xprt;
else if (srv->check.use_ssl == 1)
srv->check.xprt = xprt_get(XPRT_SSL);
srv->check.send_proxy |= (srv->pp_opts);

diff --git a/src/server.c b/src/server.c
index 2061153bc..a18836a71 100644
--- a/src/server.c
+++ b/src/server.c
@@ -2113,10 +2113,11 @@ void srv_set_ssl(struct server *s, int use_ssl)
return;

s->use_ssl = use_ssl;
-   if (s->use_ssl)
-   s->xprt = xprt_get(XPRT_SSL);
-   else
-   s->xprt = s->check.xprt = s->agent.xprt = xprt_get(XPRT_RAW);
+   s->xprt = (s->use_ssl == 1) ? xprt_get(XPRT_SSL) : xprt_get(XPRT_RAW);
+
+   if ((s->check.state & CHK_ST_CONFIGURED) && !s->check.use_ssl &&
+   !s->check.port && !is_addr(>check.addr))
+   s->check.xprt = s->xprt;
 }

 #endif /* USE_OPENSSL */

This may be done with the following 3 steps:

  * First, stop to change "srv->check.use_ssl" value

  * Then, stop to change the agent settings in srv_set_ssl() because there is 
no ssl support for agent-check.


  * Finally, fix the bug identified by William, adding the according 
documentation.

Note I don't know if all this stuff works properly with the server-state file...

--
Christopher Faulet



[ANNOUNCE] haproxy-2.5.1

2022-01-11 Thread Christopher Faulet
didn't work anymore).

  - David Carlier fixed the build with FreeBSD 14, which changes the cpuset
API to better match Linux's.

  - a small improvement, in order to help users provide exploitable cores,
there's now a new command-line option "-dL" which dumps the dynamic
libraries that were detected at run time just before forking. This
possibly includes dependencies from Lua or various other libs that do
not always appear in "ldd". Typically libgcc_s is listed. The output
format allows to pipe that to tar to produce an archive of all
executable code that apparently tends to open well with a core,
irrelevant to the distros in use. Since it eases bug reports, we've
decided to backport it.

  - another build issue, this time with clang on i386. It tries to make use
of the CMPXCHG8B instruction to perform 64-bit atomics but incorrectly
expects the operands to be 64-bit aligned while neither the ABI nor the
instruction have this requirement. So basically it complains about the
code it produces itself. The analysis showed that working around this
would require tens to hundreds of isolated hacks and that the least
dirty solution is to disable the warning. Firefox faced the same issue 3
years ago and adopted the same work around. I guess nobody's interested
anymore in i386 for anyone to expect a fix there anyway.

  - a workaround for a possibly slow malloc_trim() in modern libcs upon
reload when using many threads, that could be slow enough to panic
the old process.

  - a few other build fixes and doc fixes.

  - "show version" command was added to display the current process version.

  - "capture" action is now supported in http-after-response rulesets.

  - Empty lines were removed from "show ssl ocsp-response" command output.

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.5/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.5.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.5.git
   Changelog: http://www.haproxy.org/download/2.5/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Bertrand Jacquin (1):
  BUG/MINOR: lua: remove loop initial declarations

Christopher Faulet (15):
  BUG/MINOR: cache: Fix loop on cache entries in "show cache"
  BUG/MEDIUM: cli: Properly set stream analyzers to process one command at 
a time
  BUG/MINOR: server: Don't rely on last default-server to init server SSL 
context
  BUG/MEDIUM: resolvers: Detach query item on response error
  BUG/MEDIUM: h1: Properly reset h1m flags when headers parsing is restarted
  MINOR: mux-h1: Improve H1 traces by adding info about http parsers
  BUILD: bug: Fix error when compiling with -DDEBUG_STRICT_NOCRASH
  MINOR: http-rules: Add capture action to http-after-response ruleset
  BUG/MINOR: cli/server: Don't crash when a server is added with a custom id
  DOC: spoe: Clarify use of the event directive in spoe-message section
  DOC: config: Specify %Ta is only available in HTTP mode
  BUG/MEDIUM: mux-h1: Fix splicing by properly detecting end of message
  BUG/MINOR: mux-h1: Fix splicing for messages with unknown length
  BUG/MEDIUM: http-ana: Preserve response's FLT_END analyser on L7 retry
  BUG/MAJOR: mux-h1: Don't decrement .curr_len for unsent data

Daniel Jakots (1):
  BUILD: ssl: unbreak the build with newer libressl

David CARLIER (3):
  MINOR: cpuset: switch to sched_setaffinity for FreeBSD 14 and above.
  BUILD/MINOR: cpuset FreeBSD 14 build fix.
  BUILD: cpuset: fix build issue on macos introduced by previous change

David Carlier (1):
  BUILD/MINOR: tools: solaris build fix on dladdr.

Emeric Brun (1):
  BUG/MAJOR: segfault using multiple log forward sections.

Ilya Shipitsin (3):
  CI: Github Actions: do not show VTest failures if build failed
  CI: github actions: update OpenSSL to 3.0.1
  CI: github actions: clean default step conditions

Lukas Tribus (2):
  DOC: config: retry-on list is space-delimited
  DOC: config: fix error-log-format example

Miroslav Zagorac (1):
  BUILD: opentracing: display warning in case of using OT_USE_VARS at 
compile time

Remi Tricot-Le Breton (3):
  BUG/MINOR: vars: Fix the set-var and unset-var converters
  MINOR: ssl: Remove empty lines from "show ssl ocsp-response" output
  BUG/MINOR: ssl: Store client SNI in SSL context in case of ClientHello 
error

Thierry Fournier (1):
  D

Re: [PATCH] BUG/MEDIUM: server: avoid changing healthcheck ctx with set server ssl

2022-01-11 Thread Christopher Faulet

Le 1/10/22 à 23:19, Willy Tarreau a écrit :
w options were still configurable on the CLI by then.



"check-ssl" has been available for a long time, so that's not the
reason behind it, but I guess you were referring to something else. I
suspect I did a dumb copy/paste from the new_server function and
probably never thought was possibly wrong as my previous production
never had any check using tls.


Maybe but then I don't remember about all the detailed rules in place
that indicate when check-ssl *has* to be used. I'll have to read the
doc.



For a health-check, if no port or address is specified and no transport layer is 
forced, then the transport layer used by the check is the same than for the 
production traffic.


So, the same must be done for dynamic changes. But it is not so simple because, 
when the check inherits from the server settings, "srv->check.use_ssl" is also 
changed. I don't remember why this field is updated. But this may prevent any 
dynamic change on healtcheck. I must read the code to be sure.


--
Christopher Faulet



[ANNOUNCE] haproxy-2.4.12

2022-01-11 Thread Christopher Faulet

Hi,

HAProxy 2.4.12 was released on 2022/01/11. It added 2 new commits
after version 2.4.11.

The 2.4.11 introduced a major regression into the H1 multiplexer. The bug
affects all HTTP messages announcing a content length from the time there
are some contentions on the output buffer. The result is that an internal
error is erroneously reported when HAproxy tries to emit the last block of
payload and the message is unexpectedly truncated.

In addition, William fixed a bug in the master-worker when the master is
executed in wait mode. In this case, the master must never try to to get the
listeners FD from the previous process using _getsocks on the stat
socket. Otherwise, if a reload fails, the master exists with a EXIT_FAILURE
status, killing all the workers.

A 2.5.1 will be emitted very soon with these fixes (and many others).
However the 2.5.0 is not affected by the first issue. The 2.4.12 was emitted
first to avoid any trouble for anyone who would like to update or who have
already upgraded.

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.4/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.4.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.4.git
   Changelog: http://www.haproxy.org/download/2.4/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Christopher Faulet (1):
  BUG/MAJOR: mux-h1: Don't decrement .curr_len for unsent data

William Lallemand (1):
  BUG/MEDIUM: mworker: don't use _getsocks in wait mode

--
Christopher Faulet



Re: Issue with uploads and HAProxy 2.4.11

2022-01-10 Thread Christopher Faulet

Le 1/10/22 à 14:46, Christian Ruppert a écrit :

On 2022-01-10 13:33, Sander Klein wrote:

Hi,

I've upgraded to HAProxy 2.4.11 and now I seem to have a problem with
bigger file uploads (>70MB).

When uploading a file I get a 500 back from HAProxy, and if I retry it
immediately it most of the time succeeds. Downgrading to 2.4.10 fixes
the issue. The log I get is:

Jan 10 12:09:45 [redacted] haproxy[21823]: 2001:67c:[redacted]
[10/Jan/2022:12:09:20.543] [redacted]~ [redacted]/[redacted]
11198/0/0/-1/25137 500 1991 - - IH-- 957/282/0/0/0 0/0
{[redacted].[redacted].com|Mozilla/5.0
(Mac|80349066|https://[redacted].[redacted].com/upload} {} “POST
https://[redacted].[redacted].com/upload/process?projectId=3431=149
HTTP/2.0”

The frontend is HTTP/2.0 and the backend is NGINX talking HTTP/1.1
(non-TLS).

The config is quite large, but I think it boils down to:

---
frontend [redacted]
bind [redactes]]:80 transparent
bind 2001:67c:[redacted]:80 transparent

bind [redacted]:443 transparent ssl crt /etc/haproxy/ssl/[redacted]
strict-sni alpn h2,http/1.1 npn h2,http/1.1
bind 2001:67c:[redacted]:443 transparent ssl crt
/etc/haproxy/ssl/[redacted] strict-sni alpn h2,http/1.1 npn
h2,http/1.1

mode http
maxconn 16384

option httplog
option dontlog-normal
option http-ignore-probes
option forwardfor
option http-buffer-request

capture request header Host len 64
capture request header User-Agent   len 16
capture request header Content-Length   len 10
capture request header Referer  len 256
capture response header Content-Length  len 10


acl [some ACLs here]
 acl [some ACLs here]

http-request deny if [an ACL]
 http-request deny if [another ACL]

use_backend [failing-backend]   if [ACL]
use_backend 
%[req.hdr(host),lower,regsub(^www\.,,i),map(/etc/haproxy/map.d/file.map,yes-backend)]
default_backend another-backend

backend failing-backend
fullconn256
modehttp

balance roundrobin

option abortonclose
option prefer-last-server
option redispatch
option httpchk GET /check-thingy HTTP/1.0
http-check expect status 200

default-server weight 100 maxconn 20 check inter 2s rise 3 fall 3
slowstart 5m agent-check agent-port 8081 agent-inter 20s
server server1 [redacted]:80 cookie cookie1
server server2 [redacted]:80 cookie cookie2

# Sorry Server
server outage 127.0.0.1:80 backup

retries 1
---

If any more info is needed, please let me know.

Regards,

Sander Klein


Hi Sander,

this might be also related to
https://github.com/haproxy/haproxy/issues/1510



Thanks Christian. Indeed, it is most probably the same issue. The proposed patch 
is in attachment. The 2.6 and 2.5 are also affected.


--
Christopher FauletFrom 2984ccfc1f5c4ca1157f7a498dcdc7de2f7ba934 Mon Sep 17 00:00:00 2001
From: Christopher Faulet 
Date: Mon, 10 Jan 2022 17:27:51 +0100
Subject: [PATCH] BUG/MAJOR: mux-h1: Don't decrement .curr_len for unsent data

A regression was introduced by commit 140f1a585 ("BUG/MEDIUM: mux-h1: Fix
splicing by properly detecting end of message"). To detect end of the
outgoing message, when the content-length is announced, we count amount of
data already sent. But only data really sent must be counted.

If the output buffer is full, we can fail to send data (fully or
partially). In this case, we must take care to only count sent
data. Otherwise we may think too much data were sent and an internal error
may be erroneously reported.

This patch should solve issue #1511. It must be backported as far as 2.4.
---
 src/mux_h1.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/src/mux_h1.c b/src/mux_h1.c
index f9a6120fe4..c2dc80834d 100644
--- a/src/mux_h1.c
+++ b/src/mux_h1.c
@@ -2180,7 +2180,6 @@ static size_t h1_process_output(struct h1c *h1c, struct buffer *buf, size_t coun
 H1_EV_TX_DATA|H1_EV_STRM_ERR|H1_EV_H1C_ERR|H1_EV_H1S_ERR, h1c->conn, h1s);
 			goto error;
 		}
-		h1m->curr_len -= vlen;
 	}
 	if ((h1m->flags & H1_MF_RESP) && (h1s->flags & H1S_F_BODYLESS_RESP)) {
 		TRACE_PROTO("Skip data for bodyless response", H1_EV_TX_DATA|H1_EV_TX_BODY, h1c->conn, h1s, chn_htx);
@@ -2226,6 +2225,8 @@ static size_t h1_process_output(struct h1c *h1c, struct buffer *buf, size_t coun
 		H1_EV_TX_DATA|H1_EV_TX_BODY, h1c->conn, h1s, 0, (size_t[]){v.len});
 
 			  skip_data:
+if (h1m->state == H1_MSG_DATA && (h1m->flags & H1_MF_CLEN))
+	h1m->curr_len -= vlen;
 if (last_data)
 	goto done;
 break;
-- 
2.33.1



Re: HAP 2.3.16 A bogus STREAM [0x559faa07b4f0] at "cache store filter"

2022-01-04 Thread Christopher Faulet

Le 1/4/22 à 10:26, Aleksandar Lazic a écrit :


On 04.01.22 10:16, Christopher Faulet wrote:

Le 12/25/21 à 23:59, Aleksandar Lazic a écrit :


Hi.

as the message tell us that we should report this to the developers I do so :-)


```
Dec 24 01:10:31 lb1 haproxy[20008]: A bogus STREAM [0x559faa07b4f0] is spinning 
at 204371 calls per second
and refuses to die, aborting now!
Please report this error to developers
[strm=0x559faa07b4f0,12390e src=:::79.183.184.235 fe=https-in be=be_api 
dst=api_main2
    txn=0x559faab233e0,44000 txn.req=MSG_DONE,d txn.rsp=MSG_RPBEFORE,0 
rqf=48c4e068 rqa=4
    rpf=a000a860 rpa=0 sif=CLO,2c8002 sib=CLO,1280112 af=(nil),0 
csf=0x559faa07ba10,1059a0
    ab=(nil),0 csb=0x559faad7dcf0,1a0
    
cof=0x7f224212e5d0,80003300:H2(0x559faa7d7b00)/SSL(0x7f22424fc7a0)/tcpv6(2162)
    
cob=0x7f2240f79fe0,8982300:H1(0x559faa0ab840)/SSL(0x7f2263517770)/tcpv4(1490)
    filters={0x559faa29c520="cache store filter"}]



Hi Alex,

I think I found the issue. I'm unable to reproduce the spinning loop but I can 
freeze infinitely a stream.
It is probably just a matter of timing. On my side, it is related to L7 
retries. Could you confirm you have
a "retry-on" parameter in your configuration ?


Yes I can confirm.

```
defaults http
log global
mode http
retry-on all-retryable-errors
option forwardfor
option redispatch
option http-ignore-probes
option httplog
option dontlognull
option ssl-hello-chk
option log-health-checks
option socket-stats
timeout connect 5s
timeout client  50s
timeout server  50s
http-reuse safe
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
...
```



Thanks Alex, I pushed a fix. It will be backported as far as the 2.0 ASAP.


--
Christopher Faulet



Re: HAP 2.3.16 A bogus STREAM [0x559faa07b4f0] at "cache store filter"

2022-01-04 Thread Christopher Faulet

Le 12/25/21 à 23:59, Aleksandar Lazic a écrit :


Hi.

as the message tell us that we should report this to the developers I do so :-)


```
Dec 24 01:10:31 lb1 haproxy[20008]: A bogus STREAM [0x559faa07b4f0] is spinning 
at 204371 calls per second
and refuses to die, aborting now!
Please report this error to developers
[strm=0x559faa07b4f0,12390e src=:::79.183.184.235 fe=https-in be=be_api 
dst=api_main2
   txn=0x559faab233e0,44000 txn.req=MSG_DONE,d txn.rsp=MSG_RPBEFORE,0 
rqf=48c4e068 rqa=4
   rpf=a000a860 rpa=0 sif=CLO,2c8002 sib=CLO,1280112 af=(nil),0 
csf=0x559faa07ba10,1059a0
   ab=(nil),0 csb=0x559faad7dcf0,1a0
   
cof=0x7f224212e5d0,80003300:H2(0x559faa7d7b00)/SSL(0x7f22424fc7a0)/tcpv6(2162)
   cob=0x7f2240f79fe0,8982300:H1(0x559faa0ab840)/SSL(0x7f2263517770)/tcpv4(1490)
   filters={0x559faa29c520="cache store filter"}]



Hi Alex,

I think I found the issue. I'm unable to reproduce the spinning loop but I can 
freeze infinitely a stream. It is probably just a matter of timing. On my side, 
it is related to L7 retries. Could you confirm you have a "retry-on" parameter 
in your configuration ?


Thanks !
--
Christopher Faulet



Re: Is it expected that "capture response" does not get headers when "http-request return" is used

2021-12-08 Thread Christopher Faulet

Le 12/6/21 à 08:25, Christopher Faulet a écrit :

Le 12/4/21 à 13:25, Aleksandar Lazic a écrit :


Hi.

I try to capture the response header "dst_conn" from "http-request return" but 
in %hs isn't the value.

```
podman logs -f haproxy-dest
[NOTICE]   (1) : New worker #1 (3) forked
<6>[04/Dec/2021:12:14:34.437] 200 58 - - LR-- {} "GET / HTTP/1.1"
<6>[04/Dec/2021:12:14:34.437] 200 58 - - LR-- {} "GET / HTTP/1.1"
<6>[04/Dec/2021:12:14:34.438] 200 58 - - LR-- {} "GET / HTTP/1.1"

```

I haven't seen any "capture" in "http-after-response".
The question is also makes sense to have a capture after "http-request return" 
as in the documenation is
written that return stops the evaluation of any other rules also from capture?

http://cbonte.github.io/haproxy-dconv/2.4/configuration.html#http-response%20return
"This stops the evaluation of the rules and immediately returns a response."


Hi Alex,

Unfortunately, it is indeed not possible for now. First, the captures via
"capture request" and "capture response" directives are performed very early,
and on received messages only. Thus it is not possible to capture info from
generated responses at this stage. However, it is probably possible to add a
"capture" action to the "http-afer-response" ruleset. This would able to you to
capture your header with the following config:

 declare capture response len 4
 http-after-response capture hdr(dst_conn) id 0

At first glance it seems trivial. I will check that.


Hi,

I added it to 2.6-DEV. The patch is small enough to be backported to 2.5.

--
Christopher Faulet



Re: Is it expected that "capture response" does not get headers when "http-request return" is used

2021-12-05 Thread Christopher Faulet

Le 12/4/21 à 13:25, Aleksandar Lazic a écrit :


Hi.

I try to capture the response header "dst_conn" from "http-request return" but 
in %hs isn't the value.

```
podman logs -f haproxy-dest
[NOTICE]   (1) : New worker #1 (3) forked
<6>[04/Dec/2021:12:14:34.437] 200 58 - - LR-- {} "GET / HTTP/1.1"
<6>[04/Dec/2021:12:14:34.437] 200 58 - - LR-- {} "GET / HTTP/1.1"
<6>[04/Dec/2021:12:14:34.438] 200 58 - - LR-- {} "GET / HTTP/1.1"

```

I haven't seen any "capture" in "http-after-response".
The question is also makes sense to have a capture after "http-request return" 
as in the documenation is
written that return stops the evaluation of any other rules also from capture?

http://cbonte.github.io/haproxy-dconv/2.4/configuration.html#http-response%20return
"This stops the evaluation of the rules and immediately returns a response."


Hi Alex,

Unfortunately, it is indeed not possible for now. First, the captures via 
"capture request" and "capture response" directives are performed very early, 
and on received messages only. Thus it is not possible to capture info from 
generated responses at this stage. However, it is probably possible to add a 
"capture" action to the "http-afer-response" ruleset. This would able to you to 
capture your header with the following config:


   declare capture response len 4
   http-after-response capture hdr(dst_conn) id 0

At first glance it seems trivial. I will check that.

--
Christopher Faulet



Re: [PATCH] BUG/MEDIUM: sample: Fix memory leak in sample_conv_jwt_member_query

2021-12-03 Thread Christopher Faulet

Le 12/1/21 à 23:04, Tim Duesterhus a écrit :

The function leaked one full buffer per invocation. Fix this by simply removing
the call to alloc_trash_chunk(), the static chunk from get_trash_chunk() is
sufficient.

This bug was introduced in 0a72f5ee7c2a61bdb379436461269315c776b50a, which is
2.5-dev10. This fix needs to be backported to 2.5+.
---
  src/sample.c | 4 
  1 file changed, 4 deletions(-)

diff --git a/src/sample.c b/src/sample.c
index 5abf4712a..63816be5d 100644
--- a/src/sample.c
+++ b/src/sample.c
@@ -3584,10 +3584,6 @@ static int sample_conv_jwt_member_query(const struct arg 
*args, struct sample *s
if (item_num < member + 1)
goto end;
  
-	decoded_header = alloc_trash_chunk();

-   if (!decoded_header)
-   goto end;
-
ret = base64urldec(items[member].start, items[member].length,
   decoded_header->area, decoded_header->size);
if (ret == -1)



Thanks, merged now !

--
Christopher Faulet



Re: default-server behavior different in 2.2 vs. 2.4?

2021-12-01 Thread Christopher Faulet

Le 11/29/21 à 09:56, Christian Ruppert a écrit :

Hey,

we have something like:
  server maint 192.168.70.98:80 weight 1 backup non-stick

  default-server check maxconn 100 ssl verify required sni
str(somestr) ca-file /etc/ssl/certs/ca-certificates.crt observe layer7

  server s200010 192.168.200.10:8443 cookie somecookie weight 100
check addr 127.0.0.1 port 62041 check inter 1 fall 2 rise 2
  ...


In 2.2 (2.2.17) it is totally valid and the stats say the "maint" is
still "no check". In 2.4.9 the config verify fails. In 2.2 it seems to
only affects server's below default-server.
[ALERT](23456) : Proxy '...', server 'maint'
[/etc/haproxy/haproxy.cfg:104096] verify is enabled by default but no CA
file specified. If you're running on a LAN where you're certain to trust
the server's certificate, please set an explicit 'verify none' statement
on the 'server' line, or use 'ssl-server-verify none' in the global
section to disable server-side verifications by default.

We're using templates and share the maint server with several hundred
other listeners/backends. Only 5 are using a config like this here, with
SSL and verify.

Another problem here seems: "verify is enabled by default but no CA file
specified" while in fact it is?
Is this intended?



I pushed a patch (https://github.com/haproxy/haproxy/commit/4ab26796) that 
should fix this issue. It will be backported ASAP. In the mean time, the patch 
for the 2.4 is attached to this mail.


--
Christopher FauletFrom 7f689d5265a973d66b702e57ba02e866c1db400c Mon Sep 17 00:00:00 2001
From: Christopher Faulet 
Date: Wed, 1 Dec 2021 09:50:41 +0100
Subject: [PATCH] BUG/MINOR: server: Don't rely on last default-server to init
 server SSL context

During post-parsing stage, the SSL context of a server is initialized if SSL
is configured on the server or its default-server. It is required to be able
to enable SSL at runtime. However a regression was introduced, because the
last parsed default-server is used. But it is not necessarily the
default-server line used to configure the server. This may lead to
erroneously initialize the SSL context for a server without SSL parameter or
the skip it while it should be done.

The problem is the default-server used to configure a server is not saved
during configuration parsing. So, the information is lost during the
post-parsing. To fix the bug, the SRV_F_DEFSRV_USE_SSL flag is
introduced. It is used to know when a server was initialized with a
default-server using SSL.

For the record, the commit f63704488e ("MEDIUM: cli/ssl: configure ssl on
server at runtime") has introduced the bug.

This patch must be backported as far as 2.4.
---
 include/haproxy/server-t.h   |  3 +++
 reg-tests/server/cli_set_ssl.vtc | 20 +---
 src/cfgparse.c   |  4 ++--
 src/server.c |  4 
 4 files changed, 22 insertions(+), 9 deletions(-)

diff --git a/include/haproxy/server-t.h b/include/haproxy/server-t.h
index 4291953888..402ea79f0a 100644
--- a/include/haproxy/server-t.h
+++ b/include/haproxy/server-t.h
@@ -149,6 +149,9 @@ enum srv_initaddr {
 #define SRV_F_SOCKS4_PROXY 0x0400/* this server uses SOCKS4 proxy */
 #define SRV_F_NO_RESOLUTION 0x0800   /* disable runtime DNS resolution on this server */
 #define SRV_F_DYNAMIC  0x1000/* dynamic server instantiated at runtime */
+/* 0x2000 unused */
+#define SRV_F_DEFSRV_USE_SSL 0x4000  /* default-server uses SSL */
+
 
 /* configured server options for send-proxy (server->pp_opts) */
 #define SRV_PP_V1   0x0001   /* proxy protocol version 1 */
diff --git a/reg-tests/server/cli_set_ssl.vtc b/reg-tests/server/cli_set_ssl.vtc
index 638debea0d..f97bf4de3a 100644
--- a/reg-tests/server/cli_set_ssl.vtc
+++ b/reg-tests/server/cli_set_ssl.vtc
@@ -27,8 +27,9 @@ haproxy h1 -conf {
 default_backend test0
 
 backend test0
-default-server ssl
 server www0 ${s1_addr}:${s1_port} no-ssl
+default-server ssl
+server www1 ${s1_addr}:${s1_port} no-ssl
 
 backend test1
 server www0 ${s1_addr}:${s1_port} no-ssl
@@ -37,17 +38,22 @@ haproxy h1 -conf {
 haproxy h1 -cli {
 # supported case
 send "show servers state test0"
-expect ~ "test0 1 www0 ${s1_addr} .* - ${s1_port} - -1"
-send "set server test0/www0 ssl on"
+expect ~ "test0 2 www1 ${s1_addr} .* - ${s1_port} - -1"
+send "set server test0/www1 ssl on"
 expect ~ "server ssl setting updated"
 send "show servers state test0"
-expect ~ "test0 1 www0 ${s1_addr} .* - ${s1_port} - 1"
-send "set server test0/www0 ssl off"
+expect ~ "test0 2 www1 ${s1_addr} .* - ${s1_port} - 1"
+send "set server test0/www1 ssl off"
 expect ~ "server ssl setting updated"
 send "show

Re: default-server behavior different in 2.2 vs. 2.4?

2021-11-29 Thread Christopher Faulet

Le 11/29/21 à 09:56, Christian Ruppert a écrit :

Hey,

we have something like:
  server maint 192.168.70.98:80 weight 1 backup non-stick

  default-server check maxconn 100 ssl verify required sni
str(somestr) ca-file /etc/ssl/certs/ca-certificates.crt observe layer7

  server s200010 192.168.200.10:8443 cookie somecookie weight 100
check addr 127.0.0.1 port 62041 check inter 1 fall 2 rise 2
  ...


In 2.2 (2.2.17) it is totally valid and the stats say the "maint" is
still "no check". In 2.4.9 the config verify fails. In 2.2 it seems to
only affects server's below default-server.
[ALERT](23456) : Proxy '...', server 'maint'
[/etc/haproxy/haproxy.cfg:104096] verify is enabled by default but no CA
file specified. If you're running on a LAN where you're certain to trust
the server's certificate, please set an explicit 'verify none' statement
on the 'server' line, or use 'ssl-server-verify none' in the global
section to disable server-side verifications by default.

We're using templates and share the maint server with several hundred
other listeners/backends. Only 5 are using a config like this here, with
SSL and verify.

Another problem here seems: "verify is enabled by default but no CA file
specified" while in fact it is?
Is this intended?




Indeed. The change was introduced by the commit f63704488e ("MEDIUM: cli/ssl: 
configure ssl on server at runtime"). During the post-parsing stage, when the 
configuration is validated, we rely on the last "default-server" line parsed in 
the backend section to finalize the SSL configuration of servers in this 
backend. It is of course a bug. We may probably rely on a flag instead. I'll 
investigate.


Thanks !
--
Christopher Faulet



[ANNOUNCE] haproxy-2.2.19

2021-11-29 Thread Christopher Faulet



Hi,

HAProxy 2.2.19 was released on 2021/11/29. It added 21 new commits
after version 2.2.18.

Main changes in this release are about the shutdowns management in the H1
and H2 multiplexers to be able to perform a clean shutdown when delayed.
This fixed an issue with caching of TLS sessions. Indeed, with the migration
to the muxes in 1.9-2.0, we've lost the clean shutdown at the end of
connection that's also used to commit the TLS session cache entry. That
allowed too to fix hidden bugs. The shutdowns at the conn-stream level are
now idempotent.

In addition to the above changes, the H2 multiplexer now drains data on
shutdown to be sure to send GOAWAY frame. Among other things, this fixed
issues with h2spec tests in the CI. Finally, an old and incomplete fix for
H2 partial frames causing some high CPU usages in h2_io_cb() on some rare
occasions was fixed.

Some users reported occasional crashes in the cache (#1284 and #1451). We
finally had an explanation (a missing break). This was fixed.

http-response rulesets evaluation was not aligned with what is said in the
documentation. It was possible to inhibit the frontend rules evaluation with
an "allow" rule in the backend section while it should instead only stop
backend rules evaluation. This bug exists since the beginning and only
concerns the "allow" rule. It was fixed and http-after-response rulesets
evaluation was also fixed in the same way.

William fixed some bugs in the SSL part. First, outgoing TLS connections
involving SNI couldn't be resumed in TLS 1.3 because the call to
SSL_get_servername() on a resumed connection doesn't return the previous SNI
with TLS 1.3. Then, the wrong error was reported during SSL handshake when a
non-matching SNI was found with the strict-sni option enabled because the
clientHello callback was returning with a success code. An "handshake
failure" was reported instead of "unrecognized name". As a side effect of
this bug, the connections was accepted in case of TLS resume.

A bug in the validity checks for sample fetch functions was fixed. The
checks were only applied to the frontend capability of a proxy. This means
that using a small set of sample fetch functions (like "be_name()") in
proxies that are both a frontend and a backend ("listen" or "defaults")
would lead to a config error while it is technically valid. This problem has
always been there and never reported.

A bug with the "program" post-parser was fixed. It could be called with an
empty programs list in case of a config parsing error on reload after
another error, and could crash.

In addition, the usual bunch of some of small fixes and cleanups.

The next 2.0 release will be emitted at the end of the week.

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.2/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.2.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.2.git
   Changelog: http://www.haproxy.org/download/2.2/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Christopher Faulet (11):
  BUG/MEDIUM: stream-int: Block reads if channel cannot receive more data
  BUG/MEDIUM: sample: Cumulate frontend and backend sample validity flags
  BUG/MEDIUM: mux-h1: Fix H1C_F_ST_SILENT_SHUT value
  DOC: config: Fix typo in ssl_fc_unique_id description
  BUG/MINOR: http-ana: Apply stop to the current section for http-response 
rules
  Revert "BUG/MINOR: http-ana: Don't eval front after-response rules if stopped 
on back"
  DOC: lua: Be explicit with the Reply object limits
  BUG/MEDIUM: conn-stream: Don't reset CS flags on close
  BUG/MINOR: mux-h2: Fix H2_CF_DEM_SHORT_READ value
  BUG/MINOR: stick-table/cli: Check for invalid ipv6 key
  CLEANUP: ssl: Release cached SSL sessions on deinit

William Lallemand (3):
  BUG/MINOR: mworker: doesn't launch the program postparser
  BUG/MEDIUM: ssl: backend TLS resumption with sni and TLSv1.3
  BUG/MEDIUM: ssl: abort with the correct SSL error when SNI not found

Willy Tarreau (7):
  BUG/MINOR: sample: fix backend direction flags consecutive to last fix
  BUG/MEDIUM: connection: make cs_shutr/cs_shutw//cs_close() idempotent
  MINOR: connection: add a new CO_FL_WANT_DRAIN flag to force drain on close
  MINOR: mux-h2: perform a full cycle shutdown+drain on close
  BUG/MEDIUM: mux-h2: always process a pending shut read
  BUG/MEDIUM: shctx: leave the block allocator when enough blocks are found
  BUG/MINOR: shctx: do not look for available blocks when the first one is 
enough

--
Christopher Faulet



Re: [PATCH 1/1] BUG/MINOR: lua: remove loop initial declarations

2021-11-25 Thread Christopher Faulet

Le 11/24/21 à 22:16, Bertrand Jacquin a écrit :

HAProxy is documented to support gcc >= 3.4 as per INSTALL file, however
hlua.c makes use of c11 only loop initial declarations leading to build
failure when using gcc-4.9.4:

   x86_64-unknown-linux-gnu-gcc -Iinclude  -Wchar-subscripts -Wcomment -Wformat -Winit-self -Wmain 
-Wmissing-braces -Wno-pragmas -Wparentheses -Wreturn-type -Wsequence-point -Wstrict-aliasing 
-Wswitch -Wtrigraphs -Wuninitialized -Wunknown-pragmas -Wunused-label -Wunused-variable 
-Wunused-value -Wpointer-sign -Wimplicit -pthread -fdiagnostics-color=auto -D_LARGEFILE_SOURCE 
-D_FILE_OFFSET_BITS=64 -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -O3 -msse -mfpmath=sse 
-march=core2 -g -fPIC -g -Wall -Wextra -Wundef -Wdeclaration-after-statement -fwrapv 
-Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered 
-Wno-missing-field-initializers -Wtype-limits  -DUSE_EPOLL  -DUSE_NETFILTER   -DUSE_PCRE2 
-DUSE_PCRE2_JIT -DUSE_POLL -DUSE_THREAD -DUSE_BACKTRACE   -DUSE_TPROXY -DUSE_LINUX_TPROXY 
-DUSE_LINUX_SPLICE -DUSE_LIBCRYPT -DUSE_CRYPT_H -DUSE_GETADDRINFO -DUSE_OPENSSL -DUSE_LUA 
-DUSE_ACCEPT4   -DUSE_SLZ -DUSE_CPU_AFFINITY -DUSE_TFO -DUSE_NS -DUSE_DL -DUSE_RT  -DUSE_PRCTL  
-DUSE_THREAD_DUMP-DUSE_PCRE2 -DPCRE2_CODE_UNIT_WIDTH=8  -I/usr/local/include 
-DCONFIG_HAPROXY_VERSION=\"2.5.0\" -DCONFIG_HAPROXY_DATE=\"2021/11/23\" -c -o 
src/connection.o src/connection.c
   src/hlua.c: In function 'hlua_config_prepend_path':
   src/hlua.c:11292:2: error: 'for' loop initial declarations are only allowed 
in C99 or C11 mode
 for (size_t i = 0; i < 2; i++) {
 ^
   src/hlua.c:11292:2: note: use option -std=c99, -std=gnu99, -std=c11 or 
-std=gnu11 to compile your code

This commit moves loop iterator to an explicit declaration.

No backport needed as this issue was introduced in v2.5-dev10~69 with
commit 9e5e586e35c5 ("BUG/MINOR: lua: Fix lua error handling in
`hlua_config_prepend_path()`")
---
  src/hlua.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/src/hlua.c b/src/hlua.c
index 08735374af77..8dea91e75832 100644
--- a/src/hlua.c
+++ b/src/hlua.c
@@ -11249,6 +11249,7 @@ static int hlua_config_prepend_path(char **args, int 
section_type, struct proxy
char *path;
char *type = "path";
struct prepend_path *p = NULL;
+   size_t i;
  
  	if (too_many_args(2, args, err, NULL)) {

goto err;
@@ -11289,7 +11290,7 @@ static int hlua_config_prepend_path(char **args, int 
section_type, struct proxy
 * thread. The remaining threads will be initialized based on
 * prepend_path_list.
 */
-   for (size_t i = 0; i < 2; i++) {
+   for (i = 0; i < 2; i++) {
lua_State *L = hlua_states[i];
const char *error;
  



Thanks, merged now with backports info updated.

--
Christopher Faulet


[ANNOUNCE] haproxy-2.3.16

2021-11-24 Thread Christopher Faulet

Hi,

HAProxy 2.3.16 was released on 2021/11/24. It added 18 new commits
after version 2.3.15.

As announced for the 2.4.9, this release contains fixes about hidden bugs
recently exposed about the shutdowns management at the conn-stream
level. The client connections close could be delayed by the client
timeout. In addition, because of a failed backport, affecting the 2.2 too,
H1 responses could be truncated. All these bugs was fixed.

The H2 multiplexer fix to drains data and be sure to send GOAWAY frame was
finally backported. It was erroneously announced for the 2.3.15. As side
effect, the caching of TLS sessions is now fixed for H2 connections. Still
on the H2 multiplexer, an incomplete old fix for H2 partial frames was
fixed. It caused some high CPU usages in h2_io_cb() on some rare occasions.

Issues reported about occasional crashed in the cache (#1284 and #1451) was
fixed. A missing break statement was the explanation.

A bug with the "program" post-parser was fixed. It could be called with an
empty programs list in case of a config parsing error on reload after
another error, and could crash.

http-response rulesets evaluation was not aligned with what is said in the
documentation. It was possible to inhibit the frontend rules evaluation with
an "allow" rule in the backend section while it should instead only stop
backend rules evaluation. This bug exists since the beginning and only
concerns the "allow" rule. It was fixed and http-after-response rulesets
evaluation was also fixed in the same way.

William's fixes about the SSL was backported. First, outgoing TLS
connections involving SNI can now be resumed in TLS 1.3. Then, the right
error is not reported during SSL handshake when a non-matching SNI is found
with the strict-sni option enabled. A "unrecognized name" error is returned
instead of "handshake failure". As a side effect, this fixes the TLS resume
for non-matching SNI, rejecting the connections.

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.3/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.3.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.3.git
   Changelog: http://www.haproxy.org/download/2.3/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Christopher Faulet (8):
  BUG/MEDIUM: mux-h1: Fix H1C_F_ST_SILENT_SHUT value
  DOC: config: Fix typo in ssl_fc_unique_id description
  BUG/MINOR: http-ana: Apply stop to the current section for http-response 
rules
  Revert "BUG/MINOR: http-ana: Don't eval front after-response rules if stopped 
on back"
  DOC: lua: Be explicit with the Reply object limits
  BUG/MEDIUM: conn-stream: Don't reset CS flags on close
  BUG/MINOR: mux-h2: Fix H2_CF_DEM_SHORT_READ value
  BUG/MINOR: stick-table/cli: Check for invalid ipv6 key

William Lallemand (3):
  BUG/MEDIUM: ssl: backend TLS resumption with sni and TLSv1.3
  BUG/MINOR: mworker: doesn't launch the program postparser
  BUG/MEDIUM: ssl: abort with the correct SSL error when SNI not found

Willy Tarreau (7):
  BUG/MEDIUM: connection: make cs_shutr/cs_shutw//cs_close() idempotent
  MINOR: connection: add a new CO_FL_WANT_DRAIN flag to force drain on close
  MINOR: mux-h2: perform a full cycle shutdown+drain on close
  BUG/MEDIUM: mux-h2: always process a pending shut read
  BUG/MEDIUM: shctx: leave the block allocator when enough blocks are found
  BUG/MINOR: shctx: do not look for available blocks when the first one is 
enough
  MINOR: shctx: add a few BUG_ON() for consistency checks

--
Christopher Faulet



[ANNOUNCE] haproxy-2.4.9

2021-11-24 Thread Christopher Faulet



Hi,

HAProxy 2.4.9 was released on 2021/11/23. It added 36 new commits
after version 2.4.8.

In the previous release, fixes about shutdowns management in the muxes have
exposed some hidden bugs. Since the muxes were introduced, in the 1.8,
shutdowns at the conn-stream level were not fully idempotent. Until
recently, it was not an issue. But in the 2.4.8, some users observed delays
to close client connections on the HAProxy side corresponding to the client
timeout because the silent mode was used instead of the clean one to
shutdown the connection. In addition, true silent shutdowns were not
properly handled in the H1 multiplexer when outgoing data were blocked,
leading too to delay to close connections.

A H2 multiplexer fix to drain data and be sure to send GOAWAY frame was
announced in the 2.4.8. However a patch was missing. Another side effect of
this missing patch was the TLS sessions were not cached as expected. It is
now fixed. Still on the H2 multiplexer, an old fix for H2 partial frames was
incomplete and caused some high CPU usages in h2_io_cb() on some rare
occasions.

Some users reported occasional crashes in the cache (#1284 and #1451). We
finally had an explanation (a missing break). This was fixed. "show cache"
cli command was also fixed to be thread-safe. Under high load, it was
possible to dereference a node already reassigned, leading to
crash. Finally, parsing of "max-age" or "s-maxage" was improved to properly
ignore unparsable value in quotes.

A bug with the "program" post-parser was fixed. It could be called with an
empty programs list in case of a config parsing error on reload after
another error, and could crash.

Recent adjustments about the backend support for WebSocket over HTTP/2 were
backported. They allow to fallback on a HTTP/1 connection if the WebSockets
are not support in HTTP/2. In addition the server keyword "ws" can be used
to tune this.

http-response rulesets evaluation was not aligned with what is said in the
documentation. It was possible to inhibit the frontend rules evaluation with
an "allow" rule in the backend section while it should instead only stop
backend rules evaluation. This bug exists since the beginning and only
concerns the "allow" rule. It was fixed and http-after-response rulesets
evaluation was also fixed in the same way.

The support for backend aggregated server check status in the Prometheus
exporter was backported. Thanks to this feature, the number of server per
health-check status are now reported at the backend level.

William fixed some bugs in the SSL part. First, outgoing TLS connections
involving SNI couldn't be resumed in TLS 1.3 because the call to
SSL_get_servername() on a resumed connection doesn't return the previous SNI
with TLS 1.3. Then, the wrong error was reported during SSL handshake when a
non-matching SNI was found with the strict-sni option enabled because the
clientHello callback was returning with a success code. An "handshake
failure" was reported instead of "unrecognized name". As a side effect of
this bug, the connections was accepted in case of TLS resume. Finally,
thanks to Willy, the SSL counter are now atomically updated.

The detection of the need for libatomic in the makefile was modified so that
it's not hard-coded on the architecture but instead detects what the
compiler says it needs. This allowed to remove the arm/aarch64 hacks on
linux and also allows MIPS and RISCV to work as expected. In addition it's
now trivial to force it if desired.

In addition, the usual bunch of some of small fixes and cleanups.

The 2.3.16 will be emitted quite soon. The next 2.2 and 2.0 releases are
planned for the next week.

Thanks everyone for your help and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.4/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.4.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.4.git
   Changelog: http://www.haproxy.org/download/2.4/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Amaury Denoyelle (7):
  MINOR: mux-h2: add trace on extended connect usage
  BUG/MEDIUM: mux-h2: reject upgrade if no RFC8441 support
  MINOR: stream/mux: implement websocket stream flag
  MINOR: connection: implement function to update ALPN
  MINOR: connection: add alternative mux_ops param for conn_install_mux_be
  MEDIUM: server/backend: implement websocket protocol selection
  MINOR: server: add ws keyword

Christopher Faulet (10):
  DOC: config: Fix typo in ssl_fc_unique_id 

Re: [PATCH] MINOR: promex: backend aggregated server check status

2021-11-09 Thread Christopher Faulet

Le 11/8/21 à 14:31, William Dauchy a écrit :

On Mon, Nov 8, 2021 at 1:52 PM Willy Tarreau  wrote:

Just to be sure, is this something you want to merge into 2.5 or is it
to be queued next ? I'm fine with both, but I prefer to ask as it's not
just a one-liner and I don't know the impact on promex.


I know Christopher was possibly thinking about a more advanced
implementation but I thought it could be ok for a first version.
Probably a good idea to wait for Christopher feedbacks anyway.
I was indeed targeting a late v2.5, despite being a new thing. Sorry
for that and I forgot to add a message about it.
If it works well enough, I will also probably ask for a backport in
2.4 before the end of the year as I know large clusters are waiting
for this patch to lower their memory consumption in prometheus.

Thanks,



Hi William,

The change seems reasonable for a first implementation. I will add backports 
info in the commit message and merge it. Thanks !


--
Christopher Faulet



[ANNOUNCE] haproxy-2.2.18

2021-11-05 Thread Christopher Faulet
or sample fetch functions were only applied to
the frontend capability of a proxy. This means that using a small
set of sample fetch functions (like "be_name()") in proxies that are
both a frontend and a backend ("listen" or "defaults") would lead to
a config error while it is technically valid. This problem has always
been there and never reported.

  - automatic cast of variables to other types would fail to first verify
if a cast method was known, possibly causing a crash at runtime when
calling them for the first time (e.g. using a variable of type address
as an argument to strcmp() or a boolean with secure_memcmp()).

  - some streams could sometimes be frozen when filters were enabled (such
as compression) and an error was raised with data still left to be
processed.

  - HTTP health check could report L7 timeout when facing a parse error,
because the response is dropped before being translated to HTX, while
the check waiting for a response didn't explicitly check for a possible
end-of-input.

  - http-after-response rules must stop after an "allow" action, to match
their http-response counter-part.

  - the parsing of the "Authorization" header field would fail if more
than one space was present between the scheme and the value.

We planned to emit a new 2.0 release the next week if no major bug is
reported on this one affecting the 2.0 too. Thanks everyone for your help
and your contributions!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.2/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.2.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.2.git
   Changelog: http://www.haproxy.org/download/2.2/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Amaury Denoyelle (3):
  BUILD: ist: prevent gcc11 maybe-uninitialized warning on istalloc
  BUG/MINOR: server: allow 'enable health' only if check configured
  BUILD: fix compilation on NetBSD

Christopher Faulet (31):
  BUG/MEDIUM: stream-int: Don't block SI on a channel policy if EOI is 
reached
  Revert "REGTESTS: mark http_abortonclose as broken"
  BUG/MINOR: tcpcheck: Improve LDAP response parsing to fix LDAP check
  BUG/MINOR: h1-htx: Fix a typo when request parser is reset
  BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel 
buffer
  BUG/MEDIUM: stream-int: Notify stream that the mux wants more room to 
xfer data
  BUG/MEDIUM: stream: Stop waiting for more data if SI is blocked on 
RXBLK_ROOM
  BUG/MINOR: mux-h1/mux-fcgi: Sanitize TE header to only send "trailers"
  MINOR: arg: Be able to forbid unresolved args when building an argument 
list
  BUG/MINOR: tcpcheck: Don't use arg list for default proxies during parsing
  BUG/MINOR: tcp-rules: Stop content rules eval on read error and 
end-of-input
  BUG/MINOR: stream: Don't release a stream if FLT_END is still registered
  BUG/MEDIUM: http-ana: Reset channels analysers when returning an error
  BUG/MINOR: filters: Always set FLT_END analyser when CF_FLT_ANALYZE flag 
is set
  BUG/MINOR: filters: Set right FLT_END analyser depending on channel
  BUG/MEDIUM: filters: Fix a typo when a filter is attached blocking the 
release
  BUG/MEDIUM: http-ana: Clear request analyzers when applying redirect rule
  MINOR: htx: Add an HTX flag to know when a message is fragmented
  MINOR: htx: Add a function to know if the free space wraps
  BUG/MEDIUM: stream-int: Defrag HTX message in si_cs_recv() if necessary
  BUG/MEDIUM: mux_h2: Handle others remaining read0 cases on partial frames
  BUG/MINOR: http-ana: Don't eval front after-response rules if stopped on 
back
  BUG/MEDIUM: stream: Keep FLT_END analyzers if a stream detects a channel 
error
  BUG/MEDIUM: tcpcheck: Properly catch early HTTP parsing errors
  BUG/MINOR: mux-h1: Save shutdown mode if the shutdown is delayed
  BUG/MEDIUM: mux-h1: Perform a connection shutdown when the h1c is released
  BUG/MEDIUM: resolvers: Don't recursively perform requester unlink
  BUG/MEDIUM: resolvers: Track api calls with a counter to free resolutions
  BUG/MEDIUM: http-ana: Drain request data waiting the tarpit timeout 
expiration
  DOC: config: Fix alphabetical order of fc_* samples
  MINOR: stream: Improve dump of bogus streams

Dragan Dosen (1):
  BUG/MINOR: http-ana: increment internal_errors counter on response error

Emeric Brun (1):
  DOC: peers: fix doc "enable" statement 

Re: [PATCH] REGTESTS: Use `feature cmd` for 2.5+ tests (2)

2021-11-05 Thread Christopher Faulet

Le 11/4/21 à 21:12, Tim Duesterhus a écrit :

This patch effectively is identical to 7ba98480cc5b2ede0fd4cca162959f66beb82c82.


Merged, thanks Tim!

--
Christopher Faulet



Re: Problem with the var() sample fetch function

2021-10-27 Thread Christopher Faulet

Le 10/27/21 à 18:32, Willy Tarreau a écrit :

Christopher also found that the set-var() converter already mandates a
matching method, as the following will be rejected:

  ... if { int(12),set-var(txn.truc) 12 }

while this one will work:

  ... if { int(12),set-var(txn.truc) eq 12 }

Just a little clarification. Both are rejected with an error message. "-m int" 
is required:


... if { int(12),set-var(txn.truc) -m int eq 12 }

Emitting an error seems to be an obvious choice if var() sample is used in the 
same situations. However, this also means following ACLs will be rejected while 
it works today:


http-request set-var(txn.foo) str("forbidden")
http-request deny if { var(txn.foo) "forbidden" }

With the proposed change, "-m str" will be required.


--
Christopher Faulet



Re: compression offload ... in default section

2021-10-20 Thread Christopher Faulet

Le 10/20/21 à 01:24, Björn Jacke a écrit :

Hi,

On 19.10.21 11:06, Christopher Faulet wrote:

Sorry Björn, I missed your reply. It is strange, there is no known bug
in this area for now. There is probably something in the request or
response headers preventing the compression to be enabled.


I found the error: the "compression offload" is not honored in the
default section. This behavior is documented but it slipped from my memory.

What is actually the reason why this is silently ignored when it is set
in the the default section? If someone has a reason to set that in the
default section, why does haproxy ignore it intentionally?

If the support of compression offload shall stay intentionally
unsupported in the default section, then it would be good if this would
trigger a configure check warning, wouldn't it?

Thanks
Björn



Good catch. Indeed, this parameter is ignored in defaults sections. However 
there is no warning and the documentation is not really helpful on this point 
because it is only mentioned as a side note. I guess it may be more visible.


This parameter is not supported in defaults sections because there is no way to 
cancel it in backend sections. I guess we can add a directive to disable the 
compression offloading. This way, we will be able to support "compression 
offload" in defaults section.


--
Christopher Faulet



Re: PH disconnects, but "show errors" has 0 entries ?

2021-10-19 Thread Christopher Faulet

Le 10/19/21 à 16:49, Jim Freeman a écrit :

OK - this is weird (so don't shoot the messenger?).
With more tcpdump-ing and examination, the back-end service logs that it sent a 
response, but

  1) tcpdump running on the haproxy instance never sees the response !
      a) 2 proxies - an AWS ELB and on-instance nginx - lie between HAProxy 
instance and the service
  2) sans any response (and within 0.2 to 13 seconds of the request send), 
HAProxy initiates the PH/500 to the client!


It would make sense to me if any timeouts or disconnects were involved - HAProxy 
would report an [sS][DH] or somesuch.


And reverting the sending of the "content-security-policy: frame-ancestors ..." 
and "x-frame-options: ..." response(!) headers makes the problem disappear 
again.  You'll rightly point out that HTTP/1.1 is stateless, and that the prior 
history of the request/response stream (and response headers sent to the client) 
shouldn't affect the (non-)response to a given request.


Any clues as to how/why the PH/500 might be generated without a response to 
trigger it would be most eagerly received.  While it is entirely likely this 
will wind up being a "nut loose on the keyboard" issue, I just thought I'd share 
my observations and befuddlement ...




First of all, I missed a point. The 2.2.8 is quite old. You must upgrade first. 
Then, have you check the rewrite error counters on your backend ? Because, 
AFAIK, it is the only place where a 500 is possible with the PH termination 
state. If you are using http-after-response rules, it may explain this error.


However, share your redacted configuration too. It can help me to explain what 
you observe.


--
Christopher Faulet



Re: PH disconnects, but "show errors" has 0 entries ?

2021-10-19 Thread Christopher Faulet

Le 10/13/21 à 8:30 PM, Jim Freeman a écrit :
In adding a couple of new security response headers via haproxy.cfg (one is 112 
bytes, the other 32), a few requests are now getting 500 status (PH session 
state) responses, but "show errors" has 0 entries?  Most responses succeed (all 
have the additional headers), so it's not a problem with the new headers themselves.


If haproxy generates a PH/500, shouldn't "show errors" show details of the 
offending response ?


Thanks,
...jfree
==
# echo "show info" | socat stdio /run/haproxy/stats.sock | grep ^Version:
Version: 2.2.8-1~bpo10+1

#  echo "show errors -1" | socat - /run/haproxy/stats.sock
Total events captured on [13/Oct/2021:18:24:15.819] : 0

# cat /etc/debian_version
10.11


Hi,

Only parsing errors are reported by "show errors" command. Here PH/500 error is 
most probably due to a header rewrite error. I have not deeply checked however. 
You can verify my assumption by checking the "wrew" counter in "show stats" 
command output on the stats socket.


Header rewrite errors are triggered when there is not enough space in the buffer 
to perform the rewrites. By default, 1024 Bytes are reserved in the buffer, to 
be sure to have enough space to perform some rewrites. If you add many headers 
in the response, it may be the problem. You can increase the reserve by setting 
"tune.maxrewrite" global parameter.


When such error is encountered, HAProxy returns a 500-Internal-Error response. 
You can change that to make it fails silently. To do so, take a look at the 
"strict-mode" http-response action.


--
Christopher Faulet



Re: compression offload and http2

2021-10-19 Thread Christopher Faulet

Le 10/15/21 à 12:04 PM, Björn Jacke a écrit :

On 15.10.21 10:10, Christopher Faulet wrote:


It should work. What is your HAProxy version ?


2.4.7

Björn



Sorry Björn, I missed your reply. It is strange, there is no known bug in this 
area for now. There is probably something in the request or response headers 
preventing the compression to be enabled. If it is easy to reproduce on a test 
platform, you may try to start HAproxy in foreground in debug mode (-db -d on 
the command line). This will print the request and the response on stderr.


--
Christopher Faulet



Re: compression offload and http2

2021-10-15 Thread Christopher Faulet

Le 10/15/21 à 12:47 AM, Björn Jacke a écrit :

Hi,

I noticed that the compression offload feature is not working with
backends using h2. I couldn't find any note in the documentation that
the compression offload feature is limited to http 1 only. Is it a bug
that it doesn't work with http2 or is it by design and just the
documentation might need some clarification here.


Hi,

It should work. What is your HAProxy version ?


--
Christopher Faulet



Re: [PATCH] BUG/MINOR: lua: Fix lua error handling in `hlua_config_prepend_path()`

2021-10-12 Thread Christopher Faulet

Le 10/11/21 à 6:51 PM, Tim Duesterhus a écrit :

Set an `lua_atpanic()` handler before calling `hlua_prepend_path()` in
`hlua_config_prepend_path()`.

This prevents the process from abort()ing when `hlua_prepend_path()` fails
for some reason.

see GitHub Issue #1409

This is a very minor issue that can't happen in practice. No backport needed.
---
  src/hlua.c | 27 +--
  1 file changed, 25 insertions(+), 2 deletions(-)

diff --git a/src/hlua.c b/src/hlua.c
index b03d8bb46..1dbaaa3e0 100644
--- a/src/hlua.c
+++ b/src/hlua.c
@@ -11167,8 +11167,31 @@ static int hlua_config_prepend_path(char **args, int 
section_type, struct proxy
}
LIST_APPEND(_path_list, >l);
  
-	hlua_prepend_path(hlua_states[0], type, path);

-   hlua_prepend_path(hlua_states[1], type, path);
+   /* Handle the global state and the per-thread state for the first
+* thread. The remaining threads will be initialized based on
+* prepend_path_list.
+*/
+   for (size_t i = 0; i < 2; i++) {
+   lua_State *L = hlua_states[i];
+   const char *error;
+
+   if (setjmp(safe_ljmp_env) != 0) {
+   lua_atpanic(L, hlua_panic_safe);
+   if (lua_type(L, -1) == LUA_TSTRING)
+   error = lua_tostring(L, -1);
+   else
+   error = "critical error";
+   fprintf(stderr, "lua-prepend-path: %s.\n", error);
+   exit(1);
+   } else {
+   lua_atpanic(L, hlua_panic_ljmp);
+   }
+
+   hlua_prepend_path(L, type, path);
+
+   lua_atpanic(L, hlua_panic_safe);
+   }
+
return 0;
  
  err2:




Thanks Tim, now merged !

--
Christopher Faulet



Re: Use variables in healthchecks

2021-10-08 Thread Christopher Faulet

Le 10/7/21 à 11:59 AM, Basti Schubert a écrit :

Hi there,

i'd like to use variables in healtchecks to set the host header to the
ip of the server that is beeing checked, something like this:


http-check connect port 443 ssl
http-check send meth GET uri /ecp/healthcheck.htm ver HTTP/1.1 hdr host 
$SERVERIP
http-check expect status 200


Problem is: i don't know if this is possible and if there is already a
variable for such configs (can %si from the log format section be used
for this?)

I'm on haproxy 2.2.x



Hi,

It is possible to use sample fetches and log-format variables. Thus, you can 
indeed rely on "%si". But you must use 2.2.14 or newer.


--
Christopher Faulet



[ANNOUNCE] haproxy-2.4.7

2021-10-04 Thread Christopher Faulet

Hi,

HAProxy 2.4.7 was released on 2021/10/04. It added 2 new commits
after version 2.4.5.

No, it is no an error. The 2.4.6 was released few hours ago to fix a
regression of the 2.4.5. But I was a little too enthusiastic because
another bug was reported by Yves Lafon. Thus we decided to not announce the
2.4.6 to immediately release the 2.4.7.

The first regression was introduced by the commit feca2a453 ("BUG/MINOR:
filters: Always set FLT_END analyser when CF_FLT_ANALYZE flag is
set"). Because of a typo in the request analyzers when a filter was attached
to a stream, it was possible to hang on the filter release stage. HTTP
filters (HTTP compression and cache) didn't seem to be affected by the bug
in HTTP mode. However, in TCP mode, data forwarding was blocked until the
client or the server timeout expiration.

The second regression was introduced by the commit db8ba1069 ("BUG/MEDIUM:
http-ana: Reset channels analysers when returning an error"). The request
analyzers were not properly cleared when a redirect rule from a redirect
ruleset (via the "redirect ..." directive) was applied. This means that some
HTTP request processing were possible on an already flushed request channel,
leading to crash.

If you planned to upgrade to the 2.4.5 or the 2.4.6, please use the 2.4.7
instead.

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.4/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.4.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.4.git
   Changelog: http://www.haproxy.org/download/2.4/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Christopher Faulet (2):
  BUG/MEDIUM: http-ana: Clear request analyzers when applying redirect rule
  BUG/MEDIUM: filters: Fix a typo when a filter is attached blocking the 
release

--
Christopher Faulet



Re: [ANNOUNCE] haproxy-2.4.5

2021-10-04 Thread Christopher Faulet

Le 10/4/21 à 11:05 AM, Herzfeld, Patrick a écrit :

Ciao Christopher

I see the new release is already on the haproxy.org website, will there be an 
official release mail to the mailing list or no?



I'm adding the mailing list to notify everyone.

We are hunting another bug. Unfortunately, it was reported after the 2.4.6 was 
released. We try to figure out if it is a major problem or not for now. But it 
crashes HAProxy. So, a 2.4.7 will probably be released very soon.



--
Christopher Faulet



Re: [ANNOUNCE] haproxy-2.4.5

2021-10-03 Thread Christopher Faulet

Le 10/3/21 à 7:46 PM, Matthias Fechner a écrit :

Am 03.10.2021 um 08:53 schrieb Christopher Faulet:



Damned ! You're right...

It is a typo in the commit feca2a453 ("BUG/MINOR: filters: Always set FLT_END 
analyzer when CF_FLT_ANALYZE flag is set"). It also affects the 2.5-DEV. 


thanks a lot Christopher for the quick fix.
Just prepared a new package for FreeBSD and it is working fine.

I will ask the maintainer to include this patch till a new release is out.



Thanks Matthias. FYI, I will emit a new release this morning.


--
Christopher Faulet



Re: [ANNOUNCE] haproxy-2.4.5

2021-10-03 Thread Christopher Faulet

Le 10/3/21 à 9:06 AM, Vincent Bernat a écrit :

  ❦  3 October 2021 08:53 +02, Christopher Faulet:


I will push a fix. As a workaround, you can temporarily disable the HTTP
compression filter.


Will you release 2.4.6 or should we push packages for 2.4.5 with the
patch? For Debian/Ubuntu, I didn't push packages for 2.4.5 yet.



It is probably better to make a new release. At least for everyone relying on 
the sources instead of a distribution package.


--
Christopher Faulet



Re: [ANNOUNCE] haproxy-2.4.5

2021-10-03 Thread Christopher Faulet

Le 10/2/21 à 10:54 AM, Matthias Fechner a écrit :

Am 01.10.2021 um 18:09 schrieb Christopher Faulet:

HAProxy 2.4.5 was released on 2021/10/01. It added 69 new commits
after version 2.4.4.



Damned ! You're right...

It is a typo in the commit feca2a453 ("BUG/MINOR: filters: Always set FLT_END 
analyzer when CF_FLT_ANALYZE flag is set"). It also affects the 2.5-DEV.


The patch is pretty simple:

diff --git a/src/filters.c b/src/filters.c
index 136a3e80b..f64c192bd 100644
--- a/src/filters.c
+++ b/src/filters.c
@@ -475,7 +475,7 @@ flt_stream_start(struct stream *s)
}
if (strm_li(s) && (strm_li(s)->analysers & AN_REQ_FLT_START_FE)) {
s->req.flags |= CF_FLT_ANALYZE;
-   s->req.analysers |= AN_RES_FLT_END;
+   s->req.analysers |= AN_REQ_FLT_END;
}
return 0;
 }


I will push a fix. As a workaround, you can temporarily disable the HTTP 
compression filter.


Thanks for the report !

--
Christopher Faulet



[ANNOUNCE] haproxy-2.4.5

2021-10-01 Thread Christopher Faulet
 http://www.haproxy.org/download/2.4/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Amaury Denoyelle (8):
  BUG/MINOR: connection: prevent null deref on mux cleanup task allocation
  BUILD: ist: prevent gcc11 maybe-uninitialized warning on istalloc
  BUG/MINOR: server: allow 'enable health' only if check configured
  MINOR: server: implement a refcount for dynamic servers
  MINOR: global: define MODE_STOPPING
  BUG/MINOR: server: do not use refcount in free_server in stopping mode
  MINOR: server: return the next srv instance on free_server
  BUG/MINOR: stats: use refcount to protect dynamic server on dump

Christopher Faulet (26):
  MINOR: lua: Add a flag on lua context to know the yield capability at run 
time
  BUG/MINOR: lua: Yield in channel functions only if lua context can yield
  BUG/MINOR: lua: Don't yield in channel.append() and channel.set()
  BUG/MINOR: stream: Don't release a stream if FLT_END is still registered
  BUG/MEDIUM: http-ana: Reset channels analysers when returning an error
  BUG/MINOR: filters: Always set FLT_END analyser when CF_FLT_ANALYZE flag 
is set
  BUG/MINOR: filters: Set right FLT_END analyser depending on channel
  BUG/MEDIUM: mux-h1: Remove "Upgrade:" header for requests with payload
  MINOR: htx: Skip headers with no value when adding a header list to a 
message
  CLEANUP: mux-h1: Remove condition rejecting upgrade requests with payload
  BUG/MEDIUM: stream-int: Don't block SI on a channel policy if EOI is 
reached
  BUG/MAJOR: mux-h1: Don't eval input data if an error was reported
  BUG/MINOR: tcpcheck: Improve LDAP response parsing to fix LDAP check
  BUG/MINOR: h1-htx: Fix a typo when request parser is reset
  BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel 
buffer
  BUG/MEDIUM: stream-int: Notify stream that the mux wants more room to 
xfer data
  BUG/MEDIUM: stream: Stop waiting for more data if SI is blocked on 
RXBLK_ROOM
  MINOR: stream-int: Set CO_RFL transient/persistent flags apart in 
si_cs_rcv()
  MINOR: htx: Add an HTX flag to know when a message is fragmented
  MINOR: htx: Add a function to know if the free space wraps
  BUG/MEDIUM: stream-int: Defrag HTX message in si_cs_recv() if necessary
  MINOR: stream-int: Notify mux when the buffer is not stuck when calling 
rcv_buf
  BUG/MINOR: mux-h1/mux-fcgi: Sanitize TE header to only send "trailers"
  MINOR: arg: Be able to forbid unresolved args when building an argument 
list
  BUG/MINOR: tcpcheck: Don't use arg list for default proxies during parsing
  BUG/MINOR: tcp-rules: Stop content rules eval on read error and 
end-of-input

David Carlier (1):
  BUILD: tools: get the absolute path of the current binary on NetBSD.

Dragan Dosen (2):
  BUG/MINOR: flt-trace: fix an infinite loop when random-parsing is set
  BUG/MINOR: http-ana: increment internal_errors counter on response error

Emeric Brun (1):
  DOC: peers: fix doc "enable" statement on "peers" sections

William Lallemand (3):
  BUG/MINOR: systemd: ExecStartPre must use -Ws
  DOC: management: certificate files must be sanitized before injection
  MINOR: Makefile: add MEMORY_POOLS to the list of DEBUG_xxx options

Willy Tarreau (25):
  BUG/MINOR: compat: make sure __WORDSIZE is always defined
  CLEANUP: pools: factor all malloc_trim() calls into trim_all_pools()
  MINOR: pools: automatically disable malloc_trim() with external allocators
  MINOR: pools: use mallinfo2() when available instead of mallinfo()
  BUG/MINOR: cli/payload: do not search for args inside payload
  BUILD: activity: use #ifdef not #if on USE_MEMORY_PROFILING
  BUILD/MINOR: defaults: eliminate warning on MAXHOSTNAMELEN with -Wundef
  BUILD/MINOR: ssl: avoid a build warning on LIBRESSL_VERSION with -Wundef
  IMPORT: slz: silence a build warning with -Wundef
  BUILD/MINOR: regex: avoid a build warning on USE_PCRE2 with -Wundef
  BUILD: ssl: next round of build warnings on LIBRESSL_VERSION_NUMBER
  BUILD: ssl: fix two remaining occurrences of #if USE_OPENSSL
  BUILD: tools: properly guard __GLIBC__ with defined()
  BUG/MINOR: vars: improve accuracy of the rules used to check expression 
validity
  MINOR: sample: add missing ARGC_ entries
  BUG/MINOR: vars: properly set the argument parsing context in the 
expression
  BUG/MINOR: vars: truncate the variable name in error reports about scope.
  BUG/MINOR: vars: do not talk about global section in CLI errors for 
set-var
  BUILD: compiler: fixed a missing test on  defined(__GNUC__)
  BUILD: halog: fix a -Wundef warning on non-glibc systems
  BUILD: threads: fix -Wundef for _POSIX_PRIORITY_SCHEDULING on libmusl
  BUG/MEDIUM: leastconn: fix rare possibility of divide by zero
  

Re: AW: Disabling HTTP/1.1 pipelining

2021-09-24 Thread Christopher Faulet

Le 9/21/21 à 6:00 PM, Stefan Behte a écrit :

Hi Christopher,

thank you for the hint, I'm aware of the different ways to mitigate DDoS with 
rate limits etc., I was just curious about the pipelining vector. :)

http://www.haproxy.org/download/2.4/doc/configuration.txt says:

" By default HAProxy operates in keep-alive mode with regards to persistent
   connections: for each connection it processes each request and response, and
   leaves the connection idle on both sides between the end of a response and
   the start of a new request. This mode may be changed by several options such
   as "option http-server-close" or "option httpclose". Setting "option
   http-server-close" enables HTTP connection-close mode on the server side
   while keeping the ability to support HTTP keep-alive and pipelining on the
   client side."

"1.1. The HTTP transaction model" and " timeout http-keep-alive" also mention 
pipelining.



Section 1.1 mainly describes generalities about the HTTP protocol. Only the end 
of the section is focused on HAProxy and it is specified it only supports 
keep-alive mode, not the pipelining.  However, I agree it is pretty confusing 
because pipelining is mentioned in "option http-server-close" and "timeout 
http-keep-alive" descriptions.


In fact, the ambiguities comes from the fact that HAProxy does not performed any 
HTTP pipelining. But the client is free to send several requests in same time. 
No error will be triggered. However, the requests will be processed the one 
after the other. Thus, HAProxy does not perform any HTTP pipelining but it does 
not forbid it.



So I guess I did just misunderstand the documentation and it would be nice to 
just clarify it in the docs that haproxy does not support HTTP/1.1 pipelining.


I agree. Pipelining should at least be removed from "option http-server-close" 
description. And section 1.1 should be reword to be clear on this point.



--
Christopher Faulet



Re: Disabling HTTP/1.1 pipelining

2021-09-20 Thread Christopher Faulet

Le 9/17/21 à 1:20 PM, Stefan Behte a écrit :

Hi everyone,

surely many on this list have heard about the meris botnet 
(https://krebsonsecurity.com/2021/09/krebsonsecurity-hit-by-huge-new-iot-botnet-meris/) 
which uses HTTP/1.1 pipelining for layer 7 attacks.


As far as I can see, it's not possible to disallow HTTP pipelining in haproxy, 
so the best possibility could be "option httpclose"?


Of course, this does not solve everything when a ~100k botnet is attacking, but 
it could ease the initial load / mitigate the pipelining vector a bit, as the 
attack clients have longer RTT.


Or maybe I am missing something?


Hi,

HAproxy does not support HTTP pipelining. But it may be configured to mitigate 
ddos attack. There are several mechanisms that you can use, depending on your 
applications. A quick search on the net about "haproxy ddos prevention" will 
give you several hints.


Regards,
--
Christopher Faulet



Re: [PATCH 1/2] CLEANUP: Include check.h in flt_spoe.c

2021-09-20 Thread Christopher Faulet

Le 9/16/21 à 5:38 PM, Tim Duesterhus a écrit :

This is required for the prototype of spoe_prepare_healthcheck_request().
---
  src/flt_spoe.c | 1 +
  1 file changed, 1 insertion(+)

diff --git a/src/flt_spoe.c b/src/flt_spoe.c
index 28a5a24f8..aeb68c889 100644
--- a/src/flt_spoe.c
+++ b/src/flt_spoe.c
@@ -17,6 +17,7 @@
  #include 
  #include 
  #include 
+#include 
  #include 
  #include 
  #include 



Thanks Tim! Both patches merged now.

--
Christopher Faulet



  1   2   3   4   5   6   7   >