Re: [ANNOUNCE] haproxy-2.7-dev3

2022-08-12 Thread Amaury Denoyelle
On Tue, Aug 09, 2022 at 03:14:34PM -0600, Shawn Heisey wrote:
> On 8/7/22 10:07, Willy Tarreau wrote:
> > Given that QUIC users in 2.6 tend to be a bit forced to update to 2.7-dev
> > to improve reporting, we're starting to think about updating the QUIC stack
> > in 2.6 to match 2.7 once we're done with the pending issues.
> I am not at all familiar with the code, and I probably never will be ... at
> this time and for the foreseeable future, I have no need to be familiar with
> code for TLS and low-level networking, and it would be a huge time
> investment to become familiar with it. Perfectly happy to let others be the
> experts.
> Updating the whole QUIC stack in 2.6 to what's in 2.7-dev sounds to me like
> a good idea.  Some of my web pages are mostly unusable when doing http/3
> with 2.6 on Firefox.  Chrome works a lot better, but I have seen some
> problems there too.  I switched to 2.7 from git about the time dev2 was
> released.  So far everything that I have tried with 2.7 works very well on
> both browsers.

Indeed we tried to backport all QUIC bugs but this is not enough for the
moment and we continue to improve features and refactor the code on
2.7-dev, so it difficult to maintain a clean QUIC on 2.6. Hopefully, it
should not be too hard to align the two versions once we have fixed some
remaining issues.

> It will be very interesting to see how things shake out when the openssl
> team eventually gets around to tackling QUIC.  I won't be able to follow a
> lot of it, but I will take an occasional look.
> I love working with haproxy.  Can't thank Willy et al enough for all the
> effort involved.  For my own websites, I just use it as a reverse proxy
> handling TLS, and some minor filtering/fixup, not a load balancer, but if I
> ever get enough spare hardware for redundancy, that might change.  At a
> previous $DAYJOB I did set it up as a load balancer.  I think I got up to
> 1.5.18 by the time that job ended.
> I like this little web page I made.  A tiny bit of php code.
> https://http3test.elyograg.org/

Thank you very much for your kind words and your first reports on QUIC
which have greatly helped us. Let's hope we keep refine QUIC support
even more in the near future.

-- 
Amaury Denoyelle



[ANNOUNCE] haproxy-2.6.1

2022-06-21 Thread Amaury Denoyelle
g: http://www.haproxy.org/download/2.6/src/CHANGELOG
   Pending bugs : http://www.haproxy.org/l/pending-bugs
   Reviewed bugs: http://www.haproxy.org/l/reviewed-bugs
   Code reports : http://www.haproxy.org/l/code-reports
   Latest builds: http://www.haproxy.org/l/dev-packages

---
Complete changelog :
Amaury Denoyelle (10):
  BUG/MINOR: h3: fix frame type definition
  BUG/MEDIUM: mux-quic: fix flow control connection Tx level
  BUG/MINOR: mux-quic: fix memleak on frames rejected by transport
  BUG/MEDIUM: mux-quic: fix segfault on flow-control frame cleanup
  BUG/MINOR: qpack: support header litteral name decoding
  MINOR: qpack: add comments and remove a useless trace
  BUG/MINOR: h3/qpack: deal with too many headers
  BUG/BUILD: h3: fix wrong label name
  BUG/MINOR: quic: purge conn Rx packet list on release
  BUG/MINOR: quic: free rejected Rx packets

Christopher Faulet (36):
  BUG/MINOR: ssl_ckch: Free error msg if commit changes on a cert entry 
fails
  BUG/MINOR: ssl_ckch: Free error msg if commit changes on a CA/CRL entry 
fails
  BUG/MEDIUM: ssl_ckch: Don't delete a cert entry if it is being modified
  BUG/MEDIUM: ssl_ckch: Don't delete CA/CRL entry if it is being modified
  BUG/MINOR: ssl_ckch: Don't duplicate path when replacing a cert entry
  BUG/MINOR: ssl_ckch: Don't duplicate path when replacing a CA/CRL entry
  BUG/MEDIUM: ssl_ckch: Rework 'commit ssl cert' to handle full buffer cases
  BUG/MEDIUM: ssl_ckch: Rework 'commit ssl ca-file' to handle full buffer 
cases
  BUG/MEDIUM: ssl/crt-list: Rework 'add ssl crt-list' to handle full buffer 
cases
  BUG/MEDIUM: httpclient: Don't remove HTX header blocks before duplicating 
them
  BUG/MEDIUM: httpclient: Rework CLI I/O handler to handle full buffer cases
  MEDIUM: http-ana: Always report rewrite failures as PRXCOND in logs
  MEDIUM: httpclient: Don't close CLI applet at the end of a response
  REGTESTS: abortonclose: Add a barrier to not mix up log messages
  REGTESTS: http_request_buffer: Increase client timeout to wait "slow" 
clients
  BUG/MINOR: ssl_ckch: Use right type for old entry in show_crlfile_ctx
  BUG/MINOR: ssl_ckch: Dump CRL transaction only once if show command yield
  BUG/MINOR: ssl_ckch: Dump CA transaction only once if show command yield
  BUG/MINOR: ssl_ckch: Dump cert transaction only once if show command yield
  BUG/MINOR: ssl_ckch: Init right field when parsing "commit ssl crl-file" 
cmd
  BUG/MINOR: ssl_ckch: Fix possible uninitialized value in show_cert I/O 
handler
  BUG/MINOR: ssl_ckch: Fix possible uninitialized value in show_cafile I/O 
handler
  BUG/MINOR: ssl_ckch: Fix possible uninitialized value in show_crlfile I/O 
handler
  REGTESTS: http_abortonclose: Extend supported versions
  REGTESTS: restrict_req_hdr_names: Extend supported versions
  BUG/MEDIUM: mailers: Set the object type for check attached to an email 
alert
  BUG/MINOR: trace: Test server existence for health-checks to get proxy
  BUG/MINOR: checks: Properly handle email alerts in trace messages
  REGTESTS: healthcheckmail: Update the test to be functionnal again
  REGTESTS: healthcheckmail: Relax health-check failure condition
  BUG/MINOR: tcp-rules: Make action call final on read error and delay 
expiration
  BUG/MEDIUM: stconn: Don't wakeup applet for send if it won't consume data
  BUG/MEDIUM: cli: Notify cli applet won't consume data during request 
processing
  BUG/MEDIUM: stream: Properly handle destructive client connection upgrades
  MINOR: stream: Rely on stconn flags to abort stream destructive upgrade
  BUG/MINOR: log: Properly test connection retries to fix dontlog-normal 
option

Frédéric Lécaille (5):
  BUG/MINOR: quic: Stop hardcoding Retry packet Version field
  BUG/MINOR: quic: Wrong PTO calculation
  BUG/MINOR: quic: Unexpected half open connection counter wrapping
  BUG/MINOR: quic_stats: Duplicate "quic_streams_data_blocked_bidi" field 
name
  BUG/MINOR: quic: Acknowledgement must be forced during handshake

William Lallemand (3):
  BUG/MEDIUM: ssl/cli: crash when crt inserted into a crt-list
  BUG/MEDIUM: mworker: use default maxconn in wait mode
  REGTESTS: ssl: add the same cert for client/server

Willy Tarreau (5):
  BUILD: compiler: implement unreachable for older compilers too
  BUG/MINOR: cli/stats: add missing trailing LF after JSON outputs
  BUG/MINOR: server: do not enable DNS resolution on disabled proxies
  BUG/MINOR: cli/stats: add missing trailing LF after "show info json"
  BUG/MINOR: task: fix thread assignment in tasklet_kill()

---

-- 
Amaury Denoyelle



Re: haproxy 2.6.0 and quic

2022-06-15 Thread Amaury Denoyelle
On Fri, Jun 03, 2022 at 07:08:43AM -0600, Shawn Heisey wrote:
> [...]
> A word of warning that you would probably also get from the devs here: 
> HTTP3/QUIC support is still new and not entirely working. I have it
> configured and it only works correctly for VERY simple websites.  Any
> complex webapp I try it on will fail in some way, but if I disable HTTP3 and
> use HTTP2, it works.
> 

Hi,

I just wanted to let you and other QUIC enthusiast know that I
found a defect in haproxy QPACK implementation which prevented to
decrypt some headers. The fix has been merged and it helped greatly on
my test with a nextcloud instance.

Of course, I still have some other issues and unexpected behavior, but
if you have the time do not hesitate to test the development version and
give us your thoughts. As an alternative, we may probably emit soon a
2.6.1 with the first batch of QUIC issues resolved so far.

Regards,

-- 
Amaury Denoyelle



Re: [PATCH] CLEANUP: Re-apply xalloc_size.cocci (2)

2022-06-02 Thread Amaury Denoyelle
On Wed, Jun 01, 2022 at 09:58:37PM +0200, Tim Duesterhus wrote:
> This reapplies the xalloc_size.cocci patch across the whole `src/` tree.
> see 16cc16dd8235e7eb6c38b7abd210bd1e1d96b1d9
> see 63ee0e4c01b94aee5fc6c6dd98cfc4480ae5ea46
> ---
>  src/ncbuf.c  | 2 +-
>  src/proto_quic.c | 2 +-
>  src/quic_sock.c  | 3 ++-
>  3 files changed, 4 insertions(+), 3 deletions(-)
> diff --git a/src/ncbuf.c b/src/ncbuf.c
> index 1944cfe34..adb32b57a 100644
> --- a/src/ncbuf.c
> +++ b/src/ncbuf.c
> @@ -726,7 +726,7 @@ struct rand_off {
>  static struct rand_off *ncb_generate_rand_off(const struct ncbuf *buf)
>  {
>   struct rand_off *roff;
> - roff = calloc(1, sizeof(struct rand_off));
> + roff = calloc(1, sizeof(*roff));
>   BUG_ON(!roff);
>  
>   roff->off = rand() % (ncb_size(buf));
> diff --git a/src/proto_quic.c b/src/proto_quic.c
> index 55aa4b50f..ab1bef18f 100644
> --- a/src/proto_quic.c
> +++ b/src/proto_quic.c
> @@ -703,7 +703,7 @@ static int quic_alloc_dghdlrs(void)
>  {
>   int i;
>  
> - quic_dghdlrs = calloc(global.nbthread, sizeof(struct quic_dghdlr));
> + quic_dghdlrs = calloc(global.nbthread, sizeof(*quic_dghdlrs));
>   if (!quic_dghdlrs) {
>   ha_alert("Failed to allocate the quic datagram handlers.\n");
>   return 0;
> diff --git a/src/quic_sock.c b/src/quic_sock.c
> index 6207af703..a391006af 100644
> --- a/src/quic_sock.c
> +++ b/src/quic_sock.c
> @@ -466,7 +466,8 @@ static int quic_alloc_accept_queues(void)
>  {
>   int i;
>  
> - quic_accept_queues = calloc(global.nbthread, sizeof(struct 
> quic_accept_queue));
> + quic_accept_queues = calloc(global.nbthread,
> + sizeof(*quic_accept_queues));
>   if (!quic_accept_queues) {
>   ha_alert("Failed to allocate the quic accept queues.\n");
>   return 0;
> -- 
> 2.36.1
> 

Fine for me,
Thanks,

-- 
Amaury Denoyelle



Re: Thoughts on QUIC/HTTP3

2022-05-31 Thread Amaury Denoyelle
On Sun, May 29, 2022 at 12:37:14PM -0600, Shawn Heisey wrote:
> On 4/29/2022 10:10 AM, Shawn Heisey wrote:
> > I did a build and install this morning, a bunch of quic-related changes
> > in that.  Now everything seems to be working on my paste site.  Large
> > pastes work, and I can reload the page a ton of times without it hanging
> > until browser restart.
> I have found that the roundcube package doesn't work completely right with
> quic/http3. It has proven to be an excellent way of revealing problems.  The
> roundcube package is a php webmail app that utilizes a database and an imap
> server.  It's using many of the features that are possible with modern
> browsers.
> When I first started this http3 journey, I couldn't even log in to roundcube
> because any POST request was failing ... but that problem got fixed.  Now I
> find that when quickly jumping between folders in my big mailbox, the site
> stops responding.  If I switch back to http/2, everything works.  If it
> would help, I can give one of the devs full access to use my roundcube
> install and experiment with the config, see if maybe I am doing something
> wrong.

Hi Shawn,

Thanks for your continuing your journey on HTTP/3 :)

On our side we finish our work on improving POST, it should be fairly
complete now. We also improve other aspects and complete the
documentation for new users. Now, we are using nextcloud, which is
another big webapp, through haproxy/QUIC as a test. We have already fix
some bugs which may also help with your roundcube installation.

However, despite our efforts we know there is still a grey area in our
QUIC stack. We do not properly manage the connection closing, and it's
not completely clear how we should handle all the cases. Sadly, this
requires some reflexion time and it won't be solve before the imminent
2.6 release. For the moment, you can try on your side to
increase/decrease the haproxy client timeout to see if this change your
situation.

Regards,

-- 
Amaury Denoyelle



Re: Thoughts on QUIC/HTTP3

2022-04-25 Thread Amaury Denoyelle
Hi,

Sorry for not having answers your last mails, I was busy with some
new improvment for QUIC on haproxy side. Most notably, we improved the
transfer performance slightly by being able to use multiple buffers per
streams. Please find my answers to your remarks below,

On Sat, Apr 23, 2022 at 11:13:01PM -0600, Shawn Heisey wrote:
> After seeing http/3 (orange lightning bolt with the HTTP Version Indicator
> extension) talking to a lot of websites, I had thought the standard was
> further along than it is.  I see that the openssl team is discussing it, and
> plans to fully embrace it, but hasn't actually started putting QUIC code in
> openssl yet, and it may be quite some time before something usable shows up
> even in their master branch.

I would not put too much faith in it for the near future. The OpenSSL
team seems to have put aside a simple QUIC API integration in favor of a
brand new full QUIC stack, which should take quite some time. So for
now, manually rebuilding your SSL library seems the only way to go.

> It's been fun fiddling with it using haproxy with quictls, and I hope I can
> provide useful information to stabilize the implementation.
> I'd like to say thank you to Willy and all the other people who make haproxy
> one of the best things in my problem-solving arsenal.  It handles the
> internet side of all my web deployments.  I haven't yet put other services
> behind it.  At a previous $DAYJOB I had been testing FTP load-balancing,
> which I did get working, but didn't actually get to the deployment stage.
> At the moment, I am experiencing two problems with http3.  The second
> problem might actually just be another instance of the first problem.
> First problem:  If I do enough fiddling with an HTTP3 page, in either
> Firefox or Chrome, eventually that page will stop loading and the only way
> I've found to fix it is to completely close the browser and reopen it. 
> Restarting haproxy or Apache doesn't seem to help.

We already experienced this kind of problems but we though we have fixed
them. It seems the connection closure it not always properly handle on
haproxy side, which left the client with no notification that it should
open a new connection. It may help to increase the timeout client to be
greater than the default QUIC idle timeout, which is hardcoded to 10s on
haproxy side. For haproxy.org, we use a value of 1m and it seems to be
working. Please tell me if this makes your problem disappears or not.

> Second problem:  If I try pasting a REALLY large block of text into my paste
> website at the following URL while I have it configured to use HTTP/3, it
> won't work.  The page never loads. I can't tell if this is a separate
> problem from the one above, or just another occurrence of it that triggers
> more readily because there is more data transferred.  The reason I think it
> might be actually the first problem is that if I open another tab, I can't
> get to the website ... but if I close the browser and reopen it, then I can
> get to the website again.
> https://paste.elyograg.org/
> If I remove the paste website from the http3 ACL so it doesn't send the
> alt-svc header, then everything works once I can convince the browser to
> stop using HTTP/3.

We did not have enough time to work on POST so there is still bugs in
this part. I just recently fixed two bugs which may be enough to fix
your situattion with the latest 2.6-dev7. However, I still have issues
when I use large payloads.

Thanks for your kind compliment for haproxy reliability. We hope one day
we will reach this level for QUIC but for now this objectif is still
far.

-- 
Amaury Denoyelle



Re: HTTP/3 -- POST requests not working

2022-04-15 Thread Amaury Denoyelle
On Thu, Apr 14, 2022 at 07:29:20AM -0600, Shawn Heisey wrote:
> On 4/14/22 03:27, Amaury Denoyelle wrote:
> > So to summary, this option should be activated if you only have browsers
> > as client and the traffic is big enough to saturate haproxy queues.
> > I hope this will clarify your thoughts,
> Thanks for that detail.  For these setups, I really doubt that there will
> ever be that many requests. Unless I get hit by a DDOS attack. :)
> I am having further problems with http/3.  Randomly in the webmail it will
> get a 403 error when accessing a URL for the site.  Checking the log
> history, looks like it's on POST requests, which were already problematic. 
> I guess removing abortonclose doesn't completely fix POST requests.
> Apr 14 07:11:15 - haproxy[2357875] 199.192.164.74:54388
> [14/Apr/2022:07:11:15.227] web~ be-apache-81/bilbo 2/0/76/-1/85 403 1371 - -
> LHVN 1/1/0/0/0 0/0 {webmail.elyograg.org} "POST
> /mail/?_task=mail&_action=refresh HTTP/3.0"

Hum this is strange. Do you have a way to reproduce it easily ?
Otherwise, please know that as QUIC is still in an experimental status,
we did not have the time to test various config options with it. Indeed,
most of the time we use extra simple config files with only timeout
configuration provided. If you encounter a recurring bug, I advise you
to switch to a simple config file and inspect if the issue is still
there. This will allows us to more precisely understand the nature of
the problem.

> And something that has occasionally happened with anything I've enabled
> http/3 on ... firefox will get into a mode where accessing an http/3 enabled
> site will completely hang, and the only way to get that website working
> again is to close the browser, reopen it, and try again.  I've also seen
> this in Chrome, so it doesn't seem to be browser-specific.  I do most of my
> testing in Firefox, because on the work machine Chrome has a proxy, which
> won't do http/3.

Hum we already have encounter this issue because we did not send a
CONNECTION_CLOSE on connection closing. Now most cases seems to be fixed
but maybe there is still cases where the connection dies without a
notification to the client. Do you observe this frequently ?

> So for now I've only enabled http3 on a select subset of websites.  These
> problems make the webmail not work right.  I hope any information I gather
> will speed up development on QUIC so it can move out of experimental status.
> Ah, git pull has revealed a lot of changes, some of which are on quic code. 
> I will get that built and installed.

As I already said, QUIC in haproxy is still experimental and we expect a
lot of issues to appear :) so please use it with care and always have a
backup solution. We commit frequently changes related to QUIC but the
todo-list is still big and I do not think the latest changes will have
an improvement on your reported issues. But still, it is always useful
to have users ready to test it and report bugs, in fact we already have
fix some thanks to you. Just know that some bugs may take some time
before being addressed on our side.

Thanks,

-- 
Amaury Denoyelle



Re: HTTP/3 -- POST requests not working

2022-04-14 Thread Amaury Denoyelle
On Wed, Apr 13, 2022 at 08:16:06AM -0600, Shawn Heisey wrote:
> On 4/13/22 02:42, Amaury Denoyelle wrote:
> > Ok this seems related to 'option abortonclose'. Without this, I do not
> > have a 400 error. Can you confirm me this behavior on your side please ?
> 
> If I remove that, it works.  I can have my webmail served via http/3 and
> login still works, which it didn't before because all POST requests were
> getting a 400.
> What am I losing or gaining by removing abortonclose?  I read the docs on
> it, and wasn't able to work out whether I want it or not.  Will this failure
> when configuring abortonclose be considered a bug?

Currently, this option is buggy when used with a QUIC listener. This is
indeed a bug and will be fixed in a future patch.

> The reasons for that option being in the config are lost to the antiquities
> of time.  Most of my config is stolen from a setup I once built at an old
> ${DAYJOB} that was actually doing load balancing.  For my personal setups, I
> mainly use haproxy for SSL offloading and making multiple private-side web
> servers available through a single public IP address.

By default, connection are kept opened in haproxy side even if the
client has closed it. This is required because it's impossible for
haproxy to detect a full close or only a sending channel close. In the
last case, haproxy can still deliver the response and then fully close
the connection. This is conform to the default http specification and is
useful when dealing with clients tools such as netcat which rely heavily
on this behavior.

If all your clients are browsers and the traffic is enough to fill the
haproxy queue, it is useful to activate abortonclose. When a user hits
the "STOP" loading page of the browser, the connection is immediatly
close on haproxy, which liberate some resources to handle other
connections.

So to summary, this option should be activated if you only have browsers
as client and the traffic is big enough to saturate haproxy queues.

I hope this will clarify your thoughts,

-- 
Amaury Denoyelle



Re: HTTP/3 -- POST requests not working

2022-04-13 Thread Amaury Denoyelle
On Wed, Apr 13, 2022 at 10:36:00AM +0200, Amaury Denoyelle wrote:
> On Tue, Apr 12, 2022 at 10:30:05AM -0600, Shawn Heisey wrote:
> > On 4/12/22 09:45, Amaury Denoyelle wrote:
> > > After much analysis of the code, it may be useful to have a run with the
> > > stream traces as well :
> > > $ trace stream sink buf0; trace stream level developer; trace stream 
> > > verbosity clean; trace stream start now
> > All 3 traces enabled, this time it should only be for one connection, I
> > think the last one there were two:
> > https://paste.elyograg.org/view/a06556ef
> > > You can also provide us your haproxy config file, just remove any
> > > sensitive info if you have any.
> > https://paste.elyograg.org/view/3f884d8f
> > The acl lines with redacted info are not being matched for the connections
> > that don't work, so it should have no connection to this problem.
> I'm able to reproduce your issue with your config file. I can now
> investigate the problem. I will let you know about any update, thanks.

Ok this seems related to 'option abortonclose'. Without this, I do not
have a 400 error. Can you confirm me this behavior on your side please ?

-- 
Amaury Denoyelle



Re: HTTP/3 -- POST requests not working

2022-04-13 Thread Amaury Denoyelle
On Tue, Apr 12, 2022 at 10:30:05AM -0600, Shawn Heisey wrote:
> On 4/12/22 09:45, Amaury Denoyelle wrote:
> > After much analysis of the code, it may be useful to have a run with the
> > stream traces as well :
> > $ trace stream sink buf0; trace stream level developer; trace stream 
> > verbosity clean; trace stream start now
> All 3 traces enabled, this time it should only be for one connection, I
> think the last one there were two:
> https://paste.elyograg.org/view/a06556ef
> > You can also provide us your haproxy config file, just remove any
> > sensitive info if you have any.
> https://paste.elyograg.org/view/3f884d8f
> The acl lines with redacted info are not being matched for the connections
> that don't work, so it should have no connection to this problem.

I'm able to reproduce your issue with your config file. I can now
investigate the problem. I will let you know about any update, thanks.

-- 
Amaury Denoyelle



Re: HTTP/3 -- POST requests not working

2022-04-12 Thread Amaury Denoyelle
On Tue, Apr 12, 2022 at 05:26:41PM +0200, Amaury Denoyelle wrote:
> On Tue, Apr 12, 2022 at 08:01:59AM -0600, Shawn Heisey wrote:
> > On 4/12/22 02:22, Amaury Denoyelle wrote:
> > > then you can display the traces with the following command :
> > > $ show events buf0
> > > > For the h3 layer, the trace mechanism is not currently implemented. You
> > > should instead recompile your haproxy binary with the DEBUG options :
> > >   -DDEBUG_H3 -DDEBUG_QPACK
> > > and watch for the stderr output for your process.
> > I got some traces.  URL below for accessing it.  There's a LOT of data.
> > https://paste.elyograg.org/view/8678c875
> > I didn't do the H3 debug yet, if you still need it after looking at the
> > traces, let me know.  You'll also need to tell me how to make those debugs
> > active when I build haproxy.
> We analyzed the output of the traces but there is nothing of particular
> interest here. What may be useful is to activate the H3 traces.
> However, I notice that the -dev5 does not compile with H3 traces
> activated. So first, you have to apply the attached patch before
> recompiling. To do this, save the attachment and execute the following
> command in your source folder :
> $ patch -p1 < 0001-BUG-MINOR-h3-fix-build-with-DEBUG_H3.patch
> Alternatively, you can also use the latest master with contains the
> mentionned patch. However, I think it's safer to stay on the -dev5 tag
> for the moment.
> Once the patch is applied, recompile your haproxy binary with DEBUG_H3
> and DEBUG_QPACK flags. Here is a make example, complete it with your
> usual set of flags :
> $ make TARGET=linux-glibc USE_QUIC=1 DEBUG="-DDEBUG_H3 -DDEBUG_QPACK" ...
> You can now restart your process and watch the stderr output and report
> it to us.
> Again, thank you very much for your help,

Hi again,

After much analysis of the code, it may be useful to have a run with the
stream traces as well :
$ trace stream sink buf0; trace stream level developer; trace stream verbosity 
clean; trace stream start now

You can also provide us your haproxy config file, just remove any
sensitive info if you have any.

Thanks,

-- 
Amaury Denoyelle



Re: HTTP/3 -- POST requests not working

2022-04-12 Thread Amaury Denoyelle
On Tue, Apr 12, 2022 at 08:01:59AM -0600, Shawn Heisey wrote:
> On 4/12/22 02:22, Amaury Denoyelle wrote:
> > then you can display the traces with the following command :
> > $ show events buf0
> > > For the h3 layer, the trace mechanism is not currently implemented. You
> > should instead recompile your haproxy binary with the DEBUG options :
> >   -DDEBUG_H3 -DDEBUG_QPACK
> > and watch for the stderr output for your process.
> I got some traces.  URL below for accessing it.  There's a LOT of data.
> https://paste.elyograg.org/view/8678c875
> I didn't do the H3 debug yet, if you still need it after looking at the
> traces, let me know.  You'll also need to tell me how to make those debugs
> active when I build haproxy.

We analyzed the output of the traces but there is nothing of particular
interest here. What may be useful is to activate the H3 traces.

However, I notice that the -dev5 does not compile with H3 traces
activated. So first, you have to apply the attached patch before
recompiling. To do this, save the attachment and execute the following
command in your source folder :

$ patch -p1 < 0001-BUG-MINOR-h3-fix-build-with-DEBUG_H3.patch

Alternatively, you can also use the latest master with contains the
mentionned patch. However, I think it's safer to stay on the -dev5 tag
for the moment.

Once the patch is applied, recompile your haproxy binary with DEBUG_H3
and DEBUG_QPACK flags. Here is a make example, complete it with your
usual set of flags :

$ make TARGET=linux-glibc USE_QUIC=1 DEBUG="-DDEBUG_H3 -DDEBUG_QPACK" ...

You can now restart your process and watch the stderr output and report
it to us.

Again, thank you very much for your help,

-- 
Amaury Denoyelle
>From bb970422546c0db08b244d5e5a12a795a605dd64 Mon Sep 17 00:00:00 2001
From: Amaury Denoyelle 
Date: Tue, 12 Apr 2022 16:40:52 +0200
Subject: [PATCH] BUG/MINOR: h3: fix build with DEBUG_H3

qcs by_id field has been replaced by a new field named "id". Adjust the
h3_debug_printf traces. This is the case since the introduction of the
qc_stream_desc type.
---
 src/h3.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/h3.c b/src/h3.c
index e39c54135..2d450d8b8 100644
--- a/src/h3.c
+++ b/src/h3.c
@@ -231,7 +231,7 @@ static int h3_decode_qcs(struct qcs *qcs, int fin, void 
*ctx)
struct buffer *rxbuf = >rx.buf;
int ret;
 
-   h3_debug_printf(stderr, "%s: STREAM ID: %llu\n", __func__, 
qcs->by_id.key);
+   h3_debug_printf(stderr, "%s: STREAM ID: %lu\n", __func__, qcs->id);
if (!b_data(rxbuf))
return 0;
 
@@ -333,7 +333,7 @@ static int h3_control_recv(struct h3_uqs *h3_uqs, void *ctx)
struct buffer *rxbuf = _uqs->qcs->rx.buf;
struct h3 *h3 = ctx;
 
-   h3_debug_printf(stderr, "%s STREAM ID: %llu\n", __func__,  
h3_uqs->qcs->by_id.key);
+   h3_debug_printf(stderr, "%s STREAM ID: %lu\n", __func__,  
h3_uqs->qcs->id);
if (!b_data(rxbuf))
return 1;
 
-- 
2.35.1



Re: HTTP/3 -- POST requests not working

2022-04-12 Thread Amaury Denoyelle
On Tue, Apr 12, 2022 at 08:14:25AM +0200, Willy Tarreau wrote:
> Hi Shawn,
> [...]
> On Mon, Apr 11, 2022 at 01:05:21PM -0600, Shawn Heisey wrote:
> > An astute observer will notice that the backend says 81 while the packet
> > capture shows port 82.  This is because I changed the port number on the
> > server to point to nginx instead of apache, but did not rename the backend. 
> > My use of nginx is temporary, only using it to eliminate Apache as the
> > problem.  I know how to configure Apache; it would take me forever to fully
> > switch to nginx.
> I think that the QUIC team will soon ask you to enable traces on the QUIC
> layer, but I'll let them follow up ;-)

Hi Shawn,

First, thank you for your interest for QUIC in haproxy :)

I have quickly tested POST on my side but did not see any issue. To help
you diagnostic your problem, you can try to activate the traces on the
stats socket.

for the transport traces :
$ trace quic sink buf0; trace quic level developer; trace quic verbosity clean; 
trace quic start now

and for the mux :
$ trace qmux sink buf0; trace qmux level developer; trace qmux verbosity 
minimal; trace qmux start now

then you can display the traces with the following command :
$ show events buf0

For the h3 layer, the trace mechanism is not currently implemented. You
should instead recompile your haproxy binary with the DEBUG options :
 -DDEBUG_H3 -DDEBUG_QPACK
and watch for the stderr output for your process.

Thanks again for your help,
Regards,

-- 
Amaury Denoyelle



Re: [ANNOUNCE] haproxy-2.6-dev4

2022-03-28 Thread Amaury Denoyelle
On Sun, Mar 27, 2022 at 12:09:22AM +0500, Илья Шипицин wrote:
> сб, 26 мар. 2022 г. в 22:23, Ionel GARDAIS  >:
> > Thanks Willy for these updates.
> >
> > While skimming the result on the interop website, I was surprised that
> > haproxy is always more than 50% slower than its competitor.
> > Is it because you've enable lots of traces as part of your debugging
> > process for the runs ?
> >
> looks like this dockerfile is used
> https://github.com/haproxytech/haproxy-qns/blob/master/Dockerfile
> 

Hi,

Thanks for your interest on haproxy QUIC implementation :) Indeed the
perfs results reported by the interop test suite for haproxy look
miserable. To be honest, at the moment I did not look how these tests
are implemented. We left this part out as we dealt with plenty of
functional issues. As such, we enabled a lot of traces and debug options
to be able to quickly understand bugs, so this may have an impact on the
results of perfs test.

Now, we are focused on trying to deploy QUIC on haproxy.org to inspect
the behavior with real-life browsers. When this is done, the next
objective will be to try to improve the results of these perfs tests.

-- 
Amaury Denoyelle



Re: [PATCH] BUG/MEDIUM: server: avoid changing healthcheck ctx with set server ssl

2022-01-12 Thread Amaury Denoyelle
On Tue, Jan 11, 2022 at 06:55:10PM +0100, Christopher Faulet wrote:
> Le 1/10/22 à 23:19, Willy Tarreau a écrit :
> > At this point I'm starting to think that we should probably avoid as
> > much as possible to use implicit settings for whatever is dynamic.
> > Originally a lot of settings were implicit because we don't want to
> > have huge config lines to enforce lots of obvious settings. On the CLI
> > it's less of a problem and for example if "ssl" only deals with the
> > traffic without manipulating the checks, I'm personally not shocked,
> > because these are normally sent by bots and we don't care if they
> > have to set a few more settings to avoid multiple implicit changes
> > that may not always be desired. This is where the CLI (or any future
> > API) might differ a bit from the config, and an agent writing some
> > config might have to explicitly state certain things like "no-check-ssl"
> > for example to stay safe and avoid such implicit dependencies.
> > 
> I agree with Willy on this point. Especially because, depending the order of
> commands, the result can be different because of implicit changes and it is
> pretty hard to predict how it will behave in all cases.
> For instance, to fix William's bug about "set server ssl" command, in a way
> or another, we must stop to dynamically update the health-check if its port
> or its address is explicitly specified. With this change, the result of
> following set of commands will be different:
>   $ set server BK/SRV ssl on
>   $ set server BK/SRV check-port XXX
> ==> SSL is enabled for the server and the health-check
>   $ set server BK/SRV check-port XXX
>   $ set server BK/SRV ssl on
> ==> because the check's port was updated first, the SSL is only enabled for
> the server.
> > Note that such a brainstorming doesn't apply to your fix and should
> > not hold it from being merged in any way, I'm just speaking in the
> > general sense, as I don't feel comfortable with keep all these special
> > cases in a newly introduced API, that are only there for historical
> > reasons.
> Agree. But, if possible, a warning may be added in the documentation to warn
> about implicit changes.
> About the fix, I checked the code, and, at first glance, there is no reason
> to change "srv->check.use_ssl" value when the health-check uses the server
> settings. Thus, we may fix "set server ssl" command this way:
> diff --git a/src/check.c b/src/check.c
> index f0ae81504..8cf8a1c5b 100644
> --- a/src/check.c
> +++ b/src/check.c
> @@ -1565,10 +1565,8 @@ int init_srv_check(struct server *srv)
>  * default, unless one is specified.
>  */
> if (!srv->check.port && !is_addr(>check.addr)) {
> -   if (!srv->check.use_ssl && srv->use_ssl != -1) {
> -   srv->check.use_ssl = srv->use_ssl;
> -   srv->check.xprt= srv->xprt;
> -   }
> +   if (!srv->check.use_ssl && srv->use_ssl != -1)
> +   srv->check.xprt = srv->xprt;
> else if (srv->check.use_ssl == 1)
> srv->check.xprt = xprt_get(XPRT_SSL);
> srv->check.send_proxy |= (srv->pp_opts);
> diff --git a/src/server.c b/src/server.c
> index 2061153bc..a18836a71 100644
> --- a/src/server.c
> +++ b/src/server.c
> @@ -2113,10 +2113,11 @@ void srv_set_ssl(struct server *s, int use_ssl)
> return;
> s->use_ssl = use_ssl;
> -   if (s->use_ssl)
> -   s->xprt = xprt_get(XPRT_SSL);
> -   else
> -   s->xprt = s->check.xprt = s->agent.xprt = xprt_get(XPRT_RAW);
> +   s->xprt = (s->use_ssl == 1) ? xprt_get(XPRT_SSL) : xprt_get(XPRT_RAW);
> +
> +   if ((s->check.state & CHK_ST_CONFIGURED) && !s->check.use_ssl &&
> +   !s->check.port && !is_addr(>check.addr))
> +   s->check.xprt = s->xprt;
>  }
>  #endif /* USE_OPENSSL */
> This may be done with the following 3 steps:
>   * First, stop to change "srv->check.use_ssl" value
>   * Then, stop to change the agent settings in srv_set_ssl() because there
> is no ssl support for agent-check.
>   * Finally, fix the bug identified by William, adding the according 
> documentation.
> Note I don't know if all this stuff works properly with the server-state 
> file...
> -- 
> Christopher Faulet
> 

To add more context on this discussion, I looked at dynamic servers.
Currently, the behavior is identical with the configuration as
init_srv_check() is used in the "add server" CLI handler. This is really
problematic as "no-check-ssl" is not available for dynamic servers, so
if I understand correctly it's not possible to add a dynamic SSL server
with checks on the same port without SSL on checks.

Christopher's patch for init_srv_check() is probably a good idea, but if
not taken it should be probably at least used for servers with the
SRV_F_DYNAMIC flag.

-- 
Amaury Denoyelle



Re: [PATCH] MEDIUM numa supports for FreeBSD

2021-12-15 Thread Amaury Denoyelle
On Tue, Dec 14, 2021 at 02:12:28AM +, David CARLIER wrote:
> ping :)
> On Mon, 6 Dec 2021 at 11:07, David CARLIER  wrote:
> >
> > Hi
> >
> > Here a little patch for proper NUMA topology detection on FreeBSD.
> >
> > Thanks.
> >
> > Regards.

I have merged your patch, thanks for your contribution. Note that I have
splitted it in two to facilitate possible revert or cherry-picking of
numa_detect_topology() various implementation. I also used the BUG_ON
macro which is the idiomatic way for asserts in haproxy.

Sorry for the delay, and thanks again,

-- 
Amaury Denoyelle



Re: [EXTERNAL] Re: [PATCH] MEDIUM numa supports for FreeBSD

2021-12-14 Thread Amaury Denoyelle
On Tue, Dec 14, 2021 at 05:16:15AM +0100, Willy TARREAU wrote:
> On Tue, Dec 14, 2021 at 02:12:28AM +, David CARLIER wrote:
> > ping :)
> sorry for the delay David, we'll check today.
> Willy

I can handle this as I have already implemented the Linux part. I'm
looking at it as soon as possible.

-- 
Amaury Denoyelle



[ANNOUNCE] haproxy-2.0.26

2021-12-03 Thread Amaury Denoyelle
Hi,

HAProxy 2.0.26 was released on 2021/12/03. It added 68 new commits
after version 2.0.25.

This version contains a lot of bug fixes. One of the main area to benefit from
them is the muxers and streams infrastructure. Adjustments were made to prevent
rare occurences of blocked transfer, improper connection closing or premature
abort. Even though most users won't notice the difference, this is obviously a
critical path of the haproxy architecture, that's why it's always important to
upgrade to the latest stable in your branch.

Resolvers is another section which receive improvement. Race conditions were
fixed and the code should be more reliable. Sadly, the current architecture has
shown its limits and won't perform in the most optimal way. For users with an
important usage of resolvers, do not hesitate to have a look at the 2.5 which
have breaking changes and big progress on performance.

In the SSL area, resumption on the backend side was not functional with SNI on
TLS1.3. This is now fixed. Also, the error reporting has been improved and will
now return a proper description if a failure occurs with strict-sni.

A very subtle bug was fixed in the LUA code for the sleep() function. Most of
the time, it will run fine. However, due to an erroneous time comparison, there
is a risk to freeze the entire haproxy process when using it. However, this is
extremly rare as this can occurs only when the time is wrapping, which happens
during 1ms every 49 days.

The "block" statement in the proxy configuration was broken since the previous
release due to an incorrect refactoring and has been now restored. Note however
that this keyword is deprecated and users are encourage to use an alternative
like the "http-request deny" rule.

Some cleanups were made for samples. In some fetches such as strcmp or
secure_memcmp, variables of the improper type could cause a crash. Now a check
has been added to properly handle this case.

Thanks to everyone for this release. Enjoy !

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.0/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.0.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.0.git
   Changelog: http://www.haproxy.org/download/2.0/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/

---
Complete changelog :
Amaury Denoyelle (1):
  BUG/MINOR: server: allow 'enable health' only if check configured

Christopher Faulet (31):
  BUG/MEDIUM: stream-int: Don't block SI on a channel policy if EOI is 
reached
  Revert "REGTESTS: mark http_abortonclose as broken"
  BUG/MEDIUM: mux-h1: Adjust conditions to ask more space in the channel 
buffer
  BUG/MEDIUM: stream-int: Notify stream that the mux wants more room to 
xfer data
  BUG/MEDIUM: stream: Stop waiting for more data if SI is blocked on 
RXBLK_ROOM
  BUG/MINOR: mux-h1/mux-fcgi: Sanitize TE header to only send "trailers"
  BUG/MINOR: tcp-rules: Stop content rules eval on read error and 
end-of-input
  BUG/MINOR: stream: Don't release a stream if FLT_END is still registered
  BUG/MEDIUM: http-ana: Reset channels analysers when returning an error
  BUG/MINOR: filters: Always set FLT_END analyser when CF_FLT_ANALYZE flag 
is set
  BUG/MINOR: filters: Set right FLT_END analyser depending on channel
  BUG/MEDIUM: filters: Fix a typo when a filter is attached blocking the 
release
  BUG/MEDIUM: http-ana: Clear request analyzers when applying redirect rule
  BUG/MEDIUM: mux_h2: Handle others remaining read0 cases on partial frames
  BUG/MEDIUM: stream: Keep FLT_END analyzers if a stream detects a channel 
error
  BUG/MINOR: mux-h1: Save shutdown mode if the shutdown is delayed
  BUG/MEDIUM: mux-h1: Perform a connection shutdown when the h1c is released
  BUG/MEDIUM: http-ana: Drain request data waiting the tarpit timeout 
expiration
  DOC: config: Fix alphabetical order of fc_* samples
  MINOR: stream: Improve dump of bogus streams
  BUG/MINOR: tcpcheck: Improve LDAP response parsing to fix LDAP check
  MINOR: htx: Add an HTX flag to know when a message is fragmented
  MINOR: htx: Add a function to know if the free space wraps
  BUG/MEDIUM: stream-int: Defrag HTX message in si_cs_recv() if necessary
  BUG/MEDIUM: mux-h1: Fix H1C_F_ST_SILENT_SHUT value
  DOC: config: Fix typo in ssl_fc_unique_id description
  BUG/MINOR: http-ana: Apply stop to the current section for http-response 
rules
  BUG/MEDIUM: conn-stream: Don't reset CS flags on close
  BUG/MINOR: mux-h2: Fix H2_CF_DEM_SHORT_READ value
  BUG/MINOR: sti

Re: [ANNOUNCE] haproxy-2.4.9

2021-11-25 Thread Amaury Denoyelle
On Thu, Nov 25, 2021 at 11:42:01AM +0300, Dmitry Sivachenko wrote:
> On 24 Nov 2021, at 12:57, Christopher Faulet  wrote:
> > > > Hi,
> > > HAProxy 2.4.9 was released on 2021/11/23. It added 36 new commits
> > after version 2.4.8.
> > 
> Hello,
> version 2.4.9 fails to build with OpenSSL turned off:
>  src/server.c:207:51: error: no member named 'ssl_ctx' in 'struct server'
> if (srv->mux_proto || srv->use_ssl != 1 || !srv->ssl_ctx.alpn_str) {
> ~~~  ^
> src/server.c:241:37: error: no member named 'ssl_ctx' in 'struct server'
> const struct ist alpn = ist2(srv->ssl_ctx.alpn_str,
>  ~~~  ^
> src/server.c:242:37: error: no member named 'ssl_ctx' in 'struct server'
>  srv->ssl_ctx.alpn_len);
>  ~~~  ^
> Version 2.4.8 builds fine.
> 
> 

Dmitry, the patches that Willy provided you should fix the issue. Now,
do you need a 2.4.10 to be emitted early with it or is it possible for
you to keep the patches in your tree so we can have a more substantial
list of change for a new version ?

-- 
Amaury Denoyelle



Re: [ANNOUNCE] haproxy-2.4.9

2021-11-25 Thread Amaury Denoyelle
On Thu, Nov 25, 2021 at 11:42:01AM +0300, Dmitry Sivachenko wrote:
> On 24 Nov 2021, at 12:57, Christopher Faulet  wrote:
> > > > Hi,
> > > HAProxy 2.4.9 was released on 2021/11/23. It added 36 new commits
> > after version 2.4.8.
> > 
> Hello,
> version 2.4.9 fails to build with OpenSSL turned off:
>  src/server.c:207:51: error: no member named 'ssl_ctx' in 'struct server'
> if (srv->mux_proto || srv->use_ssl != 1 || !srv->ssl_ctx.alpn_str) {
> ~~~  ^
> src/server.c:241:37: error: no member named 'ssl_ctx' in 'struct server'
> const struct ist alpn = ist2(srv->ssl_ctx.alpn_str,
>  ~~~  ^
> src/server.c:242:37: error: no member named 'ssl_ctx' in 'struct server'
>  srv->ssl_ctx.alpn_len);
>  ~~~  ^
> Version 2.4.8 builds fine.
> 
> 

Thanks for your report. One of my commit to handle properly websocket on
the server side introduces this issue. I'm working on a fix.

-- 
Amaury Denoyelle



Re: [PATCH 1/2] BUILD: SSL: add quictls build to scripts/build-ssl.sh

2021-11-18 Thread Amaury Denoyelle
On Thu, Nov 18, 2021 at 06:27:56PM +0500, Ilya Shipitsin wrote:
> script/build-ssl.sh is used mostly in CI, let us introduce QUIC
> OpenSSL fork support
> ---
>  scripts/build-ssl.sh | 23 +++
>  1 file changed, 23 insertions(+)
> diff --git a/scripts/build-ssl.sh b/scripts/build-ssl.sh
> index e1d89a0eb..d143cec55 100755
> --- a/scripts/build-ssl.sh
> +++ b/scripts/build-ssl.sh
> @@ -86,6 +86,17 @@ download_boringssl () {
>  fi
>  }
>  
> +download_quictls () {
> +if [ ! -d "download-cache/quictls" ]; then
> +git clone --depth=1 https://github.com/quictls/openssl 
> download-cache/quictls
> +else
> +   (
> +cd download-cache/quictls
> +git pull
> +   )
> +fi
> +}
> +
>  if [ ! -z ${LIBRESSL_VERSION+x} ]; then
>   download_libressl
>   build_libressl
> @@ -121,3 +132,15 @@ if [ ! -z ${BORINGSSL+x} ]; then
>   )
>  fi
>  
> +if [ ! -z ${QUICTLS+x} ]; then
> +(
> +
> +download_quictls
> +cd download-cache/quictls
> +
> +./config shared --prefix="${HOME}/opt" --openssldir="${HOME}/opt" 
> --libdir=lib -DPURIFY
> +make -j$(nproc) build_sw
> +make install_sw
> +
> +)
> +fi
> -- 
> 2.29.2.windows.2
> 

Thank you for your patches. However, we should probably fix the GCC
error reproduced on your repository before merging it. This will prevent
to have a red test indicator indefinitely.

-- 
Amaury Denoyelle



Re: [EXTERNAL] [PATCH] BUILD/MINOR: cpuset Freebsd build fix

2021-11-02 Thread Amaury Denoyelle
On Sat, Oct 30, 2021 at 12:25:13PM +0100, David CARLIER wrote:
> Hi here a patch to address freebsd build failure.
> Kind regards.

Thanks, applied.

-- 
Amaury Denoyelle



Re: [EXTERNAL] [PATCH] BUILD/MINOR: healthcheck netbsd build fix

2021-10-21 Thread Amaury Denoyelle
On Wed, Oct 20, 2021 at 07:03:43PM +0100, David CARLIER wrote:
> Hi
> here a little patch for netbsd system.
> Kind regards.

Thanks, I'm currently trying to fix other issues with NetBSD9.2, so I'll
merge your patch with my work.

-- 
Amaury Denoyelle



Re: [EXTERNAL] testing websockets

2021-10-20 Thread Amaury Denoyelle
On Wed, Oct 20, 2021 at 09:59:59AM +0500, Илья Шипицин wrote:
> Hello,
> I've found a way how to test websockets automatically.
> this approach is able to catch https://github.com/haproxy/haproxy/issues/737
> 
> the idea is to place haproxy between browser and kestrel in SignalR tests (
> https://github.com/dotnet/aspnetcore/tree/main/src/SignalR )
> test takes 3 hours on my low end laptop.
> I collected all required parts in
> https://github.com/chipitsine/haproxy-signalr-tests
> also, I checked current maintained branches, no issues (last issue was
> fixed in issue 737).

Hi,

I did not look precisely at your testsuite, but note that now we have
some websockets related VTC regtests. Of course, this does only test the
tunnel establishment without websocket data exchange, but it's still a
good start. Also this will be soon extended with a new test to better
handle h2 websocket.

Regards,

-- 
Amaury Denoyelle



Re: haproxy hung with CPU usage at 100% Heeeelp, please!!!

2021-05-19 Thread Amaury Denoyelle
Oriol Magrane  wrote:

> Amaury,
> Your patch seems to have killed the bug: our newly recompiled haproxy is
> running for more than 24h now without glitches (while the unpatched
> versions never got past the 4h hour).
> I'll let it run for 24 more hours at full load just in case, but things are
> looking good and I don't expect to have to come back here crying.
> So thank you very much for your help!

Great news ! I will merge the fix soon in the dev branch and it should
hit the next serie of stable releases. Thank you again for your report
and your feedback.

-- 
Amaury Denoyelle



Re: haproxy hung with CPU usage at 100% Heeeelp, please!!!

2021-05-17 Thread Amaury Denoyelle
0\000\000\000\000P\000\000\000\000\000\000\000\210\334\377\377\377\377\377\377\000\000\000\000\000\000\000\000\003\000\000\000\060",
> '\000' , "[\000\000\000n\000\000\000|\000\000\000w",
> '\000' ,
> "\360\006\236\f\335\177\000\000@\274\235\f\335\177\000\000
> \000\000"
> pidfd = 
> ... so the server HA_SPIN_LOCK() in line 177 of queue.c seems to be the
> culprit.
> At this point, we don’t know what we can do in order to get rid of this
> problem. We tried with the new haproxy series 2.x and the issue is still
> there; the only difference is the haproxy process doesn’t hang, but it gets
> restarted instead (although I don’t know if the restart is systemd’s
> action).
> Any idea about what could we do?
> Thank you very much in advance!
> Oriol
> P.S. I don’t attach logs or system config to keep this first post simple,
> but if anyone wants to give them a look I’ll be happy to share; so just
> ask, please

Thanks for your report.

A deadlock may indeed arise when using 'set maxconn server' if the
function process_srv_queue is called, which is not always the case. I
had a quick look and the issue seems to be present in the latest version
as well. I will continue to inspect the problem in details.

As for the restart in the 2.x, it is probably due to the internal
haproxy watchdog which was added in the 2.0 version and is specifically
designed to detect this type of issues.

-- 
Amaury Denoyelle



[ANNOUNCE] haproxy-1.8.30

2021-04-12 Thread Amaury Denoyelle
Hi,

HAProxy 1.8.30 was released on 2021/04/12. It added 8 new commits after
version 1.8.29. The main point of this release is the fix of the
regression on the frequency counters introduced in the latest release.
Thanks to users feedback, the bug was quickly spotted and the fix
validated in higher version before being backported in the 1.8 tree.
Here is the list of the detailed changes.

- As stated, the bug on the frequency counters is fixed. Since the
  previous release, the time period was not properly calculated, so the
  counters were not properly updated. Willy was able to fix the issue
  thanks to quick user reports.

- The hdr_ip sample fetch is now stricter. It will reject a field if
  there is some garbage after a valid IPv4 address. This ensures that
  for example an invalid x-forwared-for header field is not present,
  which is better to detect a non-conformant http proxy in a network.

- The silent-drop action was not functional for IPv6 connections if the
  haproxy process is executed without admin capabilities. It now
  properly set the IPv6 header field hop-limit to 1, as explained in the
  documentation.

Thanks to everyone for this release. Enjoy !

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/1.8/src/
   Git repository   : http://git.haproxy.org/git/haproxy-1.8.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-1.8.git
   Changelog: http://www.haproxy.org/download/1.8/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/

Amaury Denoyelle
---
Complete changelog :
Willy Tarreau (8):
  MINOR: time: also provide a global, monotonic global_now_ms timer
  BUG/MEDIUM: freq_ctr/threads: use the global_now_ms variable
  BUG/MEDIUM: time: make sure to always initialize the global tick
  MINOR: tools: make url2ipv4 return the exact number of bytes parsed
  BUG/MINOR: http_fetch: make hdr_ip() reject trailing characters
  BUG/MINOR: tcp: fix silent-drop workaround for IPv6
  BUILD: tcp: use IPPROTO_IPV6 instead of SOL_IPV6 on FreeBSD/MacOS
  BUG/MINOR: http_fetch: make hdr_ip() resistant to empty fields

---



[ANNOUNCE] haproxy-2.0.22

2021-04-12 Thread Amaury Denoyelle
Hi,

HAProxy 2.0.22 was released on 2021/04/12. It added 23 new commits after
version 2.0.21. Most notably, this release fixes a regression affecting
the frequency counters. Thanks to quick users feedback, the bug was
quickly spotted and the fix validated in higher version before being
backported in the 2.0 tree. There were also important fixes in the DNS
module due to recent changes. Here is the list of all the changes.

- Since the previous release, the time period of frequency counters was
  not properly calculated, and so the counters not correctly updated.
  Willy was able to fix the issue quickly thanks to quick user reports.

- As mentionned, a bunch of fixes were made on the DNS resolvers. The
  server managed through the SRV records additional sections were not
  properly handled, causing servers in MAINT status to never be
  activated when becoming available. Also the handling of resolution
  errors is now safer.

- The hdr_ip sample fetch is now stricter. It will reject a field if
  there is some garbage after a valid IPv4 address. This ensures that
  for example an invalid x-forwared-for header field is not present,
  which is better to detect a non-conformant http proxy in a network.

- The silent-drop action was not functional for IPv6 connections if the
  haproxy process is executed without admin capabilities. It now
  properly set the IPv6 header field hop-limit to 1, as explained in the
  documentation.

- The Lua debugging has been slightly improved by Christopher with the
  implementation of an internal function to display the backtrace in
  case of failure.  This allows to output the backtrace even if a memory
  allocation failure is the cause of the bug.

- The closing of an H1 connection is now idempotent. This prevents a
  rare occurence of a crash when closing an already closed H1
  connection.

- On the html stats page, a DOWN backend in transition to the UP state
  was incorrectly displayed with the wrong color, making it
  indistinguishable with going DOWN backends.

- The unix-bind-prefix directive was incorrectly prepended to the UNIX
  socket path.

- A deadlock was fixed for process built with DEBUG_UAF when using
  thread isolation. This option is normally only activated for debugging
  purposes to detect use-after-free problems.

Thanks to everyone for this release. Enjoy !

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/2.0/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.0.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.0.git
   Changelog: http://www.haproxy.org/download/2.0/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/

Amaury Denoyelle
---
Complete changelog :
Baptiste Assmann (1):
  BUG/MAJOR: dns: disabled servers through SRV records never recover

Christopher Faulet (10):
  MINOR: lua: Slightly improve function dumping the lua traceback
  BUG/MEDIUM: debug/lua: Use internal hlua function to dump the lua 
traceback
  BUG/MEDIUM: lua: Always init the lua stack before referencing the context
  BUG/MEDIUM: thread: Fix a deadlock if an isolated thread is marked as 
harmless
  BUG/MINOR: resolvers: Unlink DNS resolution to set RMAINT on SRV 
resolution
  MINOR: resolvers: Use a function to remove answers attached to a 
resolution
  MINOR: resolvers: Purge answer items when a SRV resolution triggers an 
error
  MINOR: resolvers: Add function to change the srv status based on SRV 
resolution
  MINOR: resolvers: Directly call srvrq_update_srv_state() when possible
  BUG/MEDIUM: resolvers: Don't release resolution from a requester callbacks

Eric Salama (1):
  MINOR/BUG: mworker/cli: do not use the unix_bind prefix for the master 
CLI socket

Florian Apolloner (1):
  BUG/MINOR: stats: Apply proper styles in HTML status page.

Jerome Magnin (1):
  BUG/MAJOR: dns: fix null pointer dereference in snr_update_srv_status

Willy Tarreau (9):
  MINOR: time: also provide a global, monotonic global_now_ms timer
  BUG/MEDIUM: freq_ctr/threads: use the global_now_ms variable
  BUG/MEDIUM: time: make sure to always initialize the global tick
  MINOR: tools: make url2ipv4 return the exact number of bytes parsed
  BUG/MINOR: http_fetch: make hdr_ip() reject trailing characters
  BUG/MEDIUM: mux-h1: make h1_shutw_conn() idempotent
  BUG/MINOR: tcp: fix silent-drop workaround for IPv6
  BUILD: tcp: use IPPROTO_IPV6 instead of SOL_IPV6 on FreeBSD/MacOS
  BUG/MINOR: http_fetch: make hdr_ip() resistant to empty fields

---



Re: Request for new 1.8.x release due to freq counter bug

2021-04-09 Thread Amaury Denoyelle
"Robin H. Johnson"  wrote:

> Hi,
> Wondering if you could make a new 1.8.x release to get the fix for the
> freq counter bug into more systems?
> dde80e111 BUG/MEDIUM: time: make sure to always initialize the global tick
> 491b86ed0 BUG/MEDIUM: freq_ctr/threads: use the global_now_ms variable
> It's a problem for anybody using counters for ratelimiting purposes, who
> can't yet upgrade to 2.x series due to other issues encountered there
> (the 100% cpu bug even on the latest 2.3.9, that I have logs I'm hoping
> to post when they are approved by $work)

Hi,

I will try to emit new releases next week for 2.0 and 1.8 which are
impacted by the bug you mentionned.

-- 
Amaury Denoyelle



[ANNOUNCE] haproxy-1.8.29

2021-03-19 Thread Amaury Denoyelle
Hi,

HAProxy 1.8.29 was released on 2021/03/19. It added 40 new commits
after version 1.8.28.

As again, this is mainly a release with a collection of fixes which
improve the overall quality and stability of haproxy. Here is a list of
the most notables changes :

- William D. and Christopher give special care on the server-state file.
  Some bugs related to the servers reload from the state file are
  corrected. Most notably, the RMAINT dynamic state is now ignored if
  present in the state file. This prevents an error to be reported and
  the server to not be updated. Also, it is now possible in the
  configuration file to use the server-state directive without a
  filename. In this case, a file with the backend name will
  automatically be used as the state file. In fact, this feature was
  already documented but not functional.

- The SPOE module receives fixes. First, a crash caused by dangling
  references on released applets is now prevented. It could have
  happened on unusual various occasions, one of them being during a
  reload. Second, after receiving an incomplete frame, the SPOE stream
  processing was stucked. Now the stream is properly woken up to process
  the remaining data. Finally, some adjustments have been made to
  process streams smoother when using SPOE with multithread, especially
  with a low maxconn on the server.

- The DNS responses are matched with case-insensitive functions. This
  prevent resolution failures for the uncommon case of having uppercases
  characters in names of DNS queries.

- It is now possible to use ipv6 connections for servers with the
  special address 0.0.0.0 to reuse the client source address as a
  transparent proxy.

- A bug was fixed by Willy in the configuration parser. The queue and
  tarpit timeouts were applied from the last default section if not
  defined instead of the default section before the proxy definition.
  This might have resulted in incorrect timeout values. This bug is 13
  years old so I think it deserves a special note here :)

- The processing of the originalto option has been adjusted. Now the
  destination address is correctly tested against the except statement.
  Please be careful because the doc was also wrong on this point,
  mentioning the source instead of the destination address. It has been
  edited, so this feature is now consistent.

- There was an issue with the set-dst tcp rule. The destination tcp port
  was not correctly fixed for an ipv4 address. However its impact was
  limited on UNIX sockets.

- Christopher wrote a fix for the filters. This ensure that the filters
  will be called on the response after the request, even if the analyzis
  on the request is finished before the start of the response filtering.

- The sample fetch functions are safer and handle properly an empty
  string buffer value. Before this, it caused a memory corruption
  problem by reading bytes outside of the sample value.

- Data races could happened on hard-stop or shutdown-sessions. These
  instructions are now only executed when no other threads are running
  to enforce the thread-safety.

- Willy has fixed a data race on frequency counters, causing a
  non-monotonically updates of them. At the same occasion, an
  optimization has been backported. The counters are now incremented
  atomically without a lock. This reduces the contention on them, which
  should improve the performance as they are often used as proxies
  counters.

- The sticky counter of the sessions was also subject to a non-thread
  safe incrementation and could be slightly off on time. An atomic
  operation is now used to guarantee a correct value.

- A possible crash was averted by William when manipulating listener
  counters with an applet as a client (peers, SPOE or Lua). In this
  case, the listener instance is null, thus it cannot be dereferenced.
  However, these instructions are limited to specific scopes, so most of
  the time it won't happened for an applet.

Thanks to everyone for this release. Enjoy !

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/1.8/src/
   Git repository   : http://git.haproxy.org/git/haproxy-1.8.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-1.8.git
   Changelog: http://www.haproxy.org/download/1.8/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/

---
Complete changelog :
Amaury Denoyelle (2):
  BUG/MINOR: config: fix leak on proxy.conn_src.bind_hdr_name
  CLEANUP: remove unused src/cfgparse-listen.c

Bertrand Jacquin (1):
  BUILD/MINOR: lua: define _GNU_SOURCE for LLONG_MAX

Christopher Faulet (20):
  BUG/MINOR: stick-table: Always call smp_fetch_src

Re: Proxy protocol unique id and idle connection

2021-01-22 Thread Amaury Denoyelle
Tim Düsterhus  wrote:

> Amaury,
> Am 21.01.21 um 16:40 schrieb (Amaury Denoyelle):
> > I have a question for you on the case of the proxy protocol. One of
> > these special parameters to identify a connection is the content of the
> > proxy protocol block. However, this breaks the following reg-tests :
> > - proxy_protocol_send_unique_id.vtc (from commit
> >   cf6e0c8a836da211946fa637020e776286093633)
> > What happens in the reg-test with my modification is that on the second
> > client request, a new proxy protocol packet for connection comparison is
> > generated with the header of the second request as unique id. It is
> > identified that the previous connection cannot be reused (different
> > unique id in proxy protocol), and thus a new connection is established.
> > However, on the reg-tests, it is expected that the first connection is
> > reused and thus no proxy protocol resend.
> > What is your opinion on this ? On my side, I think the behavior of my
> > implementation is less surprising, but if it is a real use-case, I do
> > not want to break it without knowing it.
> Sending unique IDs via PROXY v2 was implemented for TCP proxying. The
> documentation already states that:
> It can
> lead to unexpected results in "mode http", because the
> generated unique ID is also used for the first HTTP request
> within a Keep-Alive connection.
> So:
> 1. No one should send unique IDs via PROXY v2 for HTTP traffic. There's
> headers for that.
> 2. It didn't make sense to actually prevent users from doing that.
> 3. Preventing sharing for such connections does not actually *break*
> anything, instead it just reduces performance.
> 4. With the reduced performance we are back to (1): You really shouldn't
> do this.
> 5. The test was just to make sure it behaves somewhat sanely (i.e. does
> not send entirely wrong information) for HTTP traffic. My intention was
> not to test that Keep-Alive is possible.
> So: Please go ahead with your modification, change the test to make sure
> keep-alive does not happen and adjust the documentation as necessary.

Hi Tim,

Thanks for your quick answer. I will continue my work and adjust the
test and the doc.

-- 
Amaury Denoyelle



Proxy protocol unique id and idle connection

2021-01-21 Thread Amaury Denoyelle
Hi Tim,

I'm currently doing a major rework of idle connection management on the
server side of haproxy. The goal is to be able to reuse connection that
are private today, such as the ones with SNI or proxy protocol. Every
server connection will be differentiated by a set of parameters to be
able to reuse it only if the new request shares the same parameters.

I have a question for you on the case of the proxy protocol. One of
these special parameters to identify a connection is the content of the
proxy protocol block. However, this breaks the following reg-tests :

- proxy_protocol_send_unique_id.vtc (from commit
  cf6e0c8a836da211946fa637020e776286093633)

What happens in the reg-test with my modification is that on the second
client request, a new proxy protocol packet for connection comparison is
generated with the header of the second request as unique id. It is
identified that the previous connection cannot be reused (different
unique id in proxy protocol), and thus a new connection is established.
However, on the reg-tests, it is expected that the first connection is
reused and thus no proxy protocol resend.

What is your opinion on this ? On my side, I think the behavior of my
implementation is less surprising, but if it is a real use-case, I do
not want to break it without knowing it.

Let me know if I'm not clear,

I'm putting the list on copy in case someone has an opinion on the
subject.

Thank you for your help,

-- 
Amaury Denoyelle



[ANNOUNCE] haproxy-1.8.28

2021-01-13 Thread Amaury Denoyelle
Hi,

HAProxy 1.8.28 was released on 2021/01/13. It added 33 new commits after
version 1.8.27. This new release is relatively light and contains only a
small set of fixes. Of particular interest are the following ones:

- A race condition has been fixed by Christopher affecting the leastconn
  load-balancer algorithm. A crash could occur due to a division by zero
  with the weight parameter. Thanks to Peter Statham for its help on the
  issue.

- The parser for haproxy command-line arguments was behaving incorrectly
  when using an argument value starting with a dash. On reload, the
  process was restarted with a wrong set of arguments. In fact, this
  should have been fixed in the earlier version but the backported
  commit was incomplete in the 1.8 branch.

- Olivier has fixed a problem affecting users of ARM-based Macs, whose
  assembler takes the semi-colon as a comment instead of an instruction
  delimiter, causing the CAS to fail and loop forever.

- Maciej has fixed the sample fetches to retrieve messages cookies when
  called without cookie name. This case was never properly handled,
  contrary to what the doc says.

- A set of minor patches from Thierry were applied to the Lua code. In
  particular, new warnings will be reported when trying to register
  action, converter, sample fetch, cli or applet with a name already
  used.

- A small fix was written by Christopher on SPOE. The connection is not
  closed anymore when receiving an ACK even if no frame is waiting an
  acknowledge.

- The time parser is now more restrictive on invalid values. Please
  update your configuration if you spot new warnings about this.

- The extra cookie attributes defined in the default section were not
  taken into account before this release. Now they are properly applied
  on each proxy instance.

- A crash could happen on startup when using in the configuration a
  disabled backend with specific server parameters.

And as usual there is a bunch of cleanup/typo and doc commits, not visible on
first sight, but they are always welcome as they improve the overall quality of
haproxy software.

Thanks to everyone for this release. Enjoy !

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/1.8/src/
   Git repository   : http://git.haproxy.org/git/haproxy-1.8.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-1.8.git
   Changelog: http://www.haproxy.org/download/1.8/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/

Amaury Denoyelle
---
Complete changelog :
Amaury Denoyelle (2):
  BUG/MINOR: config: copy extra cookie attributes from dfl proxy
  BUG/MINOR: srv: do not init address if backend is disabled

Christian Ruppert (1):
  BUILD: hpack: hpack-tbl-t.h uses VAR_ARRAY but does not include compiler.h

Christopher Faulet (7):
  BUG/MINOR: http-fetch: Fix calls w/o parentheses of the cookie sample 
fetches
  MINOR: spoe: Don't close connection in sync mode on processing timeout
  DOC: config: Move req.hdrs and req.hdrs_bin in L7 samples fetches section
  BUG/MINOR: tools: make parse_time_err() more strict on the timer validity
  BUG/MINOR: tools: Reject size format not starting by a digit
  BUG/MEDIUM: lb-leastconn: Reposition a server using the right eweight
  CLEANUP: lua: Remove declaration of an inexistant function

David Carlier (1):
  DOC: email change of the DeviceAtlas maintainer

Maciej Zdeb (1):
  BUG/MINOR: http-fetch: Extract cookie value even when no cookie name

Olivier Houchard (1):
  MINOR: atomic: don't use ; to separate instruction on aarch64.

Phil Scherer (1):
  DOC/MINOR: Fix formatting in Management Guide

Thayne McCombs (2):
  DOC: fix some spelling issues over multiple files
  SCRIPTS: announce-release: fix typo in help message

Thierry Fournier (7):
  BUG/MINOR: lua: lua-load doesn't check its parameters
  BUG/MINOR: lua: Post init register function are not executed beyond the 
first one
  BUG/MINOR: lua: Some lua init operation are processed unsafe
  MINOR: actions: Export actions lookup functions
  MINOR: actions: add a function returning a service pointer from its name
  MINOR: cli: add a function to look up a CLI service description
  BUG/MINOR: lua: warn when registering action, conv, sf, cli or applet 
multiple times

Tim Duesterhus (1):
  BUG/MINOR: cfgparse: Fail if the strdup() for `rule->be.name` for 
`use_backend` fails

William Lallemand (1):
  BUG/MEDIUM: mworker: fix again copy_argv()

Willy Tarreau (8):
  CLEANUP: stream: remove an obsolete debugging test
  BUILD: Makefile: have "make clean" destroy .o/.a/.s in co

Re: Crash when using wlc in multithreaded mode with agent checks (1.8.26).

2021-01-12 Thread Amaury Denoyelle
Mark Brookes  wrote:

> Sorry to drag up an old thread. But do we have an eta for a new
> release of version 1.8 that contains the fix? I noticed that the 2.x
> versions have been updated, and wanted to make sure that 1.8 has not
> been left out by mistake.

I have just updated the backports for haproxy 1.8, I will try to make a
release with your fix soon.

-- 
Amaury Denoyelle



Re: [PR] Fixed null pointer dereference in srv_cleanup_connections()

2021-01-06 Thread Amaury Denoyelle
PR Bot  wrote:

> Dear list!
> Author: Peter Skarpetis 
> Number of patches: 1
> This is an automated relay of the Github pull request:
>Fixed null pointer dereference in srv_cleanup_connections()
> Patch title(s): 
>Fixed null pointer dereference in srv_cleanup_connections()
> Link:
>https://github.com/haproxy/haproxy/pull/1031
> Edit locally:
>wget https://github.com/haproxy/haproxy/pull/1031.patch && vi 1031.patch
> Apply locally:
>curl https://github.com/haproxy/haproxy/pull/1031.patch | git am -
> Description:
>haproxy_srv_cleanup_connections_crash.cfg causes a null pointer
>dereference in srv_cleanup_connections . Configuration file works in
>2.0.19  branch but crashes in all subsequent versions including the
>dev branch. I did not track down the cause, I just added the null
>pointer check to stop the crashing.
>
>Crash can be reproduced
>with the following command:
>./haproxy -c -f
>haproxy_srv_cleanup_connections_crash.cfg
>haproxy_srv_cleanup_connections_crash.cfg  can be grabbed from the
>gist below:
>https://gist.github.com/peterska/769f41562f6b045df59df
>2294b2c20f0#file-haproxy_srv_cleanup_connections_crash-cfg
>configuration file has been edited enough to cause the crash while
>removing all references to certificates and CA authorities. It is not
>a production config file.
> Instructions:
>This github pull request will be closed automatically; patch should be
>reviewed on the haproxy mailing list (haproxy@formilux.org). Everyone is
>invited to comment, even the patch's author. Please keep the author and
>list CCed in replies. Please note that in absence of any response this
>pull request will be lost.

I can reproduce your issue on haproxy master. However I'm not sure why
idle_conns list is NULL in this case so I'll investigate it before
validating your patch.

-- 
Amaury Denoyelle



[ANNOUNCE] haproxy-1.8.27

2020-11-06 Thread Amaury Denoyelle
Hi,

HAProxy 1.8.27 was released on 2020/11/06. It added 44 new commits after
version 1.8.26. Every 1.8 users are encouraged to upgrade as it
contains several bug fixes.

This release contains some fixes also present on higher versions. Most
notably a fix on the h2 multiplexer, a thread-safety bug on
load-balancer algorithms, a design issue on SPOE and the skipping of
disabled proxies for filters. Also there is the possibility now to
update the server-state file without crashing haproxy. You can look the
detailled reports for them on 2.2.5 announce.

In addition, the following changes have been made :

The h2 multiplexer is more robust thanks to Christopher and Willy. First
it is now able to parse incomplete chunk formatting. An issue has also
been raised due to a certain combination of frame type and flags which
haproxy interprets wrongly as invalid and is now fixed.

Some improvement on the ssl have been made by William. Notably, a better
algorithm to choose a certificate when using wildcards with respect to
the supported encryption algorithm.

The lua engine now prevents to load map at runtime, which should never
have been permitted. There is also additional checks on arguments when
doing ip address manipulation.

A nasty bug has fixed by Willy, triggered when using multi processes
with expose-fd. In short, the "disable frontend" command had the
side-effect of pausing other process socket file-descriptors. Now this
case is properly handled, the other listeners are not impacted.

There is also a list of smaller fixes. Again, look at 2.2.5 announce that
summarizes them for more info.

Thanks to everyone for this release. Enjoy !

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Wiki : https://github.com/haproxy/wiki/wiki
   Sources  : http://www.haproxy.org/download/1.8/src/
   Git repository   : http://git.haproxy.org/git/haproxy-1.8.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-1.8.git
   Changelog: http://www.haproxy.org/download/1.8/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/

---
Complete changelog :
Amaury Denoyelle (6):
  BUG/MINOR: config: Fix memory leak on config parse listen
  MINOR: counters: fix a typo in comment
  BUG/MINOR: stats: fix validity of the json schema
  BUG/MINOR: server: fix srv downtime calcul on starting
  BUG/MINOR: server: fix down_time report for stats
  BUG/MINOR: lua: initialize sample before using it

Christopher Faulet (13):
  BUG/MEDIUM: mux-h2: Don't fail if nothing is parsed for a legacy chunk 
response
  BUG/MEDIUM: map/lua: Return an error if a map is loaded during runtime
  BUG/MINOR: lua: Check argument type to convert it to IPv4/IPv6 arg 
validation
  BUG/MINOR: lua: Check argument type to convert it to IP mask in arg 
validation
  BUG/MEDIUM: pattern: Renew the pattern expression revision when it is 
pruned
  MINOR: hlua: Display debug messages on stderr only in debug mode
  BUG/MEDIUM: spoe: Unset variable instead of set it if no data provided
  BUG/MEDIUM: lb: Always lock the server when calling 
server_{take,drop}_conn
  BUG/MAJOR: mux-h2: Don't try to send data if we know it is no longer 
possible
  BUG/MEDIUM: filters: Don't try to init filters for disabled proxies
  BUG/MINOR: server: Set server without addr but with dns in RMAINT on 
startup
  MINOR: server: Copy configuration file and line for server templates
  BUG/MINOR: filters: Skip disabled proxies during startup only

Dragan Dosen (1):
  BUG/MEDIUM: pattern: fix memory leak in regex pattern functions

Lukas Tribus (1):
  BUG/MINOR: dns: ignore trailing dot

Remi Tricot-Le Breton (1):
  BUG/MINOR: cache: Inverted variables in http_calc_maxage function

Tim Duesterhus (2):
  MINOR: Commit .gitattributes
  CLEANUP: Update .gitignore

William Dauchy (1):
  DOC: agent-check: fix typo in "fail" word expected reply

William Lallemand (5):
  BUG/MINOR: startup: haproxy -s cause 100% cpu
  BUG/MEDIUM: ssl: check OCSP calloc in ssl_sock_load_ocsp()
  BUG/MEDIUM: ssl: does not look for all SNIs before chosing a certificate
  BUG/MINOR: ssl: verifyhost is case sensitive
  DOC: ssl: crt-list negative filters are only a hint

Willy Tarreau (14):
  BUG/MINOR: stats: use strncmp() instead of memcmp() on health states
  BUG/MINOR: reload: do not fail when no socket is sent
  BUG/MINOR: threads: work around a libgcc_s issue with chrooting
  BUILD: thread: limit the libgcc_s workaround to glibc only
  BUILD: threads: better workaround for late loading of libgcc_s
  BUG/MEDIUM: h2: report frame bits only for handled types
  BUG/MEDIUM: listeners: do not pause foreign listeners
 

Re: Easy but useful feature missing, anyone interested ?

2020-09-30 Thread Amaury Denoyelle
Willy Tarreau  wrote:

> Hi all,
> while revisiting pending issues, I've come across this one, about
> the impossibility for an environment variable to produce multiple
> words in the configuration:
> https://github.com/haproxy/haproxy/issues/165
> It can be trivially addressed by adding support for ${VAR[*]} to the
> config language. This addition is relatively simple to do, roughly
> speaking, simply add the code in parse_line() in the "if" block dealing
> with "$", detect the presence of '[', terminate the variable name here,
> raise a flag, then once the variable is resolved, increase arg for each
> space found.
> We've missed it for 2.2 already, it would be nice if someone interested
> in this feature could have a look at it before we release 2.3. I'm OK
> with merging it slightly late given that the side effects are quickly
> tested.

FIY, I have looked upon the issue and seems to have now a working
implementation. I will probably be able to produce a clean patch soon.

-- 
Amaury Denoyelle



Re: Easy but useful feature missing, anyone interested ?

2020-09-30 Thread Amaury Denoyelle
Willy Tarreau  wrote:

> Hi all,
> while revisiting pending issues, I've come across this one, about
> the impossibility for an environment variable to produce multiple
> words in the configuration:
> https://github.com/haproxy/haproxy/issues/165
> It can be trivially addressed by adding support for ${VAR[*]} to the
> config language. This addition is relatively simple to do, roughly
> speaking, simply add the code in parse_line() in the "if" block dealing
> with "$", detect the presence of '[', terminate the variable name here,
> raise a flag, then once the variable is resolved, increase arg for each
> space found.
> We've missed it for 2.2 already, it would be nice if someone interested
> in this feature could have a look at it before we release 2.3. I'm OK
> with merging it slightly late given that the side effects are quickly
> tested.
> In the same vein, I've long been saying that we're missing an ifdef
> mechanism, but if we had $(if,cond,true,false) like makefiles, we could
> already do a lot, particularly in the regtests. There's quite more work
> here (define expressions for the condition, internal variable names for
> version and builtin options, recursive resolving of variable names), but
> it's likely that the same person(s) might be interested.
> If someone's interested in having a look at that, please let me know.
> There's no shame in trying and not succeeding, don't worry :-)

At the moment I'm always interested to add simple feature, but I also
would like to progress with my patch serie on stats so I'm not sure if I
will have the time for this :/

-- 
Amaury Denoyelle



Statistics improvement - usage for the json schema

2020-09-16 Thread Amaury Denoyelle
Hi,

I'm currently working on extending the statistisc capabilities for
HAProxy. My first goal is to design a small API to be able to quickly
add new counters.

I have a question about the cli command "show schema json". I am not a
json expert and do not know precisely the purpose of a json schema.
However, it seems that the currently generated schema is not valid, at
least by checking on this website :
https://www.jsonschemavalidator.net/

Is the json schema useful to anyone or can I safely ignore/remove it ?

Thanks for your feedback,

-- 
Amaury Denoyelle