[ANNOUNCE] haproxy-2.7-dev6

2022-09-17 Thread Willy Tarreau
  BUG/MEDIUM: sink: bad init sequence on tcp sink from a ring.

Erwan Le Goas (6):
  MINOR: anon: add new macros and functions to anonymize contents
  MINOR: anon: store the anonymizing key in the global structure
  MINOR: anon: store the anonymizing key in the CLI's appctx
  MINOR: cli: anonymize commands 'show sess' and 'show sess all'
  MINOR: cli: anonymize 'show servers state' and 'show servers conn'
  MINOR: config: add command-line -dC to dump the configuration file

Frédéric Lécaille (14):
  BUG/MINOR: quic: Retransmitted frames marked as acknowledged
  BUG/MINOR: quic: Possible crash with "tls-ticket-keys" on QUIC bind lines
  BUG/MINOR: quic: Possible crash when verifying certificates
  MINOR: quic: Add traces about sent or resent TX frames
  MINOR: quic: No TRACE_LEAVE() in retrieve_qc_conn_from_cid()
  BUG/MINOR: quic: Wrong connection ID to thread ID association
  BUG/MINOR: quic: Speed up the handshake completion only one time
  BUG/MINOR: quic: Trace fix about packet number space information.
  BUG/MINOR: h3: Crash when h3 trace verbosity is "minimal"
  MINOR: h3: Add the quic_conn object to h3 traces
  MINOR: h3: Missing connection argument for a TRACE_LEAVE() argument
  MINOR: h3: Send the h3 settings with others streams (requests)
  MINOR: dev/udp: Apply the corruption to both directions
  BUILD: udp-perturb: Add a make target for udp-perturb tool

Ilya Shipitsin (1):
  CI: cirrus-ci: bump FreeBSD image to 13-1

Mathias Weiersmueller (1):
  DOC: fix TOC in starter guide for subsection 3.3.8. Statistics

Matthias Wirth (1):
  BUG/MINOR: signals/poller: ensure wakeup from signals

William Lallemand (14):
  BUILD: quic: add some ifdef around the SSL_ERROR_* for libressl
  BUILD: ssl: fix ssl_sock_switchtx_cbk when no client_hello_cb
  BUILD: quic: temporarly ignore chacha20_poly1305 for libressl
  BUILD: quic: enable early data only with >= openssl 1.1.1
  BUILD: ssl: fix the ifdef mess in ssl_sock_initial_ctx
  BUILD: quic: fix the #ifdef in ssl_quic_initial_ctx()
  MINOR: quic: add QUIC support when no client_hello_cb
  BUG/MINOR: signals/poller: set the poller timeout to 0 when there are 
signals
  REGTESTS: log: test the log-forward feature
  REGTESTS: ssl/log: test the log-forward with SSL
  MEDIUM: httpclient: httpclient_create_proxy() creates a proxy for 
httpclient
  MEDIUM: httpclient: allow to use another proxy
  MINOR: httpclient: export httpclient_create_proxy()
  MEDIUM: quic: separate path for rx and tx with set_encryption_secrets

Willy Tarreau (50):
  BUG/MINOR: task: always reset a new tasklet's call date
  BUG/MINOR: task: make task_instant_wakeup() work on a task not a tasklet
  MINOR: task: permanently enable latency measurement on tasklets
  CLEANUP: task: rename ->call_date to ->wake_date
  BUG/MINOR: sched: properly account for the CPU time of dying tasks
  MINOR: sched: store the current profile entry in the thread context
  BUG/MINOR: stream/sched: take into account CPU profiling for the last call
  MINOR: tasks: do not keep cpu and latency times in struct task
  MINOR: tools: add generic pointer hashing functions
  CLEANUP: activity: make memprof use the generic ptr_hash() function
  CLEANUP: activity: make taskprof use ptr_hash()
  MINOR: debug: add struct ha_caller to describe a calling location
  CLEANUP: debug: use struct ha_caller for memstat
  DEBUG: task: define a series of wakeup types for tasks and tasklets
  DEBUG: task: use struct ha_caller instead of arrays of file:line
  DEBUG: applet: instrument appctx_wakeup() to log the caller's location
  DEBUG: task: simplify the caller recording in DEBUG_TASK
  CLEANUP: task: move tid and wake_date into the common part
  CLEANUP: sched: remove duplicate code in run_tasks_from_list()
  CLEANUP: activity: make the number of sched activity entries more 
configurable
  DEBUG: resolvers: unstatify process_resolvers() to make it appear in 
profiling
  DEBUG: quic: export the few task handlers that often appear in task dumps
  MEDIUM: tasks/activity: combine the called function with the caller
  MINOR: tasks/activity: improve the caller-callee activity hash
  MINOR: activity/cli: support aggregating task profiling outputs
  MINOR: activity/cli: support sorting task profiling by total CPU time
  DEV: flags: fix usage message to reflect available options
  DEV: flags: add missing CO_FL_FDLESS connection flag
  MINOR: flags: add a new file to host flag dumping macros
  MINOR: flags: implement a macro used to dump enums inside masks
  MINOR: flags/channel: use flag dumping for channel flags and analysers
  MINOR: flags/connection: use flag dumping for connection flags
  MINOR: flags/stconn: use flag dumping for stconn and 

Re: [PR] fix some typos

2022-09-14 Thread Willy Tarreau
On Wed, Sep 14, 2022 at 11:05:30PM +0200, Tim Düsterhus wrote:
> On 9/14/22 22:43, Willy Tarreau wrote:
> > Thanks, but quite honnestly, even this one is not a commit message.
> 
> There really is not much to be explained for some obvious typo fixes in
> comments (i.e. not even changing the binary). I don't think the commit
> message was worse than Ilya's semi-regular typo patches.

That's what I suspected as well, my point was that encouraging someone
to go figure somewhere by following the link that in the end there was
nothing more to see didn't bring any value. However, just mentioning
the isolated systems touched by the patch can help quickly spot it when
a backport for a fix fails due to slightly different context.

> I've only put the PR link in the description, because I was unable to think
> of literally anything else that might be useful for a typo fix and I didn't
> want to leave the description empty (because CONTRIBUTING says I may not do
> so).

Yep, that rule generally helps thinking about what was not said in the
subject (e.g. "this just fixes 4 trivial typos in comments in mux_quic.c,
ssl_sock.c and xprt_quic.c").

> > You get the idea. Regarding the tag, it's fine (and even preferred) to
> > keep the signed-off-by if you edit the patch, you just add yours below
> > after a short description between square brackets. Example:
> 
> Understood. I've dropped it, because you said in the past that you are not
> editing patches containing a s-o-b.

You're absolutely right, but it's in fact a rule I use to gauge what
a contributor expects without wasting their time or a round-trip on a
trivial change. I think that if you send me a patch, you've rechecked
it 3 times to be certain it's OK before signing it, so what I could
take for a typo could also be a mistunderstanding on my side, thus I
will normally not change it without asking. On the contrary, an unsigned
commit reads to me like "do what you want with it, I don't care", and if
the effort of changing it is lower than that of requesting that it's
updated, and there's no perceived benefit in teaching the submitter how
to improve their future contributions, or it's really trivial, then I 
can update it.

> I'm not sending another patch or pursuing this patch further, because that
> is not a good use of either of our time. I just came across this unhandled
> PR in my Inbox and wanted to save you the effort of replying to the author /
> fixing the message yourself. Seeing that you took the effort of writing this
> long reply, I probably should've just ignored the patch entirely, because it
> ultimately wasted time for everyone involved.

No, not at all, quite the opposite! I just took this opportunity to explain
some tiny improvements to the process for everyone and the rationale behind
them, and also to help you figure more easily how to proceed next time
without having to hesitate too long. I know what it's like to submit a
patch, whatever the project, there's a lot of hesitation, at least 4 or 5
"git commit --amend", sometimes even a last-minute re-edit of the patch
itself. The simple fact that you explained the nature of your changes
indicates that it took you some thinking. That's what I want to help make
smoother so that you don't need to invest as much of your time on such
patches in the future. I could very well have re-edited it myself, but
that would not have been respectful of your effort.

Thanks!
Willy



Re: [PR] fix some typos

2022-09-14 Thread Willy Tarreau
Hi Tim,

On Wed, Sep 14, 2022 at 07:44:27PM +0200, Tim Düsterhus wrote:
> The fixes look correct to me, but the commit message is horrible. I've
> attached the patch with a proper commit message (dropping the Signed-off-by
> and adding a Co-authored-by).
> 
> Best regards
> Tim Düsterhus

> From e2fa51bc9f7e7d7d451df2ad9cd72071211faf6a Mon Sep 17 00:00:00 2001
> From: cui fliter 
> Date: Mon, 29 Aug 2022 14:42:57 +0800
> Subject: [PATCH] CLEANUP: Fix typos in C comments
> 
> see GitHub PR #1843
> 
> [Tim: Rephrased the commit message]
> 
> Co-authored-by: Tim Duesterhus 

Thanks, but quite honnestly, even this one is not a commit message.

I mean, referencing an external issue for a complement of information
is fine, but as the sole source, please no. First, it means that those
who read the logs have zero info there, they need to open a browser,
visit the project on github and enter the PR number just to get a very
likely uninteresting context. Second it's not greppable. And third,
imagine that one day a large company would decide to acquire github
(yeah I know, purely fictional :-)) and to progressively change the
conditions to the point that we're almost forced out. Suddenly, all
such knowledge is lost. Forever.

So commit messages must accurately describe the problem that they try
to solve. External references for more info, more context etc is
perfectly fine and welcome, but that must never be a shortcut for a
lengthy message (and yes I know that here we're talking only about
typos).

Another thing to try to avoid is too generic subject lines. One good
rule of thumb is to try to invent a subject line that is likely unique
in the project. That doesn't mean it ought to be checked, just that if
it's precise enough it will likely be unique. Here it could for example
be something like:

 CLEANUP: Fix tiny typos in C comments in SSL and QUIC

  or:

 CLEANUP: quic,ssl: fix tiny typos in C comments

You get the idea. Regarding the tag, it's fine (and even preferred) to
keep the signed-off-by if you edit the patch, you just add yours below
after a short description between square brackets. Example:

Signed-off-by: joe user < >
[Tim: authored a real commit message]
Signed-off-by: Tim < >

You must just not add an s-o-b for someone else if there wasn't (though
you can always add yours). Regarding "co-authored", normally it's more
for when the patch itself was adapted, but usually the first author
places it from the start because all participants generally acknowledge
co-authorship.

Thanks!
Willy



Re: [PATCH] DOC: fix TOC in starter guide for subsection 3.3.8. and 3.4.9

2022-09-13 Thread Willy Tarreau
On Mon, Sep 12, 2022 at 12:19:42PM +0200, Tim Düsterhus wrote:
> Mathias,
> 
> On 9/10/22 20:08, Mathias Weiersmüller (cyberheads GmbH) wrote:
> > The subsection 3.4.9 (Standard features : Statistics)  in the starter guide 
> > from 2.4 up to latest points to a non-existing anchor. It looks like this 
> > subsection was moved from 3.4.9 to 3.3.8  (Basic features : Statistics), 
> > but the TOC was not updated accordingly. Please check the attached patch to 
> > fix this.
> > 
> 
> This appears to be introduced in e68e7e8426dc0297a757327e342dd5a212a5f2d9
> and was likely backported.

Now applied, thank you guys!
Willy



Re: lua workers and peer stick tables

2022-09-09 Thread Willy Tarreau
Hi Dave,

On Thu, Sep 08, 2022 at 01:20:57AM +, Dave Cottlehuber wrote:
> The second part, is it possible to access peer stick tables?
> 
> I don't see them in the objects listed by Thierry, nor when recursively
> dumping the core object.
> 
> https://www.arpalert.org/src/haproxy-lua-api/2.6/#external-lua-libraries

I'm far from being a specialist of the Lua part, but to the best of my
knowledge, there's no binding for stick-tables (there are even a few
issues opened here and there about adding this or that binding to Lua).

If you just need to read the table, you should have a look at the
possibility to call some sample fetch functions or converters that are
able to access stick-tables. For example, table_http_req_rate() and
friends. This one takes a sample on input that is looked up as a key
in the designated table and returns the value. Converters and sample
fetch functions can be called from Lua. I never know how to use them
but I know it's possible. Please have a look at "TXN.c" in the Lua
doc above, you should understand how to proceed better than me :-)

Hoping this helps,
Willy



Re: [PATCH] Allow disabling "option forwardfor"

2022-09-09 Thread Willy Tarreau
Hi Samuel,

On Thu, Sep 08, 2022 at 09:04:22AM +0200, Samuel Maftoul wrote:
> Hi,
> 
> This is the continuation of a discussion that happened on github (
> https://github.com/haproxy/haproxy/pull/1853).

Thanks for this.

> Here, I applied Willy's 3rd proposal: adding a keyword to the forwardfor
> option to disable the header. I used 'none' as the keyword, as I looked in
> the documentation to see if I should use 'disable' or 'disabled', it looked
> to me 'disable' is more consistent, but 'none' is even more consistent (we
> already have options using 'none' as keyword, but no option uses 'disable'
> as keyword).
> 
> Also, the implementation is simpler than what was proposed in the PR,
> Christopher, I think I don't need to free anything, and also , I added the
> edge case you cited as a test.
> 
> Is this approach right ?

I think the approach is right, however I suspect that it won't work if
the option is enabled in the frontend, which I understood was the initial
concern. In your case it will only disable it if it was present in the
defaults section, which a "no option ..." should handle right.

If we want to cover the cancellation of a frontend option, I suspect that
we need to set a flag on the backend to indicate that the header must not
be added when the request leaves, so that it can cancel a possible
forwardfor in the frontend.

So there are two possibilities:
  - "no option ..." => revert to default settings for the option, i.e.
do not force it enabled, which means that it can revert what appears
in the defaults section, but that frontend presence remains sufficient
to enable itx

  - "option forwardfor none" => explicitly disable XFF addition for requests
flowing through this backend, even if they passed through a frontend
which had the option explicitly enabled.

We can even make sure to support both, by the way, if "no option..." does
not work right now.

Hoping this clarifies it a little bit,
Willy



Re: [PATCH] cirrus-ci: bump FreeBSD image to 13.1

2022-09-09 Thread Willy Tarreau
On Fri, Sep 09, 2022 at 12:27:09PM +0200, Tim Düsterhus wrote:
> Willy,
> 
> On 9/8/22 19:04,  ??? wrote:
> > as we install freebsd binary packages, we need to bump image from time to
> > time
> > to match prebuilt packages.
> > 
> 
> The patch LGTM and should unbreak CI. Please take it.

Applied and pushed, thank you both!
Willy



Re: lua workers and peer stick tables

2022-09-07 Thread Willy Tarreau
Hi Dave,

On Wed, Sep 07, 2022 at 09:04:44PM +, Dave Cottlehuber wrote:
> hi,
> 
> I'm working towards dumping a list of top N http requesters via a
> lua-driven HTTP response, from a peer synced table.
> 
> The first stage is to dump without peers. I have found the stick table
> object, but can't call any of the info, dump, or lookup methods on it.
> 
> Using this example[0] from the blog[1], I end up with this lua snippet:
> 
> NB s/dump/info/ below to work around possible url spam filtering...
[...]

just a quick response to confirm it reached the list. Thanks for your
patience and sorry for the inconvenience.

Willy



Re: server cookie value uniqueness

2022-09-07 Thread Willy Tarreau
Hello Artur,

On Tue, Sep 06, 2022 at 05:50:47PM +0200, Artur wrote:
> Hello !
> 
> I'm adding two servers s01 and s02 to the current config and setting the
> same cookie value as for existing s1 and s2.
> These cookies are here to permit sticky sessions.
> What may be the behaviour of haproxy in this situation ?

Haproxy will pick the first operating server using the requested cookie.
That's seldom used when an alternate paths to the same server is available,
but most commonly when using active and backup servers in fact.

> 'haproxy -c' on the configuration file does not show any warning or error.

That's expected because we must not emit warnings on valid configs that
cannot be expressed differently (otherwise users won't read warnings
anymore). However there's a diagnostic mode which already detects such
unusual setups and which will emit a few reports to help you spot such
cases:

> backend one
> ...
> option redispatch
> balance roundrobin
> server s1 10.100.0.93:2000 cookie s1
> server s2 10.100.0.93:2001 cookie s2
> server s01 10.100.3.101:2000 cookie s1
> server s02 10.100.3.101:2001 cookie s2
> option allbackups
> server sb1 10.100.131.33:2000 cookie s1 backup
> server sb2 10.100.131.33:2001 cookie s2 backup

  $ ./haproxy -dD -f foo.cfg
  (...)
  [DIAG] (22717) : config : parsing [foo.cfg:7] : 'server s01' : same 
cookie value is set for a previous non-backup server in the same backend, it 
may break connection persistence
  [DIAG] (22717) : config : parsing [foo.cfg:8] : 'server s02' : same 
cookie value is set for a previous non-backup server in the same backend, it 
may break connection persistence

The purpose of the diag really is to distinguish what's likely valid
and unlikely valid, compared to what's valid and not valid, thus it
will mostly help you spot typos or copy-paste errors.

Willy



Re: Warnings with gcc 12.2.0 ... is the community interested in those, or already aware?

2022-09-05 Thread Willy Tarreau
On Mon, Sep 05, 2022 at 08:43:18AM -0600, Shawn Heisey wrote:
> On 9/5/22 02:45, Tim Düsterhus wrote:
> > I'd say: Create an issue at https://github.com/haproxy/haproxy/issues.
> > In the worst case it's just a duplicate and will be labeled as such.
> > 
> > In the ideal case you would've simply shared the logs, the effort to
> > reply to your email is similar to triaging an issue ?
> 
> Apologies.  I was trying to avoid wasting somebody's precious time, looks
> like that backfired in a big way.

No worries, really. You don't have to apologize for reporting something.
I prefer, and by far, to deal with duplicates than with lack of reports!

> Noted for the future. I will still
> report here first for issues when I am not certain that it's a bug or a
> problem in the build system.  Some of the problems I run into end up being
> user error.  Additionally, some projects prefer to discuss things before an
> issue is filed.

OK!

> I have opened a code-report issue on the github repo:
> 
> https://github.com/haproxy/haproxy/issues/1852
> 
> This message probably does not need a reply.

Thank you. Indeed, it's the same that Ilya already reported and that comes
from something illogical in this version of gcc. We have not identified any
easy workaround for this one, in the worst case it might end up with a
pragma in the code around it :-/  I must confess I'm a bit more concerned
about the reliability of code that is emitted with such mistaken
assumptions though.

Thanks,
Willy



Re: MINOR: Revert part of clarifying samples support per os commit

2022-09-02 Thread Willy Tarreau
On Sat, Sep 03, 2022 at 12:21:56AM -0400, Brad Smith wrote:
> On 9/3/2022 12:12 AM, Willy Tarreau wrote:
> > On Thu, Aug 25, 2022 at 11:13:38PM -0400, Brad Smith wrote:
> > > Commit 5c83e3a1563cd7face299bf08037e51f976eb5e3 made some adjustments
> > > to clarify which TCP_INFO information is supported by each respective
> > > OS.
> > (...)
> > 
> > Merged, thank you Brad!
> > Willy
> 
> Can you add this to 2.6?

Yes if that doesn't break any OS. What is the real impact of this issue
exactly ?

Willy



Re: [PATCH] BUILD: makefile: enable crypt(3) for NetBSD

2022-09-02 Thread Willy Tarreau
On Sat, Sep 03, 2022 at 12:19:24AM -0400, Brad Smith wrote:
> On 9/3/2022 12:11 AM, Willy Tarreau wrote:
> > On Sat, Aug 13, 2022 at 12:57:31AM -0400, Brad Smith wrote:
> > > Allow NetBSD to support encrypted passwords in Userlists.
> > > 
> > Mergd, thank you Brad!
> > Willy
> 
> You should be able to bring that back to 2.4.

OK thanks. I generally prefer not to backport new features, especially
to versions older than the latest LTS branch, unless there's a compelling
reason (i.e. needed for a fix or requested by users with a reasonable
justification), as occasional build-time and boot-time regressions
discourage users from staying up to date. But if you really need it
and you've tested it, that can be done.

Thanks,
Willy



Re: Server state file: port doesn't change after config update

2022-09-02 Thread Willy Tarreau
Hi Bren,

On Mon, Aug 22, 2022 at 05:20:37PM +, Bren wrote:
> Hello,
> 
> We've been seeing another minor issue I've been meaning to ask about. We're 
> using a server state file:
> 
> server-state-file /var/lib/haproxy/server_state
> 
> In my systemd config for haproxy I've added a couple lines to save the server 
> state on reload/stop:
> 
> ExecReload=/usr/local/sbin/haproxy -Ws -f $CONFIG -c -q $EXTRAOPTS
> ExecReload=/opt/bin/save_server_state.sh
> ExecReload=/bin/kill -USR2 $MAINPID
> ExecStop=/opt/bin/save_server_state.sh
> 
> The script simply runs:
> 
> echo "show servers state" | socat /var/run/haproxy/admin.sock - > \
>   /var/lib/haproxy/server_state
> 
> I've noticed that when I change the port on a backend server and reload,
> haproxy does not update the port for that server. I have to shut down
> haproxy, delete the state file, then start it back up for it to update the
> port (changing the port and renaming the server, reloading, then renaming it
> back and reloading again works too).
> 
> When I change other parts of the config, for example, if I add "disabled" to
> a server line and reload, haproxy updates the status of the server. It seems
> like haproxy isn't looking at the port when deciding if it should update the
> config for a server when using a state file.
> 
> Is this expected behavior?

It's neither expected nor unexpected. The problem is that the state file
only reflects *last* known values for elements that come both from the
config and from dynamic changes (e.g. CLI commands or DNS SRV records).
As such when haproxy restarts its parses its config and updates it based
on the state file. Some discrepancies are resolved by considering the
config first, but not all can otherwise the state file would be useless.

The right way to proceed for the state file would be to provide both the
config and the dynamic elements so that on startup, the entry is ignored
if the config changed, but it's not how it currently works. Thus as you
see it's a design issue. There's already an open issue on this topic:

   https://github.com/haproxy/haproxy/issues/953

It just happens that this is a tedious task and that there are few
enough users affected by such issues to convince anyone to dive into
this :-/

Regards,
Willy



Re: most probably next LibreSSL release will come with ... QUIC

2022-09-02 Thread Willy Tarreau
Hi,

On Wed, Aug 31, 2022 at 10:20:42PM +0200, Lukas Tribus wrote:
> Hello,
> 
> 
> wolfSSL has also chosen to use the same API for QUIC:
> 
> https://www.wolfssl.com/wolfssl-quic-support/
> 
> > The wolfSSL QUIC API is aligned with the corresponding APIs in other *SSL
> > libraries, making integration with QUIC protocol stacks easier and
> > protecting investments. This is a departure from past customs where OpenSSL
> > used to drive the design of APIs. However OpenSSL declined to participate
> > and offers no QUIC support for the foreseeable future.
> 
> 
> This is probably less useful for haproxy specifically, given that we
> don't support wolfssl in the first place, but interesting nonetheless.

Definitely, and we're currently having a look at all of this. GnuTLS
also supports QUIC using the same API (at least that's my understanding),
so in the end, OpenSSL will be the *only* mainstream SSL library that
continues to reject it. That obstination to not listen to their users
tells a lot about that project's governance and its life expectancy,
and if you factor in the massive performance regression that plagues
distros that ship with 3.0 such as Ubuntu 22, that basically limits its
use cases to command-line certificate generation and maybe SMTP/IMAP
daemons, but the future of the web will clearly be without OpenSSL now.
It's their decision, it's really sad and it negatively impacts all of
the web infrastructure ecosystem, but it's their project. Many of us
implored them to open their ears but there's not much more that can be
done at this point, they've started to plant the nails in the coffin.
We'll need to move on.

Willy



Re: MINOR: Revert part of clarifying samples support per os commit

2022-09-02 Thread Willy Tarreau
On Thu, Aug 25, 2022 at 11:13:38PM -0400, Brad Smith wrote:
> Commit 5c83e3a1563cd7face299bf08037e51f976eb5e3 made some adjustments
> to clarify which TCP_INFO information is supported by each respective
> OS.
(...)

Merged, thank you Brad!
Willy



Re: [PATCH] BUILD: makefile: enable crypt(3) for NetBSD

2022-09-02 Thread Willy Tarreau
On Sat, Aug 13, 2022 at 12:57:31AM -0400, Brad Smith wrote:
> Allow NetBSD to support encrypted passwords in Userlists.
> 

Mergd, thank you Brad!
Willy



[ANNOUNCE] haproxy-2.6.5

2022-09-02 Thread Willy Tarreau
e
  BUG/MINOR: ssl: leak of ckch_inst_link in ckch_inst_free()
  BUG/MINOR: ssl: revert two wrong fixes with ckhi_link
  BUG/MINOR: ssl: leak of ckch_inst_link in ckch_inst_free() v2

Willy Tarreau (26):
  BUG/MEDIUM: applet: fix incorrect check for abnormal return condition 
from handler
  BUG/MINOR: applet: make the call_rate only count the no-progress calls
  BUG/MEDIUM: mux-h1: do not refrain from signaling errors after end of 
input
  BUG/MINOR: dev/udp: properly preset the rx address size
  CLEANUP: quic: use task_new_on() for single-threaded tasks
  CLEANUP: pool/quic: remove suffix "_pool" from certain pool names
  BUILD: ring: forward-declare struct appctx to avoid a build warning
  MINOR: ring: support creating a ring from a linear area
  MINOR: ring: add support for a backing-file
  BUILD: sink: replace S_IRUSR, S_IWUSR with their octal value
  MINOR: ring: archive a previous file-backed ring on startup
  MINOR: sink/ring: rotate non-empty file-backed contents only
  DEV: haring: add a simple utility to read file-backed rings
  DEV: haring: support remapping LF in contents with CR VT
  BUILD: debug: make sure debug macros are never empty
  BUG/MEDIUM: mux-h1: always use RST to kill idle connections in pools
  MINOR: backend: always satisfy the first req reuse rule with l7 retries
  BUG/MINOR: h2: properly set the direction flag on HTX response
  BUG/MEDIUM: httpclient: always detach the caller before self-killing
  BUG/MINOR: httpclient: keep-alive was accidentely disabled
  BUG/MINOR: mux-h2: fix the "show fd" dest buffer for the subscriber
  BUG/MINOR: mux-h1: fix the "show fd" dest buffer for the subscriber
  BUG/MINOR: mux-fcgi: fix the "show fd" dest buffer for the subscriber
  DEBUG: stream: minor rearrangement of a few fields in struct stream.
  MINOR: debug: report applet pointer and handler in crashes when known
  BUG/MINOR: http-act: initialize http fmt head earlier

---



[ANNOUNCE] haproxy-2.7-dev5

2022-09-02 Thread Willy Tarreau
s
   Reviewed bugs: http://www.haproxy.org/l/reviewed-bugs
   Code reports : http://www.haproxy.org/l/code-reports
   Latest builds: http://www.haproxy.org/l/dev-packages

Willy
---
Complete changelog :
Brad Smith (1):
  BUILD: tcp_sample: fix build of get_tcp_info() on OpenBSD

Christopher Faulet (11):
  BUG/MINOR: tcpcheck: Disable QUICKACK only if data should be sent after 
connect
  REGTESTS: Fix prometheus script to perform HTTP health-checks
  BUG/MEDIUM: spoe: Properly update streams waiting for a ACK in async mode
  BUG/MEDIUM: peers: Add connect and server timeut to peers proxy
  BUG/MEDIUM: peers: Don't use resync timer when local resync is in progress
  BUG/MEDIUM: peers: Don't start resync on reload if local peer is not 
up-to-date
  BUG/MINOR: hlua: Rely on CF_EOI to detect end of message in HTTP applets
  BUG/MINOR: tcpcheck: Disable QUICKACK for default tcp-check (with no rule)
  BUG/MEDIUM: ssl: Fix a UAF when old ckch instances are released
  BUG/MINOR: regex: Properly handle PCRE2 lib compiled without JIT support
  REGTESTS: http_request_buffer: Add a barrier to not mix up log messages

Emeric Brun (1):
  BUG/MAJOR: mworker: fix infinite loop on master with no proxies.

Frédéric Lécaille (22):
  BUG/MINOR: mux-quic: Fix memleak on QUIC stream buffer for unacknowledged 
data
  BUG/MINOR: quix: Memleak for non in flight TX packets
  BUG/MINOR: quic: Wrong list_for_each_entry() use when building packets 
from qc_do_build_pkt()
  BUG/MINOR: quic: Safer QUIC frame builders
  MINOR: quic: Replace MT_LISTs by LISTs for RX packets.
  Revert "BUG/MINOR: quix: Memleak for non in flight TX packets"
  BUG/MINOR: quic: Leak in qc_release_lost_pkts() for non in flight TX 
packets
  BUG/MINOR: quic: Stalled connections (missing I/O handler wakeup)
  CLEANUP: quic: No more use ->rx_list MT_LIST entry point (quic_rx_packet)
  CLEANUP: quic: Remove a useless check in qc_lstnr_pkt_rcv()
  MINOR: quic: Remove useless traces about references to TX packets
  Revert "MINOR: quic: Remove useless traces about references to TX packets"
  BUG/MINOR: quic: Null packet dereferencing from qc_dup_pkt_frms() trace
  BUG/MINOR: quic: Frames added to packets even if not built.
  BUG/MINOR: quic: Missing header protection AES cipher context 
initialisations (draft-v2)
  MINOR: quic: Add a trace to distinguish the datagram from the packets 
inside
  MINOR: quic: Move traces about RX/TX bytes from QUIC_EV_CONN_PRSAFRM event
  BUG/MINOR: quic: TX frames memleak
  BUG/MINOR: quic: Do not ack when probing
  MINOR: quic: Add TX frames addresses to traces to several trace events
  MINOR: quic: Trace typo fix in qc_release_frm()
  BUG/MINOR: quic: Frames leak during retransmissions

William Lallemand (12):
  REGTESTS: launch http_reuse_always in mworker mode
  BUG/MINOR: resolvers: return the correct value in 
resolvers_finalize_config()
  BUG/MINOR: mworker: does not create the "default" resolvers in wait mode
  MINOR: resolvers: shut the warning when "default" resolvers is implicit
  DOC: configuration: do-resolve doesn't work with a port in the string
  MINOR: sample: add the host_only and port_only converters
  BUG/MINOR: httpclient: fix resolution with port
  DOC: configuration.txt: do-resolve must use host_only to remove its port.
  BUG/MINOR: ssl: fix deinit of the ca-file tree
  BUG/MINOR: ssl: leak of ckch_inst_link in ckch_inst_free()
  BUG/MINOR: ssl: revert two wrong fixes with ckhi_link
  BUG/MINOR: ssl: leak of ckch_inst_link in ckch_inst_free() v2

Willy Tarreau (32):
  BUG/MEDIUM: cpu-map: fix thread 1's affinity affecting all threads
  MINOR: cpu-map: remove obsolete diag warning about combined ranges
  BUG/MEDIUM: applet: fix incorrect check for abnormal return condition 
from handler
  BUG/MINOR: applet: make the call_rate only count the no-progress calls
  MEDIUM: peers: limit the number of updates sent at once
  BUG/MEDIUM: mux-h1: do not refrain from signaling errors after end of 
input
  BUG/MINOR: epoll: do not actively poll for Rx after an error
  MINOR: raw-sock: don't try to send if an error was already reported
  BUG/MINOR: dev/udp: properly preset the rx address size
  BUILD: debug: make sure debug macros are never empty
  MINOR: sink/ring: rotate non-empty file-backed contents only
  BUG/MEDIUM: mux-h1: always use RST to kill idle connections in pools
  MINOR: backend: always satisfy the first req reuse rule with l7 retries
  BUG/MINOR: h2: properly set the direction flag on HTX response
  BUG/MEDIUM: httpclient: always detach the caller before self-killing
  BUG/MINOR: httpclient: only ask for more room on failed writes
  BUG/MINOR: httpclient: keep-alive was accidentely disabled
  MEDIUM: htt

Re: [PATCH] MINOR: tcp_sample: extend support for get_tcp_info to OpenBSD

2022-08-23 Thread Willy Tarreau
On Tue, Aug 23, 2022 at 10:37:28PM -0400, Brad Smith wrote:
> On 8/23/2022 10:22 PM, Willy Tarreau wrote:
> > Hi Brad,
> > 
> > On Sat, Aug 13, 2022 at 11:25:32PM -0400, Brad Smith wrote:
> > > I'm not sure if MINOR is right. Currently the build is broken since 
> > > TCP_INFO
> > > was added.
> > Just to be certain, you mean the build is broken without your patch or
> > with it ? If it's broken without, it means that your patch is a build
> > fix, but it also indicates there's something wrong in the whole code's
> > arrangement there that we'll have to re-adjust, because we must not
> > break things for operating systems that are not explicitely handled.
> 
> The build is broken without the diff. I was looking at older releases and
> this
> only affects 2.7 and 2.6.
> 
> It looks like this changed things in such a way that if TCP_INFO is added
> but the OS is not added to the list of OS's it will not build.
> 
> http://git.haproxy.org/?p=haproxy-2.6.git;a=commitdiff;h=5c83e3a1563cd7face299bf08037e51f976eb5e3;hp=39e436e36c4e6c0efe95c304fe89fd01e111

Oh I see... What a mess! Thanks for the clarifiation.

I'll merge your patch as a build fix and with such indications. It's
suitable for backporting. As a next step for 2.7 and further, we should
change this to have a new USE_TCP_INFO build options that will be set
by default in the makefile for supported operating systems and tested
instead of testing all OS combinations.

If you want to work on this, you're welcome :-)

Thanks!
Willy



Re: [PATCH] MINOR: tcp_sample: extend support for get_tcp_info to OpenBSD

2022-08-23 Thread Willy Tarreau
Hi Brad,

On Sat, Aug 13, 2022 at 11:25:32PM -0400, Brad Smith wrote:
> I'm not sure if MINOR is right. Currently the build is broken since TCP_INFO
> was added.

Just to be certain, you mean the build is broken without your patch or
with it ? If it's broken without, it means that your patch is a build
fix, but it also indicates there's something wrong in the whole code's
arrangement there that we'll have to re-adjust, because we must not
break things for operating systems that are not explicitely handled.

If you mean that the build is broken on certain versions once your
patch is applied, that means we'll need to re-adjust it or to ease
configuration of the feature.

Thanks,
Willy



Re: Understanding show table output and rate limiting weirdness

2022-08-23 Thread Willy Tarreau
Hi Corin,

On Fri, Aug 19, 2022 at 08:55:17AM +, Corin Langosch wrote:
> Hello guys,
> 
> I'm using the docker image 2.5.7-2ef551d with basic rate limiting configured 
> like this:
> 
>   backend test
> acl test_rate_limit_by_ip_exceeds_limit 
> src,table_http_req_rate(test_rate_limit_by_ip) gt 5
> http-request deny deny_status 429 if test_rate_limit_by_ip_exceeds_limit
> http-request track-sc0 src table test_rate_limit_by_ip
> 
> acl test_rate_limit_api_exceeds_limit 
> req.hdr(authorization),table_http_req_rate(test_rate_limit_api) gt 100
> http-request deny deny_status 429 if test_rate_limit_api_exceeds_limit
> http-request track-sc1 req.hdr(authorization) table test_rate_limit_api 
> if { path -i -m beg "/api" }
> 
> acl test_rate_limit_graphql_exceeds_limit 
> req.hdr(x-api-key),table_http_req_rate(test_rate_limit_graphql) gt 100
> http-request deny deny_status 429 if test_rate_limit_graphql_exceeds_limit
> http-request track-sc2 req.hdr(authorization) table 
> test_rate_limit_graphql if { path -i -m beg "/graphql" }
> 
> server s1 10.0.0.1:80
> server s2 10.0.0.2:80
> 
>   backend test_rate_limit_by_ip
> stick-table type ipv6 size 1m expire 24h store http_req_rate(5m)
> 
>   backend test_rate_limit_api
> stick-table type string size 1m expire 24h store http_req_rate(3m)
> 
>   backend test_rate_limit_graphql
> stick-table type string size 1m expire 24h store http_req_rate(3m)
> 
> Now I have some users reporting they are blocked (getting a 429) even though
> they don't perform a lot of requests. To analyze I ran "show table
> table_rate_limit_by_ip", but the output looks a bit weierd to me. Here's an
> anonymized extract:
> 
>   0x55cfd58df130: key=:::1.2.3.1 use=0 exp=1498762521 
> http_req_rate(30)=1181926
>   0x55cfd627f840: key=:::1.2.3.2 use=0 exp=1599966154 
> http_req_rate(30)=80740
>   0x55cfda287e00: key=:::1.2.3.3 use=0 exp=58273431 
> http_req_rate(30)=0
>   0x55cfd5cd5320: key=:::1.2.3.4 use=0 exp=2006327751 
> http_req_rate(30)=1606589
> 
> I wonder what's the value of exp? It doesn't seem to be a unix timestamp (not
> even in milliseconds), nor does it seem to be the number of (milli)seconds
> till the entry expires. Unfortunately, I can't find any documentation about
> it.

For me it was the remaining time before expiration in milliseconds, and
I've just rechecked the code to be sure, it's this. It obviously doesn't
fit with your tables. Your tables have 24h expirations and these ones are
up to 23 days.

> The http_req_rate (which should be the rate within the last 5 minutes for
> this table) is extremely (unrealistic) high for many entries. How is this
> possible?

It makes me think that something is wrong with the date, though I can't
imagine what. Is it entirely reproducible or does it happen spuriously ?
I'm wondering if it could be a memory corruption for example. Could you
please share the output of "show info" ? (beware there's the host name
there that you might want to delete if it's sensitive to you). Maybe
we'll find something weird there.

> Is my configuration broken? Is it a bug? ...?

Just thinking, would it be possible that you first started with a very
high expiration value (by accident) and that you then adjusted the
config and reloaded ? From what I remember, the time left is preserved
in exchanges between peers, so having, say, a 24 days value, then a
reconfig to 24 hours and a reload would preserve the 24 days one
(arguably we should enforce the new limit), but that's definitely
something that would make sense to me.

> Thanks for any help,
> Corin
> 
> PS: Is there any way to filter the output of show table
> table_rate_limit_by_ip by key (by ip in my case)? In the docs, I only find
> how to filter by value (by http_req_rate in my case).

Yes, there's "key" (look at the end of the line):

  show table  [ data.   [data. ...]] | [ key 
 ]

So you should be able to issue:

   show table test_rate_limit_by_ip key :::1.2.3.1

And it should work.

Willy



Re: haproxy 2.6.2 warnings while installation

2022-08-22 Thread Willy Tarreau
Hi Amol,

On Mon, Aug 22, 2022 at 11:42:01AM +0530, Amol Arote wrote:
> We are using haproxy 2.6.2 on Centos 7.9 ,
> 
> Following are warning seen during
> 
> *make TARGET=linux-glibc USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1*
> 
> [image: image.png]
> 
> src/http_fetch.c:356:6: warning: âhtxâ may be used uninitialized in this
> function [-Wmaybe-uninitialized]
>sl = http_get_stline(htx);
>   ^
> src/http_fetch.c: At top level:
> cc1: warning: unrecognized command line option "-Wno-atomic-alignment"
> [enabled by default]
> cc1: warning: unrecognized command line option "-Wno-string-plus-int"
> [enabled by default]
> cc1: warning: unrecognized command line option "-Wno-cast-function-type"
> [enabled by default]
> cc1: warning: unrecognized command line option
> "-Wno-address-of-packed-member" [enabled by default]
> 
> 
> 
> Please let us know if any concern

These are harmless and already fixed in 2.6.3.

Regards,
Willy



Re: [*EXT*] [ANNOUNCE] haproxy-2.6.3

2022-08-20 Thread Willy Tarreau
On Sat, Aug 20, 2022 at 09:36:21PM +0200, Ionel GARDAIS wrote:
> That was it :
> - remove the EXTRAOPTS from /etc/default/haproxy
> - stop the running process referencing -x /run/haproxy/admin.sock on the CLI
> - upgrade
> 
> All is OK.
> First processes do not list -x on the CLI and a reload spawn a process with 
> -x sockpair@

Very interesting, thank you Ionel.

There is this fix in 2.6.3 which seems in part related to this:

  commit ae1e5a8c72ced4c9527f05bd5b14916d1b9d92c8
  Author: William Lallemand 
  Date:   Mon Jul 25 15:51:30 2022 +0200

BUG/MINOR: sockpair: wrong return value for fd_send_uxst()

The fd_send_uxst() function which is used to send a socket over the
socketpair returns 1 upon error instead of -1, which means the error
case of the sendmsg() is never catched correctly.

Must be backported as far as 1.9.

(cherry picked from commit f67e8fb92c795808f60b2406ae395ebc0ca180c5)
Signed-off-by: Christopher Faulet 

However I'm confused by that part in the code because it seems to try
to send FDs in the function that's supposed to retrieve them, so I'm
missing something related to the context where this is used. But if
the caller was not able to properly handle an error, it could possibly
explain (at least in part) why it failed for you after the upgrade when
trying to connect if there's an issue with this socket. I'll have to
check with William on Monday to clarify all this and try to reproduce. 

Thanks,
Willy



Re: [*EXT*] [ANNOUNCE] haproxy-2.6.3

2022-08-20 Thread Willy Tarreau
Hi Bren,

On Sat, Aug 20, 2022 at 02:05:37PM +, Bren wrote:
> I also had to roll back. I compile from source and push out the binary with
> Ansible which hung on reload. I observed an haproxy process running as root
> using 100% CPU. It never restarted - I had to kill the processes.
> 
> When I started haproxy back up it began using 100% CPU again so I rolled
> back. This is on Debian 11. No "expose-fd listeners" in the config and no
> unusual log entries that I can see.

Thanks for reporting. Hmmm that's not cool and I really don't see what
can cause this, especially since I haven't witnessed any such a thing
yet. I've pushed 2.7-dev4 to haproxy.org, which contains the same fixes,
I'll keep an eye on it, hoping to catch a problem if it may happen.

Did you notice if it failed to serve anything and ate CPU from start or
if it completely started and only then ate CPU ? Do not take risks on
your production, I'm just asking in case you noticed anything particular.

Thanks,
Willy



[ANNOUNCE] haproxy-2.7-dev4

2022-08-20 Thread Willy Tarreau
c: adjust enter/leave traces
  MINOR: mux-quic: define protocol error traces
  CLEANUP: mux-quic: adjust traces level
  MINOR: mux-quic: define new traces
  BUG/MEDIUM: mux-quic: fix crash due to invalid trace arg
  BUG/MINOR: mux-quic: fix crash with traces in qc_detach()
  CLEANUP: exclude haring with .gitignore
  MINOR: quic: adjust quic_frame flag manipulation
  MINOR: h3: report error on control stream close
  MINOR: qpack: report error on enc/dec stream close
  BUG/MEDIUM: mux-quic: reject uni stream ID exceeding flow control
  MINOR: mux-quic: adjust traces on stream init
  MINOR: mux-quic: add missing args on some traces
  MINOR: quic: refactor application send
  BUG/MINOR: quic: do not notify MUX on frame retransmit
  BUG/MEDIUM: quic: fix crash on MUX send notification
  REORG: h2: extract cookies concat function in http_htx
  REGTESTS: add test for HTTP/2 cookies concatenation
  MEDIUM: h3: concatenate multiple cookie headers

Emeric Brun (2):
  BUG/MAJOR: log-forward: Fix log-forward proxies not fully initialized
  BUG/MAJOR: log-forward: Fix ssl layer not initialized on bind even if 
configured

Frédéric Lécaille (21):
  BUG/MEDIUM: quic: Wrong packet length check in qc_do_rm_hp()
  MINOR: quic: Too much useless traces in qc_build_frms()
  BUG/MEDIUM: quic: Missing AEAD TAG check after removing header protection
  MINOR: quic: Replace pool_zalloc() by pool_malloc() for fake datagrams
  MEDIUM: quic: xprt traces rework
  MINOR: quic: Remove useless lock for RX packets
  BUG/MINOR: quic: Possible infinite loop in 
quic_build_post_handshake_frames()
  CLEANUP: quic: Remove trailing spaces
  BUG/MEDIUM: quic: Possible use of uninitialized  variable in 
qc_lstnr_params_init()
  BUG/MEDIUM: quic: Wrong use of  in qc_lsntr_pkt_rcv()
  BUG/MINOR: quic: memleak on wrong datagram receipt
  BUG/MINOR: quic: MIssing check when building TX packets
  BUG/MINOR: quic: Wrong status returned by qc_pkt_decrypt()
  MINOR: stick-table: Add table_expire() and table_idle() new converters
  BUG/MINOR: quic: Missing initializations for ducplicated frames.
  BUG/MINOR: quic: Possible crashes when dereferencing ->pkt quic_frame 
struct member
  MINOR: quic: Add frame addresses to QUIC_EV_CONN_PRSAFRM event traces
  BUG/MINOR: quic: Wrong splitted duplicated frames handling
  MINOR: quic: Add the QUIC connection to mux traces
  MINOR: quic: Trace fix in qc_release_frm()
  MINOR: quic: Add reusable cipher contexts for header protection

Mateusz Malek (1):
  BUG/MEDIUM: http-ana: fix crash or wrong header deletion by 
http-restrict-req-hdr-names

William Lallemand (3):
  BUG/MINOR: ssl/cli: error when the ca-file is empty
  MINOR: ssl: handle ca-file appending in cafile_entry
  MINOR: ssl/cli: implement "add ssl ca-file"

Willy Tarreau (27):
  MINOR: debug: make the mem_stats section aligned to void*
  MINOR: debug: store and report the pool's name in struct mem_stats
  MINOR: debug: also store the function name in struct mem_stats
  MINOR: debug/memstats: automatically determine first column size
  MINOR: debug/memstats: permit to pass the size to free()
  BUG/MEDIUM: quic: always remove the connection from the accept list on 
close
  BUG/MEDIUM: poller: use fd_delete() to release the poller pipes
  BUG/MEDIUM: task: relax one thread consistency check in task_unlink_wq()
  BUILD: stconn: fix build warning at -O3 about possible null sc
  BUG/MEDIUM: ring: fix too lax 'size' parser
  BUILD: ring: forward-declare struct appctx to avoid a build warning
  MINOR: ring: support creating a ring from a linear area
  MINOR: ring: add support for a backing-file
  DEV: haring: add a simple utility to read file-backed rings
  DEV: haring: support remapping LF in contents with CR VT
  BUILD: sink: replace S_IRUSR, S_IWUSR with their octal value
  MINOR: ring: archive a previous file-backed ring on startup
  MINOR: memprof: export the minimum definitions for memory profiling
  MINOR: pool/memprof: report pool alloc/free in memory profiling
  MINOR: pools/memprof: store and report the pool's name in each bin
  MINOR: chunk: inline alloc_trash_chunk()
  MINOR: applet: add a function to reset the svcctx of an applet
  BUG/MEDIUM: cli: always reset the service context between commands
  BUG/MEDIUM: mux-h2: do not fiddle with ->dsi to indicate demux is idle
  MINOR: mux-h2/traces: report transition to SETTINGS1 before not after
  MINOR: mux-h2: make streams know if they need to send more data
  BUG/MINOR: mux-h2: send a CANCEL instead of ES on truncated writes

---



Re: [*EXT*] [ANNOUNCE] haproxy-2.6.3

2022-08-19 Thread Willy Tarreau
On Fri, Aug 19, 2022 at 11:37:47PM +0200, Vincent Bernat wrote:
> On 2022-08-19 23:09, Ionel GARDAIS wrote:
> > Aug 19 22:09:09 haproxy-2 haproxy[1280]: [WARNING]  (1280) : Failed to 
> > connect to the old process socket '/run/haproxy/admin.sock'
> > Aug 19 22:09:09 haproxy-2 haproxy[1280]: [ALERT](1280) : Failed to get 
> > the sockets from the old process!
> 
> There was a change in 2.6.0 (but not in 2.6.3) where "expose-fd listeners"
> for stats socket is not needed anymore. Is the line present in your
> configuration? (grep admin.sock /etc/haproxy/haproxy.cfg)
> 
> What's the output of systemctl cat haproxy?
> 
> > Aug 19 22:09:09 haproxy-2 haproxy[1280]: [NOTICE]   (1280) : New worker 
> > (1282) forked
> > Aug 19 22:09:09 haproxy-2 haproxy[1280]: [NOTICE]   (1280) : Loading 
> > success.
> > Aug 19 22:09:09 haproxy-2 haproxy[1282]: [WARNING]  (1282) : Server 
> > bck-speedtest/go-v6 is DOWN, reason: Layer4 connection problem, info: 
> > "Connection>
> > Aug 19 22:09:09 haproxy-2 haproxy[1282]: Server bck-speedtest/go-v6 is 
> > DOWN, reason: Layer4 connection problem, info: "Connection refused", check 
> > dur>
> > Aug 19 22:09:09 haproxy-2 haproxy[1282]: Server bck-speedtest/go-v6 is 
> > DOWN, reason: Layer4 connection problem, info: "Connection refused", check 
> > dur>
> > Aug 19 22:09:09 haproxy-2 haproxy[1282]: [WARNING]  (1282) : Server 
> > bck-nuxeo-arch/nuxeo-arch is DOWN, reason: Layer4 connection problem, info: 
> > "No r>
> > Aug 19 22:09:09 haproxy-2 haproxy[1282]: [ALERT](1282) : backend 
> > 'bck-nuxeo-arch' has no server available!
> > Aug 19 22:09:09 haproxy-2 haproxy[1282]: Server bck-nuxeo-arch/nuxeo-arch 
> > is DOWN, reason: Layer4 connection problem, info: "No route to host", check>
> > Aug 19 22:09:09 haproxy-2 haproxy[1282]: Server bck-nuxeo-arch/nuxeo-arch 
> > is DOWN, reason: Layer4 connection problem, info: "No route to host", check>
> > Aug 19 22:09:09 haproxy-2 haproxy[1282]: backend bck-nuxeo-arch has no 
> > server available!
> > Aug 19 22:09:09 haproxy-2 haproxy[1282]: backend bck-nuxeo-arch has no 
> > server available!
> > [...] // others few backends not responding
> > // then
> 
> Are these expected?

That's what I would like to know as well. I'm bothered by the systemd
message indicating that haproxy is not responding, I don't really know
what it corresponds to, and could possibly indicate that the sd_notify()
message was not sent before switching to wait mode.

Ideally the startup messages from 2.6.2 where it's working fine would help
as well.

Thanks,
Willy



[ANNOUNCE] haproxy-2.6.3

2022-08-19 Thread Willy Tarreau
ates, but more on that below.

A few build issues were addressed (essentially warnings with older compilers).
Two new converters were backported, table_expire() and table_idle() which
look up a key in a table and respectively report how long is left before the
entry expires and how long the entry was left untouched. They're trivial,
harmless and we've faced they were missing quite a few times already in
environments that want to emit a retry-after for example (issue #1535 opened
6 months ago already).

Regarding QUIC, there are roughly 25 fixes in this version, and we have
prepared the 80 missing ones to match the state of 2.7-dev that's getting
good according to our kind testers (mainly Tristan). I didn't want to add
them into this release, but they will be merged into 2.6.4 so that those
who want to use it have something more reliable that doesn't risk to take
their process down too fast. Thus if you're impatient about QUIC, better
jump to 2.7-dev or wait for 2.6.4.

Thanks to all participants, these were two quite busy weeks but it was
worth it!

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Documentation: http://docs.haproxy.org/
   Wiki : https://github.com/haproxy/wiki/wiki
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Sources  : http://www.haproxy.org/download/2.6/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.6.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.6.git
   Changelog: http://www.haproxy.org/download/2.6/src/CHANGELOG
   Pending bugs : http://www.haproxy.org/l/pending-bugs
   Reviewed bugs: http://www.haproxy.org/l/reviewed-bugs
   Code reports : http://www.haproxy.org/l/code-reports
   Latest builds: http://www.haproxy.org/l/dev-packages

Willy
---
Complete changelog :
Amaury Denoyelle (6):
  BUG/MEDIUM: mux-quic: fix missing EOI flag to prevent streams leaks
  BUG/MINOR: mux-quic: prevent crash if conn released during IO callback
  CLEANUP: mux-quic: remove useless app_ops is_active callback
  BUG/MINOR: mux-quic: do not free conn if attached streams
  MINOR: quic: explicitely ignore sendto error
  CLEANUP: mux-quic: remove loop on sending frames

Christopher Faulet (11):
  Revert "BUG/MINOR: peers: set the proxy's name to the peers section name"
  MINOR: peers: Use a dedicated reconnect timeout when stopping the local 
peer
  BUG/MEDIUM: peers: limit reconnect attempts of the old process on reload
  BUG/MINOR: peers: Use right channel flag to consider the peer as connected
  BUG/MEDIUM: dns: Properly initialize new DNS session
  BUG/MINOR: backend: Don't increment conn_retries counter too early
  MINOR: server: Constify source server to copy its settings
  REORG: server: Export srv_settings_cpy() function
  BUG/MEDIUM: proxy: Perform a custom copy for default server settings
  MINOR: peers: Add a warning about incompatible SSL config for the local 
peer
  BUG/MEDIUM: sink: Set the sink ref for forwarders created during ring 
parsing

Emeric Brun (2):
  BUG/MAJOR: log-forward: Fix log-forward proxies not fully initialized
  BUG/MAJOR: log-forward: Fix ssl layer not initialized on bind even if 
configured

Frédéric Lécaille (17):
  MINOR: quic: Congestion control architecture refactoring
  MEDIUM: quic: Cubic congestion control algorithm implementation
  MINOR: quic: New "quic-cc-algo" bind keyword
  BUG/MINOR: quic: loss time limit variable computed but not used
  MINOR: quic: Stop looking for packet loss asap
  BUG/MAJOR: quic: Useless resource intensive loop qc_ackrng_pkts()
  MINOR: quic: Send packets as much as possible from qc_send_app_pkts()
  BUG/MINOR: quic: Missing in flight ack eliciting packet counter decrement
  BUG/MEDIUM: quic: Floating point exception in cubic_root()
  BUG/MINOR: quic: Avoid sending truncated datagrams
  BUG/MINOR: quic: Missing Initial packet dropping case
  BUG/MEDIUM: quic: Wrong packet length check in qc_do_rm_hp()
  MINOR: quic: Too much useless traces in qc_build_frms()
  BUG/MEDIUM: quic: Missing AEAD TAG check after removing header protection
  BUG/MINOR: quic: Possible infinite loop in 
quic_build_post_handshake_frames()
  BUG/MINOR: quic: memleak on wrong datagram receipt
  MINOR: stick-table: Add table_expire() and table_idle() new converters

Ilya Shipitsin (1):
  CLEANUP: assorted typo fixes in the code and comments

Mateusz Malek (1):
  BUG/MEDIUM: http-ana: fix crash or wrong header deletion by 
http-restrict-req-hdr-names

William Lallemand (4):
  BUG/MINOR: sockpair: wrong return value for fd_send_uxst()
  DEBUG: fd: split the fd check
      MEDIUM: resolvers: continue startup if network is unavailable
  BUG/MINOR: mw

Re: [PATCH] BUG/MEDIUM: http-ana: fix crash or wrong header deletion by http-restrict-req-hdr-names

2022-08-17 Thread Willy Tarreau
On Wed, Aug 17, 2022 at 01:05:55PM +, Mateusz Malek wrote:
> On 17.08.2022 14:59, Mateusz Malek wrote:
> > Sure - here you go:
> 
> Sorry, wrong file. Patch in previous email had a typo (double /req9
> call instead of /req9 and /req10) in VTest test case.

Perfect, thank you, now applied!

I've updated the issue to remember the need to backport as far as 2.2
as you mentioned. I've just backported it to 2.6 already for the upcoming
release and will handle other versions soon.

Thanks again!
Willy



Re: [RFC] [PATCH] BUG: http-ana: fix crash or wrong header deletion by http-restrict-req-hdr-names

2022-08-16 Thread Willy Tarreau
Hi Mateusz,

On Tue, Aug 16, 2022 at 11:58:02PM +, hapr...@sl.damisa.net wrote:
> Hi,
> 
> as suggested by Willy on GitHub, I'm submitting my patch for
> https://github.com/haproxy/haproxy/issues/1822.

Thank you!

> This is my first contribution, so I'm tagging it as RFC for now ;)
> 
> I'm not entirely happy with using goto (suggested by Tim) to avoid
> hitting htx_get_next_blk call at the end of the loop, but I'm not
> familiar with HAproxy coding standards.

We're generally fine with gotos when they're the most efficient
solution (anyway, 1/5 of instructions executed by a CPU are jumps,
so it's pointless to hate gotos, it's just that using too many of
them will make the code less readable).

However here the goto at the beginning potentially maintains a
problem: I'm wondering how we can be certain that htx_remove_blk()
doesn't return a NULL if we remove the last block, in which case
going back to the beginning of the loop without testing it can
be a problem.

> I think it would be nicer to:
> 
> 1. Introduce flag variable preserve_blk
> 2. Reset it to 1 at the beginning of the while (blk) loop
> 3. Replace htx_remove_blk + continue with preserve_blk = 0 + break
> 4. At the end of the loop, call htx_get_next_blk if preserve_blk is set
>or call htx_remove_blk otherwise

Yep I agree with this. I was thinking that an even better option
would be to remove the branches from the inner loop and use it only
to spot unusual chars, something like:

  end = istlen(n);
  for (i = 0; i < end; i++) {
 if (!isalnum() ...)
   break;
  }

  if (i < end) {
   /* unwanted char found */
   if (px->options...)
goto block;
   blk = htx_remove_blk();
   continue;
  }

The for() loop could even be replaced with strcspn(). I just suspect
that for large sets it's slower if it needs to rebuild an internal map,
so we'd probably rather keep the for() loop as-is.

> I have not included severity of the patch, because on GitHub issue is
> still marked as `status: needs-triage`. I think MEDIUM would be
> appropriate.

Yes I think so as well.

> By the way, while writing VTest to cover this bug, I spotted something
> "suspicious" about reg tests for FCGI backends - my-fcgi-app FCGI app is
> defined, but it is not used anywhere? be-fcgi* backends look exactly
> like be-http* to me.

Indeed, I don't know either. I'm wondering if it's not needed just to
enable something by just being present, but I don't see what (and no
comment in the file indicates this). Maybe it's a leftover of an
initial attempt at testing fcgi, I don't know. Let's keep it for now,
we'll check with Christopher when he's back.

But if you can provide an updated patch for the loop, I'm fine with
merging your patch as-is.

Thanks!
Willy



[ANNOUNCE] haproxy-2.7-dev3

2022-08-07 Thread Willy Tarreau
 BUG/MINOR: backend: Fallback on RR algo if balance on source is impossible
  Revert "BUG/MINOR: peers: set the proxy's name to the peers section name"
  MINOR: peers: Add a warning about incompatible SSL config for the local 
peer
  MINOR: peers: Use a dedicated reconnect timeout when stopping the local 
peer
  BUG/MEDIUM: peers: limit reconnect attempts of the old process on reload
  BUG/MINOR: peers: Use right channel flag to consider the peer as connected
  BUG/MEDIUM: dns: Properly initialize new DNS session
  BUG/MINOR: backend: Don't increment conn_retries counter too early
  MINOR: server: Constify source server to copy its settings
  REORG: server: Export srv_settings_cpy() function
  BUG/MEDIUM: proxy: Perform a custom copy for default server settings
  BUG/MEDIUM: sink: Set the sink ref for forwarders created during ring 
parsing

Frédéric Lécaille (13):
  BUG/MAJOR: mux_quic: fix invalid PROTOCOL_VIOLATION on POST data overlap
  MINOR: quic: Congestion control architecture refactoring
  MEDIUM: quic: Cubic congestion control algorithm implementation
  MINOR: quic: New "quic-cc-algo" bind keyword
  BUG/MINOR: quic: loss time limit variable computed but not used
  MINOR: quic: Stop looking for packet loss asap
  BUG/MAJOR: quic: Useless resource intensive loop qc_ackrng_pkts()
  MINOR: quic: Send packets as much as possible from qc_send_app_pkts()
  BUG/MINOR: quic: Missing in flight ack eliciting packet counter decrement
  BUG/MEDIUM: quic: Floating point exception in cubic_root()
  BUG/MINOR: quic: Avoid sending truncated datagrams
  MINOR: quic: Add two new stats counters for sendto() errors
  BUG/MINOR: quic: Missing Initial packet dropping case

Ilya Shipitsin (7):
  BUILD: SSL: allow to pass additional configure args to QUICTLS
  CI: enable weekly "m32" builds on x86_64
  CLEANUP: assorted typo fixes in the code and comments
  BUG/MEDIUM: fix DH length when EC key is used
  REGTESTS: ssl: adopt tests to OpenSSL-3.0.N
  REGTESTS: ssl: adopt tests to OpenSSL-3.0.N
  REGTESTS: ssl: fix grep invocation to use extended regex in 
ssl_generate_certificate.vtc

William Lallemand (15):
  MINOR: resolvers: resolvers_destroy() deinit and free a resolver
  BUG/MINOR: resolvers: shut off the warning for the default resolvers
  BUG/MINOR: ssl: allow duplicate certificates in ca-file directories
  MINOR: init: load OpenSSL error strings
  MINOR: ssl: enhance ca-file error emitting
  BUG/MINOR: mworker/cli: relative pid prefix not validated anymore
  BUG/MEDIUM: mworker: proc_self incorrectly set crashes upon reload
  BUG/MINOR: sockpair: wrong return value for fd_send_uxst()
  MINOR: sockpair: move send_fd_uxst() error message in caller
  DEBUG: fd: split the fd check
  MEDIUM: resolvers: continue startup if network is unavailable
  MINOR: cli: emit a warning when _getsocks was used more than once
  BUG/MINOR: mworker: PROC_O_LEAVING used but not updated
  Revert "MINOR: cli: emit a warning when _getsocks was used more than once"
  MINOR: cli: warning on _getsocks when socket were closed

Willy Tarreau (21):
  BUG/MEDIUM: tools: avoid calling dlsym() in static builds (try 2)
  BUG/MINOR: tools: fix statistical_prng_range()'s output range
  BUG/MEDIUM: fd/threads: fix incorrect thread selection in wakeup broadcast
  BUILD: add detection for unsupported compiler models
  BUG/MEDIUM: master: force the thread count earlier
  BUG/MAJOR: poller: drop FD's tgid when masks don't match
  DEBUG: fd: detect possibly invalid tgid in fd_insert()
  BUG/MINOR: fd: always remove late updates when freeing fd_updt[]
  BUG/MEDIUM: queue/threads: limit the number of entries dequeued at once
  MAJOR: threads/plock: update the embedded library
  MINOR: thread: provide an alternative to pthread's rwlock
  DEBUG: tools: provide a tree dump function for ebmbtrees as well
  MINOR: ebtree: add ebmb_lookup_shorter() to pursue lookups
  BUG/MEDIUM: pattern: only visit equivalent nodes when skipping versions
  BUG/MINOR: ring/cli: fix a race condition between the writer and the 
reader
  BUG/MINOR: sink: fix a race condition between the writer and the reader
  BUG/MINOR: quic: do not reject datagrams matching minimum permitted size
  BUG/MEDIUM: quic: break out of the loop in quic_lstnr_dghdlr
  MINOR: threads: report the number of thread groups in build options
  MINOR: config: automatically preset MAX_THREADS based on MAX_TGROUPS
  BUILD: cfgparse: always defined _GNU_SOURCE for sched.h and crypt.h

---



Re: [PATCH] ubuntu-22.04 related ssl fixes (SECLEVEL related and ec curves renamed)

2022-08-06 Thread Willy Tarreau
On Sat, Aug 06, 2022 at 10:50:15PM +0500,  ??? wrote:
> I accidently lost "-E' flag on grep.
> follow up patch attached.

No problem, thanks for the quic response. At least it seems to work for
me locally, I've just pushed it and we'll see.

Thanks!
Willy



Re: [PATCH] ubuntu-22.04 related ssl fixes (SECLEVEL related and ec curves renamed)

2022-08-06 Thread Willy Tarreau
On Sat, Aug 06, 2022 at 05:48:56PM +0200, Willy Tarreau wrote:
> On Fri, Jul 29, 2022 at 09:37:46PM +0500,  ??? wrote:
> > gentle ping
> 
> Sorry Ilya, but William is in vacation right now. Since I don't think
> there's any risk with your patch, I took it. In the worst case should
> William disagree with it, we could still patch later.

Hmmm I should have been more careful, that broke half of the SSL tests
on ssl_generate_cert, for example:

  https://github.com/haproxy/haproxy/runs/7705930410?check_suite_focus=true

It's exactly on the last changed grep expression. I need to go right now,
so I'll do dev3 later. In the worst case I'll temporarily revert the
series.

Cheers,
Willy



Re: [PATCH] ubuntu-22.04 related ssl fixes (SECLEVEL related and ec curves renamed)

2022-08-06 Thread Willy Tarreau
On Fri, Jul 29, 2022 at 09:37:46PM +0500,  ??? wrote:
> gentle ping

Sorry Ilya, but William is in vacation right now. Since I don't think
there's any risk with your patch, I took it. In the worst case should
William disagree with it, we could still patch later.

Thanks!
Willy



Re: Server timeouts since HAProxy 2.2

2022-08-06 Thread Willy Tarreau
On Thu, Aug 04, 2022 at 12:14:04PM +0200, Vincent Bernat wrote:
> On 2022-08-04 10:35, William Edwards wrote:
> 
> > However,
> > https://haproxy.debian.net/#distribution=Debian=buster=2.2
> > says:
> > 
> > "The Debian HAProxy packaging team provides various versions of HAProxy
> > packages for use on different Debian or Ubuntu systems. The following
> > wizard helps you to find the package suitable for your system. [...] You
> > will get a stable release of HAProxy 2.2: you may not get the latest
> > version but important fixes from later versions are included. Moreover,
> > regressions are unlikely."
> > 
> > The bugs page tries to get users to ALWAYS use the latest version. But
> > the haproxy.debian.org page says that it's okay not to use the latest
> > version.
> 
> That's two different point of views, one from Debian, one from upstream.
> They are difficult to reconcile.

They will not for the simple reason that both have different goals and
difficulties:
  - a project's goal is to reduce the number of bugs because bugs have
direct impact on users experience, cause insatisfaction, and waste
development time trying to analyse already fixed issues.

  - a distro's goal is to limit the risk of *regressions*, because a
distro doesn't have the manpower nor skills to fix issues in every
project, and they're on the first line of bug reports. As such they
prefer to keep known bugs, than risking to break something for
existing users. Users continually experiencing issues will naturally
try another project / distro / version.

The only way for distros to limit the amount of bugs without risking
regressions currently is to ship proven stable versions. But in the
perpetually evolving world of the WWW, standards are dictated by users
(browsers, application componetns etc) and it's not always simple for
users to accept to stay on an older but much stable version.

The best solution to address the needs of users that are in between is what
you're doing, Vincent, with your packages on https://haproxy.debian.net/.
This is by far the best offer one can think of, and I confess that we're
extremely lucky as a project to benefit from this. I understand that not
all projects in a distro could have this, it's a significant extra work.
But it perfectly plugs the hole, and that's why I strongly encourage users
to switch to these packages. They remove some hassle from the distro since
upstream can handle bugs, and improve the users' experience by delivering
fixes for all known bugs.

If that would help, we could even add links to alternate repositories in
the output of "haproxy -v" so that users are more naturally invited to
switch if they feel like it better matches their needs.

Regards,
Willy



Re: haproxy listening on lots of UDP ports

2022-08-06 Thread Willy Tarreau
Hi Shawn,

On Fri, Aug 05, 2022 at 05:18:06PM -0600, Shawn Heisey wrote:
> I am running haproxy in a couple of places.  It is listening on multiple
> seemingly random high UDP ports.

These typically are syslog sockets. In fact the ports are not really
"listening", it's just that in UDP there's no direction so as soon as
the socket is bound, it appears. I think that using "netstat -anu" or
"ss -anu" you should see your syslog servers' addresses in the peer
column.

Note that for syslog, we still take care of setting the rcvbuf to zero
and shutting down the read side so that in the event any datagram would
appear there, it wouldn't uselessly consume socket buffers.

Regards,
Willy



Re: [PATCH] CI: enable weekly "m32" builds

2022-08-06 Thread Willy Tarreau
On Mon, Aug 01, 2022 at 07:40:43PM +0200, Tim Düsterhus wrote:
> The updated patches LGTM.

Thanks guys, now applied!
Willy



Re: [PATCH] speling fixes

2022-08-06 Thread Willy Tarreau
On Fri, Jul 29, 2022 at 10:30:39PM +0500,  ??? wrote:
> Hello,
> 
> yet another spell check fiexs.

Now applied, thanks Ilya!
Willy



Re: [PR] Fix -v flag usage with install(1) on OpenBSD/NetBSD/Solaris/AIX

2022-07-16 Thread Willy Tarreau
On Sat, Jul 16, 2022 at 07:01:22PM -0400, Brad Smith wrote:
> On 7/16/2022 12:52 PM, Willy Tarreau wrote:
> > On Sat, Jul 16, 2022 at 05:18:50AM -0400, Brad Smith wrote:
> > > On Sat, Jul 16, 2022 at 11:09:19AM +0200, Willy Tarreau wrote:
> > > > Looks good. Let's just add a commit message and I'll merge it.
> > > 
> > > BUILD: makefile: Fix install(1) handling for OpenBSD/NetBSD/Solaris/AIX
> > (...)
> > 
> > Applied, thank you Brad!
> > Willy
> 
> Thanks. Can this please be backported to at least 2.6?

Yes, this is reasonable and can indeed be backported.

Willy



Re: Higher Tc than timeout server

2022-07-16 Thread Willy Tarreau
Hi William,

On Sat, Jul 16, 2022 at 06:43:09PM +0200, William Edwards wrote:
> Hi,
> 
> Sorry to bump this, but I haven't made any progress with this on my own.
> Does anyone see what I'm missing here?
> 
> > The Tc timer is documented as:
> > 
> > > - Tc: total time to establish the TCP connection to the server. It's
> > > the time
> > >   elapsed between the moment the proxy sent the connection request,
> > > and the
> > >   moment it was acknowledged by the server, or between the TCP SYN
> > > packet and
> > >   the matching SYN/ACK packet in return. The value "-1" means that the
> > >   connection never established.
> > 
> > The `timeout server` option is documented as:
> > 
> > > The inactivity timeout applies when the server is expected to
> > > acknowledge or
> > > send data.
> > 
> > However, I have a few connections with a higher Tc than `timeout
> > server` (5m), e.g.:
> > 
> > May 25 20:15:33 admin-msh haproxy[1015]: :::127.0.0.1:33406
> > [25/May/2022:03:18:44.905] redis
> > redis/http-msh02.efw.ha.cyberfusion.cloud 1/0/61008353 87670 --
> > 134/113/112/112/0 0/0
> > 
> > Shouldn't the `timeout server` have closed the connection with the sD
> > termination state after 5m if it took 61008353 ms for the server to
> > acknowledge the connection request? Or, more likely, am I misreading
> > the documentation?

Here you're using TCP logs so the timer you're seeing is the total
connection life time from connect to close. The connect time was
zero.

In addition to this, "timeout server" doesn't affect the connect timeout,
only the transfers. It's "timeout connect" which affects the connect
timeout.

Hoping this helps,
Willy



Re: [PR] Fix -v flag usage with install(1) on OpenBSD/NetBSD/Solaris/AIX

2022-07-16 Thread Willy Tarreau
On Sat, Jul 16, 2022 at 05:18:50AM -0400, Brad Smith wrote:
> On Sat, Jul 16, 2022 at 11:09:19AM +0200, Willy Tarreau wrote:
> > Looks good. Let's just add a commit message and I'll merge it.
> 
> 
> BUILD: makefile: Fix install(1) handling for OpenBSD/NetBSD/Solaris/AIX
(...)

Applied, thank you Brad!
Willy



[ANNOUNCE] haproxy-2.7-dev2

2022-07-16 Thread Willy Tarreau
  BUG/MINOR: http-htx: Fix scheme based normalization for URIs wih userinfo
  MINOR: http: Add function to get port part of a host
  MINOR: http: Add function to detect default port
  BUG/MEDIUM: h1: Improve authority validation for CONNCET request
  MINOR: http-htx: Use new HTTP functions for the scheme based normalization
  BUG/MEDIUM: http-fetch: Don't fetch the method if there is no stream
  REGTEESTS: filters: Fix CONNECT request in random-forwarding script
  BUG/MINOR: mux-h1: Be sure to commit htx changes in the demux buffer
  BUG/MEDIUM: http-ana: Don't wait to have an empty buf to switch in TUNNEL 
state
  BUG/MEDIUM: mux-h1: Handle connection error after a synchronous send

Emeric Brun (3):
  MINOR: fd: add a new FD_DISOWN flag to prevent from closing a deleted FD
  BUG/MEDIUM: ssl/fd: unexpected fd close using async engine
  MINOR: fd: Add BUG_ON checks on fd_insert()

Frédéric Lécaille (7):
  MINOR: task: Add tasklet_wakeup_after()
  BUG/MINOR: quic: Dropped packets not counted (with RX buffers full)
  MINOR: quic: Add new stats counter to diagnose RX buffer overrun
  MINOR: quic: Duplicated QUIC_RX_BUFSZ definition
  MINOR: quic: Improvements for the datagrams receipt
  CLEANUP: h2: Typo fix in h2_unsubcribe() traces
  MINOR: quic: Increase the QUIC connections RX buffer size (upto 64Kb)

Ilya Shipitsin (1):
  CI: re-enable gcc asan builds

William Lallemand (4):
  MEDIUM: mworker: set the iocb of the socketpair without using fd_insert()
  CLEANUP: mworker: rename mworker_pipe to mworker_sockpair
  BUG/MINOR: peers: fix possible NULL dereferences at config parsing
  MEDIUM: mworker/systemd: send STATUS over sd_notify

Willy Tarreau (112):
  MINOR: tinfo: make tid temporarily still reflect global ID
  CLEANUP: config: remove unused proc_mask()
  MINOR: debug: remove mask support from "debug dev sched"
  MEDIUM: task: add and preset a thread ID in the task struct
  MEDIUM: task/debug: move the ->thread_mask integrity checks to ->tid
  MAJOR: task: use t->tid instead of ffsl(t->thread_mask) to take the 
thread ID
  MAJOR: task: replace t->thread_mask with 1<tid when thread mask is 
needed
  CLEANUP: task: remove thread_mask from the struct task
  MEDIUM: applet: only keep appctx_new_*() and drop appctx_new()
  MEDIUM: task: only keep task_new_*() and drop task_new()
  MINOR: applet: always use task_new_on() on applet creation
  MEDIUM: task: remove TASK_SHARED_WQ and only use t->tid
  MINOR: task: replace task_set_affinity() with task_set_thread()
  CLEANUP: task: remove the unused task_unlink_rq()
  CLEANUP: task: remove the now unused TASK_GLOBAL flag
  MINOR: task: make rqueue_ticks atomic
  MEDIUM: task: move the shared runqueue to one per thread
  MEDIUM: task: replace the global rq_lock with a per-rq one
  MINOR: task: remove grq_total and use rq_total instead
  MINOR: task: replace global_tasks_mask with a check for tree's emptiness
  MEDIUM: task: use regular eb32 trees for the run queues
  MEDIUM: queue: revert to regular inter-task wakeups
  MINOR: thread: make wake_thread() take care of the sleeping threads mask
  MINOR: thread: move the flags to the shared cache line
  MINOR: thread: only use atomic ops to touch the flags
  MINOR: poller: centralize poll return handling
  MEDIUM: polling: make update_fd_polling() not care about sleeping threads
  MINOR: poller: update_fd_polling: wake a random other thread
  MEDIUM: thread: add a new per-thread flag TH_FL_NOTIFIED to remember 
wakeups
  MEDIUM: tasks/fd: replace sleeping_thread_mask with a TH_FL_SLEEPING flag
  MINOR: tinfo: add the tgid to the thread_info struct
  MINOR: tinfo: replace the tgid with tgid_bit in tgroup_info
  MINOR: tinfo: add the mask of enabled threads in each group
  MINOR: debug: use ltid_bit in ha_thread_dump()
  MINOR: wdt: use ltid_bit in wdt_handler()
  MINOR: clock: use ltid_bit in clock_report_idle()
  MINOR: thread: use ltid_bit in ha_tkillall()
  MINOR: thread: add a new all_tgroups_mask variable to know about active 
tgroups
  CLEANUP: thread: remove thread_sync_release() and thread_sync_mask
  MEDIUM: tinfo: add a dynamic thread-group context
  MEDIUM: thread: make stopping_threads per-group and add stopping_tgroups
  MAJOR: threads: change thread_isolate to support inter-group 
synchronization
  MINOR: thread: add is_thread_harmless() to know if a thread already is 
harmless
  MINOR: debug: mark oneself harmless while waiting for threads to finish
  MINOR: wdt: do not rely on threads_to_dump anymore
  MEDIUM: debug: make the thread dumper not rely on a thread mask anymore
  BUILD: debug: fix build issue on clang with previous commit
  BUILD: debug: re-export thread_dump_state
  BUG/MEDIUM

Re: [PR] Fix -v flag usage with install(1) on OpenBSD/NetBSD/Solaris/AIX

2022-07-16 Thread Willy Tarreau
On Sat, Jul 16, 2022 at 12:57:14AM -0400, Brad Smith wrote:
> How about something like the following?
> 
> 
> diff --git a/Makefile b/Makefile
> index 85f6c632d..5cc59a061 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -81,6 +81,9 @@
>  #   DESTDIR is not set by default and is used for installation only.
>  #   It might be useful to set DESTDIR if you want to install haproxy
>  #   in a sandbox.
> +#   INSTALL is set to "install" by default and is used to provide the name of
> +#   the install binary used by the install targets and any additional
> +#   flags.
>  #   PREFIX  is set to "/usr/local" by default and is used for installation 
> only.
>  #   SBINDIR is set to "$(PREFIX)/sbin" by default and is used for 
> installation
>  #   only.
> @@ -170,6 +173,7 @@ cc-nowarn = $(if $(cc-anywno),-Wno-$(1),$(shell set -e; 
> if $(CC) -Werror -W$(1)
>  
>   Installation options.
>  DESTDIR =
> +INSTALL = install
>  PREFIX = /usr/local
>  SBINDIR = $(PREFIX)/sbin
>  MANDIR = $(PREFIX)/share/man
> @@ -378,6 +382,7 @@ ifeq ($(TARGET),linux-glibc)
>  USE_CPU_AFFINITY USE_THREAD USE_EPOLL USE_LINUX_TPROXY   
>  \
>  USE_ACCEPT4 USE_LINUX_SPLICE USE_PRCTL USE_THREAD_DUMP USE_NS USE_TFO
>  \
>  USE_GETADDRINFO USE_BACKTRACE)
> +  INSTALL = install -v
>  endif
>  
>  # For linux >= 2.6.28, glibc without new features
> @@ -386,6 +391,7 @@ ifeq ($(TARGET),linux-glibc-legacy)
>  USE_POLL USE_TPROXY USE_LIBCRYPT USE_DL USE_RT USE_CRYPT_H USE_NETFILTER 
>  \
>  USE_CPU_AFFINITY USE_THREAD USE_EPOLL USE_LINUX_TPROXY   
>  \
>  USE_ACCEPT4 USE_LINUX_SPLICE USE_PRCTL USE_THREAD_DUMP USE_GETADDRINFO)
> +  INSTALL = install -v
>  endif
>  
>  # For linux >= 2.6.28 and musl
> @@ -395,6 +401,7 @@ ifeq ($(TARGET),linux-musl)
>  USE_CPU_AFFINITY USE_THREAD USE_EPOLL USE_LINUX_TPROXY   
>  \
>  USE_ACCEPT4 USE_LINUX_SPLICE USE_PRCTL USE_THREAD_DUMP USE_NS USE_TFO
>  \
>  USE_GETADDRINFO)
> +  INSTALL = install -v
>  endif
>  
>  # Solaris 10 and above
> @@ -1043,16 +1050,16 @@ src/haproxy.o:src/haproxy.c $(DEP)
>  -c -o $@ $<
>  
>  install-man:
> - $(Q)install -v -d "$(DESTDIR)$(MANDIR)"/man1
> - $(Q)install -v -m 644 doc/haproxy.1 "$(DESTDIR)$(MANDIR)"/man1
> + $(Q)$(INSTALL) -d "$(DESTDIR)$(MANDIR)"/man1
> + $(Q)$(INSTALL) -m 644 doc/haproxy.1 "$(DESTDIR)$(MANDIR)"/man1
>  
>  EXCLUDE_DOCUMENTATION = lgpl gpl coding-style
>  DOCUMENTATION = $(filter-out $(EXCLUDE_DOCUMENTATION),$(patsubst 
> doc/%.txt,%,$(wildcard doc/*.txt)))
>  
>  install-doc:
> - $(Q)install -v -d "$(DESTDIR)$(DOCDIR)"
> + $(Q)$(INSTALL) -d "$(DESTDIR)$(DOCDIR)"
>   $(Q)for x in $(DOCUMENTATION); do \
> - install -v -m 644 doc/$$x.txt "$(DESTDIR)$(DOCDIR)" ; \
> + $(INSTALL) -m 644 doc/$$x.txt "$(DESTDIR)$(DOCDIR)" ; \
>   done
>  
>  install-bin:
> @@ -1062,8 +1069,8 @@ install-bin:
>   exit 1; \
>   fi; \
>   done
> - $(Q)install -v -d "$(DESTDIR)$(SBINDIR)"
> - $(Q)install -v haproxy $(EXTRA) "$(DESTDIR)$(SBINDIR)"
> + $(Q)$(INSTALL) -d "$(DESTDIR)$(SBINDIR)"
> + $(Q)$(INSTALL) haproxy $(EXTRA) "$(DESTDIR)$(SBINDIR)"
>  
>  install: install-bin install-man install-doc

Looks good. Let's just add a commit message and I'll merge it.

Thanks!
Willy



Re: [PR] Fix -v flag usage with install(1) on OpenBSD/NetBSD/Solaris/AIX

2022-07-15 Thread Willy Tarreau
On Sat, Jul 16, 2022 at 12:22:49AM -0400, Brad Smith wrote:
> On 7/15/2022 11:59 PM, Willy Tarreau wrote:
> > Hello,
> > 
> > On Fri, Jul 15, 2022 at 07:27:12PM -0400, Brad Smith wrote:
> > > On 7/15/2022 1:34 AM,  ??? wrote:
> > > > I wonder how do NetBSD/OpenBSD ports work, do they use their own
> > > > "install" invocation instead of "make install" ?
> > > > shouldn't they switch to "make install" ?
> > > NetBSD uses the Makefile's install targets but patches out the -v flag.
> > > OpenBSD used it's own install target
> > > but I'm trying to remove that special casing and I had basically the same
> > > sort of diff NetBSD has. Can't switch
> > > to the make install target until the Makfile is fixed.
> > I agree with the principle of your patch, just not with the way it's done,
> > because this variable "IV" is a bit cryptic and not easy to follow. Other
> > programs also like to redefine the install program, thus I'd propose a
> > simpler and more flexible approach:
> > 
> >- define "INSTALL = install" early, next to DESTDIR and friends
> >- set "INSTALL = install -v" in the LINUX targets
> >- use $(INSTALL) in the install targets
> > 
> > This way it even allows users of any platform to simply pass the INSTALL
> > variable to match their needs (including setting it to "ginstall -v" on
> > non-linux platforms where this often points to GNU install).
> 
> 
> Hi Willy,
> 
> I kind of figured you would not be Ok with it as is. I did think about doing
> something
> as you suggested today as I was thinking about this. I was trying to keep
> the
> variable name short and I agree with you. What you have suggested is typical
> in autoconf / automake environments.

Yes, INSTALL is among the common variables to look for in makefiles.

Willy



Re: SV: Suggestion

2022-07-15 Thread Willy Tarreau
Hi,

On Sun, Jul 10, 2022 at 02:23:41PM +, Henning Svane wrote:
> About IPv4 and IPv6 I was of that impression that when you declared the
> stick-table you also declared it with a type for either ipv4 or ipv6, and it
> was not possible to save both of them in the same table. I have no problem to
> use the extra space if that will work.

If you set "type ipv6" it will indeed store IPv6 addresses, and IPv4
ones will be automatically converted to IPv6 before being looked up,
so an ipv6 table can store both, though if you enumerate it you'll
see IPv6 representation for all addresses.

> But still I can see the need to select between IPv4 and IPv6 when your are
> running dual stack. 

The above will work for this. Since you mentioned the usefulness of options
in your earlier message, let me also point the ".if" preprocessor directives.
These allow to ignore some configuration blocks based on conditions. Thus
you could have something like this in your config to adapt to different
environments:

  global
  presetenv ADDRTYPE "v4"  # or "v6" or "dual"

Then later you can rely on this variable, e.g.

  backend table
  .if streq("${ADDRTYPE}",v6) || streq("${ADDRTYPE}",dual)
stick-table type ipv6 ...
  .else
stick-table type ip ...
  .endif

Similarly you can imagine that some frontends have an IPv4 address and
an optional IPv6 one, and only bind those that are defined:

  global
  presetenv BIND_FOO_IPV4 192.168.10.10
  presetenv BIND_FOO_IPV6 2001:bd8::1010

  frontend foo
  .if defined(BIND_FOO_IPV4)
  bind "${BIND_FOO_IPV4}":1234 name ipv4
  .endif
  .if defined(BIND_FOO_IPV6)
  bind "${BIND_FOO_IPV6}":1234 name ipv6
  .endif

Or even:
  frontend foo
  .if defined(BIND_FOO_IPV4) && defined(BIND_FOO_IPV6)
  bind "${BIND_FOO_IPV4}":1234,"${BIND_FOO_IPV6}":1234 name dual
  .elif defined(BIND_FOO_IPV4)
  bind "${BIND_FOO_IPV4}":1234 name ipv4
  .elif defined(BIND_FOO_IPV6)
  bind "${BIND_FOO_IPV6}":1234 name ipv6
  .endif

To be honest, it makes the confs completely unreadable, but it can ease
config versioning and deployment. I know that some users are placing
all their variables in a separate file, so that they can share a common
core config between multiple machines/clusters and only have a few
variables in system-local config file.

Regards,
Willy



Re: [PR] Fix -v flag usage with install(1) on OpenBSD/NetBSD/Solaris/AIX

2022-07-15 Thread Willy Tarreau
Hello,

On Fri, Jul 15, 2022 at 07:27:12PM -0400, Brad Smith wrote:
> On 7/15/2022 1:34 AM,  ??? wrote:
> > I wonder how do NetBSD/OpenBSD ports work, do they use their own
> > "install" invocation instead of "make install" ?
> > shouldn't they switch to "make install" ?
> 
> NetBSD uses the Makefile's install targets but patches out the -v flag.
> OpenBSD used it's own install target
> but I'm trying to remove that special casing and I had basically the same
> sort of diff NetBSD has. Can't switch
> to the make install target until the Makfile is fixed.

I agree with the principle of your patch, just not with the way it's done,
because this variable "IV" is a bit cryptic and not easy to follow. Other
programs also like to redefine the install program, thus I'd propose a
simpler and more flexible approach:

  - define "INSTALL = install" early, next to DESTDIR and friends
  - set "INSTALL = install -v" in the LINUX targets
  - use $(INSTALL) in the install targets

This way it even allows users of any platform to simply pass the INSTALL
variable to match their needs (including setting it to "ginstall -v" on
non-linux platforms where this often points to GNU install).

Thanks,
Willy



Re: SV: SV: Config will not start on 2.6.1 on Ubuntu 22.04

2022-07-09 Thread Willy Tarreau
On Sat, Jul 09, 2022 at 01:46:03PM +0200, Vincent Bernat wrote:
> On 7/9/22 10:55, Willy Tarreau wrote:
> > On Sat, Jul 09, 2022 at 12:03:02AM +0200, Vincent Bernat wrote:
> > > The error when not running as root is expected. However, the fact it does
> > > not work on boot, then works after is odd. Can you share a minimal
> > > configuration file which exhibits this issue?
> > 
> > That's very strange, it sounds as if the service was not started as
> > root. Was there any change in ubuntu 22 regarding the definition of
> > what user a service starts under ?
> 
> We did some debugging off-list and the IP addresses were handled by
> keepalived and ip_nonlocal_bind sysctl was not enabled.

Ah OK, that indeed totally makes sense in this case. Thanks for the
update!

That makes me think that maybe we could provide a diagnostic here. When
we fail to bind to an ip:port, we could retry with ip:0 and see if we
face the same error. If so we could indicate that the IP isn't present,
this would be helpful in such situations.

Willy



Re: SV: SV: Config will not start on 2.6.1 on Ubuntu 22.04

2022-07-09 Thread Willy Tarreau
On Sat, Jul 09, 2022 at 12:03:02AM +0200, Vincent Bernat wrote:
> The error when not running as root is expected. However, the fact it does
> not work on boot, then works after is odd. Can you share a minimal
> configuration file which exhibits this issue?

That's very strange, it sounds as if the service was not started as
root. Was there any change in ubuntu 22 regarding the definition of
what user a service starts under ?

Willy



[ANNOUNCE] haproxy-2.7-dev1

2022-06-24 Thread Willy Tarreau
xpiration
  CLEANUP: check: Remove useless tests on check's stream-connector
  BUG/MEDIUM: stconn: Don't wakeup applet for send if it won't consume data
  BUG/MEDIUM: cli: Notify cli applet won't consume data during request 
processing
  BUG/MEDIUM: stream: Properly handle destructive client connection upgrades
  MINOR: stream: Rely on stconn flags to abort stream destructive upgrade
  CLEANUP: stconn: Don't expect to have no sedesc on detach
  BUG/MINOR: log: Properly test connection retries to fix dontlog-normal 
option
  BUG/MINOR: http-ana: Set method to HTTP_METH_OTHER when an HTTP txn is 
created
  BUG/MINOR: http-fetch: Use integer value when possible in "method" sample 
fetch
  MINOR: freq_ctr: Add a function to get events excess over the current 
period
  MEDIUM: bwlim: Add support of bandwith limitation at the stream level

Frédéric Lécaille (19):
  BUG/MINOR: quic: Stop hardcoding Retry packet Version field
  MINOR: quic: Add several nonce and key definitions for Retry tag
  BUG/MINOR: quic: Wrong PTO calculation
  MINOR: quic: Parse long packet version from qc_parse_hd_form()
  CLEANUP: quid: QUIC draft-28 no more supported
  MEDIUM: quic: Add QUIC v2 draft support
  MINOR: quic: Released QUIC TLS extension for QUIC v2 draft
  MEDIUM: quic: Compatible version negotiation implementation (draft-08)
  CLEANUP: quic: Remove any reference to boringssl
  BUILD: quic: Wrong HKDF label constant variable initializations
  BUG/MINOR: quic: Unexpected half open connection counter wrapping
  BUG/MINOR: quic_stats: Duplicate "quic_streams_data_blocked_bidi" field 
name
  BUG/MINOR: quic: Acknowledgement must be forced during handshake
  MINOR: quic: Dump version_information transport parameter
  BUG/MINOR: quic: Missing acknowledgments for trailing packets
  BUG/MINOR: quic: Wrong reuse of fulfilled dgram RX buffer
  BUG/MAJOR: quic: Big RX dgrams leak when fulfilling a buffer
  BUG/MAJOR: quic: Big RX dgrams leak with POST requests
  BUILD: quic+h3: 32-bit compilation errors fixes

Glenn Strauss (1):
  OPTIM: mux-h2: increase h2_settings_initial_window_size default to 64k

Remi Tricot-Le Breton (1):
  BUG/MINOR: ssl: Do not look for key in extra files if already in pem

Tim Duesterhus (1):
  CLEANUP: Re-apply xalloc_size.cocci (2)

William Lallemand (3):
  BUG/MEDIUM: ssl/cli: crash when crt inserted into a crt-list
  REGTESTS: ssl: add the same cert for client/server
  BUG/MEDIUM: mworker: use default maxconn in wait mode

Willy Tarreau (31):
  BUILD: compiler: implement unreachable for older compilers too
  DEV: tcploop: reorder options in the usage message
  DEV: tcploop: make the current address the default address
  DEV: tcploop: make it possible to change the target address of a connect()
  DEV: tcploop: factor out the socket creation
  DEV: tcploop: permit port 0 to ease handling of default options
  DEV: tcploop: add a new "bind" command to bind to ip/port.
  DEV: tcploop: add minimal UDP support
  MEDIUM: mux-h2: try to coalesce outgoing WINDOW_UPDATE frames
  BUG/MINOR: cli/stats: add missing trailing LF after JSON outputs
  BUG/MINOR: server: do not enable DNS resolution on disabled proxies
  BUG/MINOR: cli/stats: add missing trailing LF after "show info json"
  DOC: design: update the notes on thread groups
  MINOR: task: move profiling bit to per-thread
  CLEANUP: quic: use task_new_on() for single-threaded tasks
  MINOR: tinfo: remove the global thread ID bit (tid_bit)
  CLEANUP: hlua: check for at least 2 threads on a task
  MINOR: thread: get rid of MAX_THREADS_MASK
  OPTIM: task: do not consult shared WQ when we're already full
  DOC: design: update the task vs thread affinity requirements
  BUG/MINOR: task: fix thread assignment in tasklet_kill()
  MINOR: hlua: don't dump empty entries in hlua_traceback()
  MINOR: hlua: add a new hlua_show_current_location() function
  MEDIUM: debug: add a tainted flag when a shared library is loaded
  MEDIUM: debug: detect redefinition of symbols upon dlopen()
  MINOR: intops: add a function to return a valid bit position from a mask
  TESTS: add a unit test for one_among_mask()
  BUILD: ssl_ckch: fix "maybe-uninitialized" build error on gcc-9.4 + ARM
  BUG/MINOR: stream: only free the req/res captures when set
  CLEANUP: pool/tree-wide: remove suffix "_pool" from certain pool names
  MEDIUM: debug: improve DEBUG_MEM_STATS to also report pool alloc/free

---



Re: lua: Add missed lua 5.4 references

2022-06-21 Thread Willy Tarreau
Hi Christian,

On Tue, Jun 21, 2022 at 11:05:09PM +0200, Christian Ruppert wrote:
> Hey guys,
> 
> is there any news on this or got this one just lost? I couldn't find a
> response to it so I assume it just got lost.
> Or is there anything against it?
> To bad forwarding doesn't work and since this mail is quite old already:
> https://www.mail-archive.com/haproxy@formilux.org/msg39689.html

Oh it definitely got lost in the noise. Could someone please repost it
with a slightly clearer description and fixing wrapping issues so that
we can apply it ?

Thanks!
Willy



Re: [PATCH 1/1]: MINOR __builtin_memcpy_inline usage introduction

2022-06-20 Thread Willy Tarreau
Hi David,

On Sat, Jun 18, 2022 at 12:52:23PM +0100, David CARLIER wrote:
> From 9d7b6448a2407451c3115b701c51f97ab2bf6a59 Mon Sep 17 00:00:00 2001
> From: David Carlier 
> Date: Sat, 18 Jun 2022 12:41:11 +0100
> Subject: [PATCH] MINOR: compiler __builtin_memcpy_inline usage introduction.
> 
> Optimised version of the existing __builtin_memcpy builtin, differences
> reside by the fact it works only with constant time sizes and does
>  generate extra calls.
> At the moment, is clang exclusive even tough GCC does not seem to
>  want to implement it, the comments try not to reflect this current
> state.
> Usage can be expanded, used purposely only in few places for starter.
> ---
>  include/haproxy/compiler.h | 12 
>  src/haproxy.c  |  4 ++--
>  src/log.c  |  2 +-
>  src/proxy.c|  2 +-
>  4 files changed, 16 insertions(+), 4 deletions(-)
> 
> diff --git a/include/haproxy/compiler.h b/include/haproxy/compiler.h
> index 66b5f5835..dca9f6aef 100644
> --- a/include/haproxy/compiler.h
> +++ b/include/haproxy/compiler.h
> @@ -192,6 +192,18 @@
>  #endif
>  #endif
>  
> +/*
> + * This builtin works only with compile time lengths unlike __builtin_memcpy.
> + * Also guarantees no external call is generated.
> + * We could `replicate` the aforementioned constraint with a
> + * _Static_assert/__builtin_constant_p theoretically, but that might be
> + * too much trouble to make it happens (C11 min)
> + * so here we trust the proper usage with other compilers (and the CI 
> infrastructure).
> + */
> +#if !defined(__clang__) || __clang_major__ < 11
> +#define __builtin_memcpy_inline(x, y, s) memcpy(x, y, s)
> +#endif
> +

Hmmm please no, this is not the right way to do it for two reasons:
  - it will encourage use of __builtin_memcpy_inline() making one believe
it's the expected one when it isn't ;

  - developing on non-clang systems will not report the problem with the
non-constant size that __builtin_memcpy_inline() expects, resulting
in breakage from time to time.

I'd suggest that instead you create a macro named ha_memcpy_inline() that
maps to __builtin_memcpy_inline() when present and s is a builtin constant,
or maps to __builtin_memcpy() for the rest. Please note, by the way, that
gcc since at least 3.4 has been emitting the same instructions (rep movsb)
as clang's __builtin_memcpy_inline(), but only when optimizing for size.
When optimizing for anything else, it can emit ton of inlined garbage^Wvector
instructions.

Thus I suspect we could achieve the same result by doing a clever
arrangement of #pragma GCC push_options / #pragma GCC optimize("Os,inline")
around an inline function definition that does the memcpy, and that is
called by the macro. I have not tried, but probably something like this
would do it:

#pragma GCC push_options
#pragma GCC optimize("Os,inline")
static inline void _ha_memcpy_inline(void *restrict dest, const void *restrict 
src, size_t n)
{
__builtin_memcpy(dest, src, n);
}
#pragma GCC pop_options

#if defined(clang...)
#define ha_memcpy_inline(d,s,n) do { if (__builtin_constant_p(n)) 
__builtin_memcpy_inline(d,s,n); else _ha_memcpy_inline(d,s,n); } while (0)
#else
#define ha_memcpy_inline(d,s,n) _ha_memcpy_inline(d,s,n)
#endif

That's not even compile-tested and I haven't looked at the result, but
you get the idea.

Now regarding where to use it... I'd say it depends on a lot of factors,
size, speed, undesired dependency on external libs. I think that each and
every call place does warrant an individual commit with a justification
and a check that the benefit matches expectations. There could be some
value in using this in various low-level protocol layers (QUIC, HTX, SPOE).
Maybe more, I don't know.

Thanks,
Willy



Re: [haproxy/haproxy] OPTIM/MINOR: h2_settings_initial_window_size default 64k (PR #1732)

2022-06-08 Thread Willy Tarreau
On Wed, Jun 08, 2022 at 09:22:31AM -0400, Glenn Strauss wrote:
> Since DATA frames might be in flight on the network, the server may want
> to be able to buffer twice the advertisted window size and defer sending
> WINDOW_UPDATE once the advertised window size is buffered.  Doing so
> gives the elephant in the pipe just enough space for a well-behaved
> client adhering to the window size restrictions, and without the
> undesirable thought of delaying the send WINDOW_UPDATE frames by RTT.

Yes exactly. I wasn't sure if you had taken into consideration that it
required extra buffering, which was the point of me raising that warning.
But it looks like you're handling this so it looks OK from my POV as
well.

Willy



Re: [haproxy/haproxy] OPTIM/MINOR: h2_settings_initial_window_size default 64k (PR #1732)

2022-06-08 Thread Willy Tarreau
On Wed, Jun 08, 2022 at 08:29:48AM -0400, Glenn Strauss wrote:
> > I agree that it's independent but it's the one that is not expected to
> > cause any regression with any possible client. That's why I'd like to
> > have the two. First that one because it should be durable. Second, your
> > patch as an optimization, and if anyone complains about trouble with it
> > we can safely revert it or decide what to do without worrying about all
> > deployed curl clients that will not be updated.
> 
> Were this to cause a theoretical regression, this default initial window
> size can be configured in haproxy, so the workaround is to configure the
> value back to 65535.

Sure, that can be a workaround. But when it causes regressions in the
middle of a stable branch and affects users, the correct fix is to revert
and keep it only for major releases, with release notes advertising the
change of behavior.

> > > The upcoming lighttpd 1.4.65
> > > does just that, collecting updates until 16384 is reached, and then
> > > sending a WINDOW_UPDATE for 16384.  Cost: a single 1-RTT delay in
> > > sending a WINDOW_UPDATE to continue uploads larger than 112k-1.
> > 
> > It can definitely cause some inefficient clients to hang if they need
> > their buffer to be fully acked and their size is not a multiple of the
> > frame size. Be careful about this.
> 
> I ended up adjusting this before releasing lighttpd 1.4.65.
> 
> For the same few instructions, lighttpd 1.4.65 now sends WINDOW_UPDATE
> for 16384 when lighttpd receives the first DATA frame containing data
> (1-16384 bytes), and then does not send WINDOW_UPDATE until the next
> 16384 block is started.  This avoids any delay in sending WINDOW_UPDATE,
> though it does (temporarily) increase the window size allowed in the
> client if the client has not already sent data consuming the window.

OK.

> > > Another approach might be to pre-emptively effectively double the
> > > window size by sending WINDOW_UPDATE with the entire window size (65535)
> > > back to the client upon receive of first DATA frame, and then keeping
> > > track on server-side until that 65535 is exhausted before sending
> > > another WINDOW_UPDATE.
> > 
> > Sending too large stream window sizes is problematic when some streams
> > can be processed slowly, as it easily causes head-of-line blocking. If
> > the advertised window is larger than the available storage, a paused
> > stream will block other ones.
> 
>  we know that poorly written code knows no bounds.  Since the
> server is permitted to increase the window size, inability by the client
> to increase the window size is a bug in the client.  The client need
> only track the window size -- the client need not actually use the extra
> window size if the client does not wish to do so.

I was not speaking about the client but the server. If the server
advertises more than it can buffer itself, you can end up with blocked
streams: there's an elephant in the pipe that you can't pull completely
because you have nowhere to store it, and because of this you can't
access other frames that follow (the funniest ones being window updates
from the client that are expected to release some room on the response
side, but that basically only happens with echo servers). But you can
leave headers frames hanging making the client think the server is taking
time to process the requests while in fact the requests are not yet parsed,
they're just blocked.

Willy



Re: [haproxy/haproxy] OPTIM/MINOR: h2_settings_initial_window_size default 64k (PR #1732)

2022-06-08 Thread Willy Tarreau
Hello Glenn,

On Tue, Jun 07, 2022 at 05:24:09PM -0400, Glenn Strauss wrote:
> On Tue, Jun 07, 2022 at 09:27:43AM -0700, Willy Tarreau wrote:
> > Hello Glenn,
> > 
> > Thanks for your report, I understand the problem, that's very interesting.
> > I would say it's not even an optim, rather an approach trying to help the
> > whole ecosystem at a very low cost by preventing known degenerative cases
> > from happening (or helping them recover).
> 
> Reference to issue (for the list)
> https://github.com/haproxy/haproxy/pull/1732
> https://github.com/nghttp2/nghttp2/issues/1722
> https://github.com/curl/curl/pull/8965

Yeah I've read those as well, thanks for the background and for reaching
out various implementations, by the way. You could even have brought the
discussion to the IETF HTTP working group to reach all of them at once,
and maybe someone would remember why there was this choice of 65535 back
then (I suspect it was done with IoT in mind where a window size had to
be stored on 16 bit to save space, but I could be wrong).

> Summary: degenerative HTTP/2 client behavior sending mostly 1-byte DATA
> frames for a fast client which continually exhausts the HTTP/2 send
> window (e.g. for a large file upload) *and* for a simple client which
> empties and reuses a single power-2 sized buffer.  A simple server
> sending WINDOW_UPDATE for each DATA frame to replenish the window may
> reflect back the exact size of the DATA frame.
> 
> > However I don't fully agree with the solution, it might just push the issue
> > a bit sidewards.
> 
> Changing the default SETTINGS_INITIAL_WINDOW_SIZE does push the issue a
> bit sideways, but in doing so mitigates a demonstrated (not theoretical)
> issue which might befall 6+ years of curl installations, a sizable
> number of which might not be upgraded for a very long time.
> 
> > For example a client might already have worked around this issue by using a
> > 65537-bytes buffer and will now face issues again.
> 
> I do not follow.  How might that be?
> Also, do you know of any such implementations?

No, it's just to illustrate. In the HTTP ecosystem, you rarely work
around one implementation's corner case without affecting another
workload. Or similarly, a client that would read exactly 65535 at
once could end up doing this with the change (please note that I'm
purposely making this up to illustrate):

 read 65535
 send 16384, 16384, 16384, 16383
 read 1
 send 1
 receive WU for 65535 (the first 4 frames)
 read 65535
 send 16384, 16384, 16384, 16383
 receive WU for 1
 read 1
 send 1
 ...

This would result in a client being less efficient by doubling the
number of read() syscalls. I'm not particularly worried about this,
to be honest, it's just that it's typically what could cause an issue
to be filed about an observed regression in field. In this case we'd
have to revert it and that would reopen the problem with curl.

> For a slightly more intelligent client actively trying to avoid the
> degenerative behavior, a simple approach is to avoid injecting tiny DATA
> frames when the window is larger and there is more data to be sent.  I
> have proposed a patch to curl in https://github.com/curl/curl/pull/8965

I totally agree, but we also know that it's harder to fix embedded
clients in cars and central heating devices than to deploy a
reasonable workaround on the few servers that receive their connections.

> > Thus I'd first prefer to solve it at the core, then possibly merge this
> > patch later as an optimization but nor for the problem, rather to make sure
> > clients can make 4 full 16kB frames, and improve bandwidth and CPU
> > efficiency.
> 
> Yes, having the server merge the sizes of small DATA frames into a
> larger WINDOW_UPDATE might be useful, too, but is independent and
> complimentary to my patch proposed in
> https://github.com/haproxy/haproxy/pull/1732

I agree that it's independent but it's the one that is not expected to
cause any regression with any possible client. That's why I'd like to
have the two. First that one because it should be durable. Second, your
patch as an optimization, and if anyone complains about trouble with it
we can safely revert it or decide what to do without worrying about all
deployed curl clients that will not be updated.

> > Given the nature of the problem, I don't imagine we'd meet this issue with
> > interleaved 1-byte frames from multiple streams over the same connection.
> 
> Actually, haproxy sets the session window size to be a very large
> number, effectively removing HTTP/2 application-level flow-control
> at the session level.  That leaves independent streams with the
> default SETTINGS_INITIAL_WINDOW_SIZE of 65535, so multiple uploads
> can independently exhaust the stream send window and run into this
> d

Re: Rate Limiting with token/leaky bucket algorithm

2022-06-07 Thread Willy Tarreau
On Tue, Jun 07, 2022 at 01:51:06PM +0200, Seena Fallah wrote:
> I also tried with this one but this will give me 20req/s 200 OK and the
> rest of it 429 too many requests
> ```
> listen test
> bind :8000
> stick-table  type ip  size 100k expire 30s store http_req_rate(1s)
> acl exceeds_limit src_http_req_rate gt 100
> http-request track-sc0 src unless exceeds_limit
> http-request deny deny_status 429 if exceeds_limit
> http-request return status 200 content-type "text/plain" lf-string "200
> OK"
> ```
> 
> Maybe the "1s" isn't handled correctly? when I fetch the current value for
> the http_req_rate it is 100 so that makes sense other requests get 429 but
> actually, only 20req/s is responding "200" because the http_req_rate is not
> decreasing in the correct intervals!

There is a reason to this, which is subtle: the counter is updated when
the track action is performed. As such, each new request refreshes the
counter and the counter reports the total number of *received* requests
and not the number of accepted requests.

There are different ways to deal with this, usually they involve a check
*before* the track. With your config it's trivial since you're already
using src_http_req_rate which performs its own lookup. Just move the
track_sc rule at the end and it should be OK.

Willy



Re: [ANNOUNCE] haproxy-2.6.0

2022-06-03 Thread Willy Tarreau
On Fri, Jun 03, 2022 at 11:43:32PM +0200, Vincent Bernat wrote:
>  ? 31 May 2022 17:56 +02, Willy Tarreau:
> 
> > HAProxy 2.6.0 was released on 2022/05/31. It added 57 new commits
> > after version 2.6-dev12, essentially small bug fixes, QUIC counters
> > and doc updates.
> 
> It's available on haproxy.debian.net. No QUIC support as neither Debian
> nor Ubuntu has the appropriate library.

Many thanks for this, Vincent!

Willy



Re: deviceatlas compiler error

2022-06-03 Thread Willy Tarreau
Hello Amol,

On Fri, Jun 03, 2022 at 11:09:07AM +0530, Amol Arote wrote:
> We are trying to upgrade deviceatlas for HAProxy version 2.4.2-553dee3, but
> while compiling deviceatlas its showing some error.
> Below are the versions and steps which we perform for the same.

Thanks for the report. Adding David who's the maintainer as I don't know
if he watches the list often.

Amol, please be aware that 235 fixes among which 21 rated as "major"
were applied to the 2.4 branch after your version, as such you're urged
to stop using it and to update it. But that shouldn't be related to your
DA build issue.

David, please find the rest of the report below.

Thanks!
Willy

> ---
> *Versions*
> ---
> HAProxy version 2.4.2-553dee3 2021/07/07
> cmake version 2.8.12.2
> CentOS Linux release 7.6.1810 (Core)
> deviceatlas-enterprise-c-2.4.0.zip
> ---
> *deviceatlas compile steps*
> ---
> # yum install cmake
> # unzip deviceatlas-enterprise-c-2.4.0.zip
> # mv deviceatlas-enterprise-c-2.4.0  deviceatlas
> # cd /opt/deviceatlas/Src/
> # cmake . -DCMAKE_INSTALL_PREFIX=/opt/deviceatlas
> # make
> ---
> *Error while compiling woth command *
> * # cmake . -DCMAKE_INSTALL_PREFIX=/opt/deviceatlas   *
> ---
> [root@tt Src]## cmake . -DCMAKE_INSTALL_PREFIX=/opt/deviceatlas
> -- The C compiler identification is GNU 4.8.5
> -- The CXX compiler identification is GNU 4.8.5
> -- Check for working C compiler: /usr/bin/cc
> -- Check for working C compiler: /usr/bin/cc -- works
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Check for working CXX compiler: /usr/bin/c++
> -- Check for working CXX compiler: /usr/bin/c++ -- works
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Found PCRE: /usr/include
> -- Could NOT find CURL (missing:  CURL_LIBRARY CURL_INCLUDE_DIR)
> -- Found ZLIB: /usr/lib64/libz.so (found version "1.2.7")
> -- Performing Test HAVE_BUILTIN__BOOL
> -- Performing Test HAVE_BUILTIN__BOOL - Success
> -- Could NOT find ZIP
> -- Performing Test HAS_STD_ATOMICS
> -- Performing Test HAS_STD_ATOMICS - Failed
> -- Performing Test HAS_BUILTINS_ATOMICS
> -- Performing Test HAS_BUILTINS_ATOMICS - Success
> -- Performing Test HAS_ATTR_COLD
> -- Performing Test HAS_ATTR_COLD - Success
> -- Performing Test HAS_ATTR_ALLOC
> -- Performing Test HAS_ATTR_ALLOC - Failed
> -- Performing Test HAS_WIN32_ATOMICS
> -- Performing Test HAS_WIN32_ATOMICS - Failed
> -- Performing Test HAS_WIN32_ATTR_ALLOC
> -- Performing Test HAS_WIN32_ATTR_ALLOC - Failed
> -- Performing Test HAS_WIN32_UNUSED
> -- Performing Test HAS_WIN32_UNUSED - Failed
> --  version
> -- Configuring done
> -- Generating done
> -- Build files have been written to: /opt/deviceatlas/Src
> [root@tt Src]#  #
> 
> Request you to please guide us on above matter
> 
> 
> -- 
> 
> Regards,
> 
> 
> 
> Amol Arote
> 
> Senior IT Manager
> 
> 
> 
> *Mobile*: 9773868585 | 8097988585
> 
> *Phone:*  (022) 61934700 Ext 444
> 
> *Email:* amol.ar...@naaptol.com
> 
> *Web:* *https://www.naaptol.com *
> 
> -- 
> 



Re: [ANNOUNCE] haproxy-2.6.0

2022-05-31 Thread Willy Tarreau
On Tue, May 31, 2022 at 07:16:31PM +0200, Tim Düsterhus wrote:
> Willy,
> 
> you're probably expected this type of email from me :-)
> 
> On 5/31/22 17:56, Willy Tarreau wrote:
> > HAProxy 2.6.0 was released on 2022/05/31. It added 57 new commits
> 
> I guess the release of 2.6.0 is an appropriate time to put 2.0 into
> "critical fixes" only. Back in January we discussed doing that in May and
> luckily it's may for another ~5 hours here in central Europe :-)
> 
> see https://www.mail-archive.com/haproxy@formilux.org/msg41672.html

Ah yes, nice reminder, now done just in time!

> With that HAProxy 2.3 can likely get its final release for the last two
> commits and then be EOL'd.

Yes, we can either release as-is or have a quick check for anything
absolutely earth-shattering and backport yet a few more fixes, or we
could completely ignore it. I'm going to mark it unmaintained right
now, and we'll likely issue a last one in a few days, for those who
are a bit late to upgrade.

> On another note: For the Docker fans, I've already created a Pull Request to
> add 2.6 proper / 2.7 dev, so those should be available soon-ish:
> 
> https://github.com/docker-library/haproxy/pull/185

Thank you!

> Enough of being partypooper Tim. Enjoy your release party!

Will likely happen next week, I'm busy at Kernel Recipes till the
end of the week now. 2.5 years without a conf was too much, let's
switch back a little bit to kernel mode for a few days ;-)

Cheers,
Willy



Re: [haproxy/docs PATCH] Replace `primary` with `info` for HAProxy 2.5 on index.html

2022-05-31 Thread Willy Tarreau
On Tue, May 31, 2022 at 07:01:37PM +0200, Tim Düsterhus wrote:
> Willy,
> 
> On 5/31/22 18:26, Willy Tarreau wrote:
> > On Tue, May 31, 2022 at 06:15:48PM +0200, Tim Duesterhus wrote:
> > > 2.5 is neither the newest stable version, nor the newest LTS version, thus
> > > there is no reason for it to be highlighted.
> > 
> > Ah you're absolutely right. I left it on purpose but I guess my brain
> > was completely washed by the long release process :-)
> 
> I was happy to see that you didn't forgot about the (new) docs repository,
> so that's a win :-)

I take notes to avoid missing steps ;-)

> > Ack by me, feel free to push it!
> > 
> 
> Perfect, done: 
> https://github.com/haproxy/docs/commit/edbd0284d6bd0507295ebe7e7e72beee2e39abb4

Thanks!
Willy



Re: [haproxy/docs PATCH] Replace `primary` with `info` for HAProxy 2.5 on index.html

2022-05-31 Thread Willy Tarreau
On Tue, May 31, 2022 at 06:15:48PM +0200, Tim Duesterhus wrote:
> 2.5 is neither the newest stable version, nor the newest LTS version, thus
> there is no reason for it to be highlighted.

Ah you're absolutely right. I left it on purpose but I guess my brain
was completely washed by the long release process :-)

Ack by me, feel free to push it!

Thanks,
Willy



[ANNOUNCE] haproxy-2.6.0

2022-05-31 Thread Willy Tarreau
ets.

Version 2.6 also improves management: the dynamic servers feature is
no longer experimental, so those who were waiting for a stabilized
language before writing their tools will be pleased to know that it's
time to adopt it!  In addition, the new CertCache Lua class allows the
certificates to be entirely manipulated and updated from Lua code, thus
providing even more live updates.

Another issue that was met a few times on modern systems was processes
starting with too high a file-descriptor limit (typically one billion),
eating a lot of memory on startup, because it would match the hard limit
of the process. There's now a global option "fd-hard-limit" that allows
to set an upper bound to the FDs but still respect the limit assigned to
the process if it's lower. This should improve reliability in field and
resource management in general.

Some users rightfully complained about CPU peaks on reload due to all
older connections being stopped in a very short time and having to be
reopened on the new process. A new global option "close-spread-time" now
allows to define a time window over which the unused idle connections
will be closed, and it's even possible never to close them if desired.

Speaking of connections, a recurring criticism (even by ourselves as
users) was the difficulty of figuring which "deny" or "reject" rule
caused the termination of a request or connection. A pair of sample fetch
functions "last_rule_file" and "last_rule_line" will respectively report
the location of the rule that was last executed. This is conveniently
used in logs.

As usual with a new major release, performance considerations were not
put at rest! HAProxy 2.6 features some refinements to its multi-threaded
scheduler that allowed to completely remove the lock contention when
processing server queues. Some performance gains up to 20 were observed
on large machines, where the scalability is now much more linear.

The low-level connection layer was significantly reworked to make it much
more maintainable and to shorten its depth. That's rare but there's both
a better abstraction and a shorter path to access everything, and this
resulted in the ability to scale outgoing peers connections to all threads
instead of starting them all on the first one like before.

Finally the long discussions in the issue tracker continue to feed ideas
to provide better diagnostics. In HAProxy 2.6, the master CLI now features
a debug mode that provides the entirety of the regular CLI commands,
allowing to spot (or eliminate) dead connections or any other issue. The
different memory models can be finely tuned without rebuilding, via the
extended "-dM" option, which means that in case of doubt about a possible
memory corruption it will now be possible to just restart with an option
and watch the process. The list of configuration, command-line keywords,
actions, converters etc can be dumped by "-dK". This may be used by those
writing config parsers or syntax coloring rules for editors.

Please note that there are a few potentially user-visible changes in this
version:
  - SSL engines now being disabled by default, as mentioned above ;
  - openssl 0.9.8 support being dropped
  - the HTTP version in HTTP/1.1 requests now no longer accepts "RTSP"
unless "option accept-invalid-http-requests" is used; that was causing
grief to a number of users because there was no easy way to prevent
these requests from passing, causing a 502 error on the response path
that appeared in the logs and stats despite the server being innocent.

This version benefitted from these new contributors, who I hope found the
experience fun and will continue to participate in the future:

   Andrew McDermott, Boyang Li, Dhruv Jain, Julien Thomas,
   Nikola Sale, Thomas Prückl, vigneshsp

and the following returning contributors:

   Aleksandar Lazic, Amaury Denoyelle, Bertrand Jacquin,
   Christian Ruppert, Christopher Faulet, Daniel Jakots,
   David Carlier, Emeric Brun, Frédéric Lécaille, Ilya Shipitsin,
   Lukas Tribus, Maciej Zdeb, Marno Krahmer, Miroslav Zagorac,
   Remi Tricot-Le Breton, Thayne McCombs, Thierry Fournier,
   Tim Duesterhus, William Dauchy, William Lallemand, Willy Tarreau

During HAProxy 2.6 development, some nice updates were made to help
users and early adopters. Cyril Bonté helped Tim and I integrate the doc
generation in the CI to publish it on https://docs.haproxy.org/. Cyril
should now be less bothered by updates and the online doc is now always
the freshest possible. That's great and I really want to thank Cyril for
his dedication over the last decade to maintain this level of quality.

During this last development cycle, William also put in place a build
system packages of the development version, which we hope will help those
who want to deploy development configurations without going through the
burden of having to often rebuild themselves:

   

Re: [PATCH] DOC: Fix formatting in configuration.txt to fix dconv

2022-05-27 Thread Willy Tarreau
On Fri, May 27, 2022 at 11:20:36PM +0200, Tim Duesterhus wrote:
> The missing space before the colon causes haproxy-dconv to misparse the
> configuration.txt.

Thanks Tim, now merged.
Willy



[ANNOUNCE] haproxy-2.6-dev12

2022-05-27 Thread Willy Tarreau
overflow when parsing SETTINGS
  MINOR: h3: refactor h3_control_send()
  MINOR: quic: support CONNECTION_CLOSE_APP emission
  MINOR: mux-quic: disable read on CONNECTION_CLOSE emission
  MINOR: h3: reject too big frames
  MINOR: mux-quic: emit STREAM_STATE_ERROR in qcc_recv
  BUG/MINOR: mux-quic: refactor uni streams TX/send H3 SETTINGS
  MINOR: h3/qpack: use qcs as type in decode callbacks
  MINOR: h3: define stream type
  MINOR: h3: refactor uni streams initialization
  MINOR: h3: check if frame is valid for stream type
  MINOR: h3: define non-h3 generic parsing function
  MEDIUM: quic: refactor uni streams RX
  CLEANUP: h3: remove h3 uni tasklet
  MINOR: h3: abort read on unknown uni stream
  MINOR: h3: refactor SETTINGS parsing/error reporting

Christopher Faulet (3):
  BUG/MEDIUM: resolvers: Don't defer resolutions release in deinit function
  BUG/MINOR: task: Don't defer tasks release when HAProxy is stopping
  Revert "BUG/MINOR: task: Don't defer tasks release when HAProxy is 
stopping"

Emeric Brun (2):
  BUG/MEDIUM: peers: fix segfault using multiple bind on peers sections
  BUG/MEDIUM: peers: prevent unitialized multiple listeners on peers section

Thayne McCombs (1):
  BUG/MEDIUM: sample: Fix adjusting size in word converter

Tim Duesterhus (5):
  CLEANUP: tools: Clean up non-QUIC error message handling in str2sa_range()
  BUG/MEDIUM: tools: Fix `inet_ntop` usage in sa2str
  CLEANUP: tools: Crash if inet_ntop fails due to ENOSPC in sa2str
  BUG/MEDIUM: http: Properly reject non-HTTP/1.x protocols
  REGTESTS: Do not use REQUIRE_VERSION for HAProxy 2.5+ (2)

William Lallemand (2):
  DOC: configuration: add a warning for @system-ca on bind
      BUG/MINOR: ssl/lua: use correctly cert_ext in CertCache.set()

Willy Tarreau (110):
  CLEANUP: init: address a coverity warning about possible multiply overflow
  MEDIUM: h1: enlarge the scope of accepted version chars with 
accept-invalid-http-request
  CLEANUP: init: address another coverity warning about a possible multiply 
overflow
  CLEANUP: conn_stream: remove unneeded exclusion of RX_WAIT_EP from 
RXBLK_ANY
  CLEANUP: conn_stream: rename the cs_endpoint's context to "conn"
  MINOR: conn_stream: add new sets of functions to set/get endpoint flags
  DEV: coccinelle: add cs_endp_flags.cocci
  CLEANUP: conn_stream: apply cs_endp_flags.cocci tree-wide
  DEV: coccinelle: add endp_flags.cocci
  CLEANUP: conn_stream: apply endp_flags.cocci tree-wide
  CLEANUP: conn_stream: rename the stream endpoint flags CS_EP_* to  SE_FL_*
  CLEANUP: conn_stream: rename the cs_endpoint's target to "se"
  CLEANUP: conn_stream: rename cs_endpoint to sedesc (stream endpoint 
descriptor)
  CLEANUP: applet: rename the sedesc pointer from "endp" to "sedesc"
  CLEANUP: conn_stream: rename the conn_stream's endp to sedesc
  CLEANUP: conn_stream: rename cs_app_* to sc_app_*
  CLEANUP: conn_stream: tree-wide rename to stconn (stream connector)
  CLEANUP: mux-h1: add and use h1s_sc() to retrieve the stream connector
  CLEANUP: mux-h2: add and use h2s_sc() to retrieve the stream connector
  CLEANUP: mux-fcgi: add and use fcgi_strm_sc() to retrieve the stream 
connector
  CLEANUP: mux-pt: add and use pt_sc() to retrieve the stream connector
  CLEANUP: stdesc: rename the stream connector ->cs field to ->sc
  CLEANUP: stream: rename "csf" and "csb" to "scf" and "scb"
  CLEANUP: stconn: tree-wide rename stream connector flags CS_FL_* to 
SC_FL_*
  CLEANUP: stconn: tree-wide rename stconn states CS_ST/SB_* to SC_ST/SB_*
  MINOR: check: export wake_srv_chk()
  MINOR: conn_stream: test the various ops functions before calling them
  MEDIUM: stconn: merge the app_ops and the data_cb fields
  MINOR: applet: add new wrappers to put chk/blk/str/chr to channel from 
appctx
  CLEANUP: applet: use applet_put*() everywhere possible
  CLEANUP: stconn: rename cs_{i,o}{b,c} to sc_{i,o}{b,c}
  CLEANUP: stconn: rename cs_{check,strm,strm_task} to sc_strm_*
  CLEANUP: stconn: rename cs_conn() to sc_conn()
  CLEANUP: stconn: rename cs_mux() to sc_mux_strm()
  CLEANUP: stconn: rename cs_conn_mux() to sc_mux_ops()
  CLEANUP: stconn: rename cs_appctx() to sc_appctx()
  CLEANUP: stconn: rename __cs_endp_target() to __sc_endp()
  CLEANUP: stconn: rename cs_get_data_name() to sc_get_data_name()
  CLEANUP: stconn: rename cs_conn_*() to sc_conn_*()
  CLEANUP: stconn: rename cs_conn_get_first() to conn_get_first_sc()
  CLEANUP: stconn: rename cs_ep_set_error() to se_fl_set_error()
  CLEANUP: stconn: make a few functions take a const argument
  CLEANUP: stconn: use a single function to know if SC may send to SE
  MINOR: stconn: consider CF_SHUT

Re: [PATCH] BUG/MEDIUM: sample: Fix adjusting size in word converter

2022-05-27 Thread Willy Tarreau
On Wed, May 25, 2022 at 10:58:51PM -0600, astrotha...@gmail.com wrote:
> From: Thayne McCombs 
> 
> Adjust the size of the sample buffer before we change the "area"
> pointer. Otherwise, we end up not changing the size, because the area
> pointer is already the same as "start" before we compute the difference
> between the two.
> 
> This is similar to the change in b28430591d18f7fda5bac2e0ea590b3a34f04601
> but for the word converter instead of field.

Good catch, now merged, thank you Thayne!
Willy



Re: [PATCH] REGTESTS: Do not use REQUIRE_VERSION for HAProxy 2.5+ (2)

2022-05-27 Thread Willy Tarreau
On Mon, May 23, 2022 at 10:45:36PM +0200, Tim Duesterhus wrote:
> Introduced in:
> 
> 18c13d3bd MEDIUM: http-ana: Add a proxy option to restrict chars in request 
> header names
(...)

Merged, thanks Tim!
Willy



Re: [ANNOUNCE] haproxy-2.6-dev11

2022-05-23 Thread Willy Tarreau
Hi Ilya,

On Tue, May 24, 2022 at 09:53:01AM +0500,  ??? wrote:
> Hello,
> 
> can we please address https://github.com/haproxy/haproxy/issues/1585 before
> final 2.6 ?

I thought it was since I replied it was an FP but OK, I pushed a patch
to silence it.

Thanks,
Willy



Re: [PATCH v2] CLEANUP: tools: Crash if inet_ntop fails due to ENOSPC in sa2str

2022-05-23 Thread Willy Tarreau
On Mon, May 23, 2022 at 09:30:49AM +0200, Tim Duesterhus wrote:
> This is impossible, because we pass a destination buffer that is appropriately
> sized to hold an IPv6 address.

Applied now, thank you Tim!
Willy



Re: Peers using heavily single cpu core

2022-05-23 Thread Willy Tarreau
Hi Maciej,

On Mon, May 23, 2022 at 08:50:53AM +0200, Maciej Zdeb wrote:
> Hi Christopher,
> I've verified that outgoing connections are now spread between multiple
> threads! Thank you very much!

That's really great, thank you for testing! I, too, thought it was
worth being merged even this late in the dev cycle. The way it was
done allows to postpone conversion of other applets, so the risk of
breaking something remains fairly limited. We'll see.

Cheers,
Willy



Re: [PATCH 2/2] CLEANUP: tools: Crash if inet_ntop fails in sa2str

2022-05-23 Thread Willy Tarreau
On Sun, May 22, 2022 at 01:06:28PM +0200, Tim Duesterhus wrote:
> @@ -1374,7 +1374,10 @@ char * sa2str(const struct sockaddr_storage *addr, int 
> port, int map_ports)
>   default:
>   return NULL;
>   }
> - inet_ntop(addr->ss_family, ptr, buffer, sizeof(buffer));
> + if (inet_ntop(addr->ss_family, ptr, buffer, sizeof(buffer)) == NULL) {
> + BUG_ON("inet_ntop failed to convert");
> + return NULL;
> + }

Hmm no, please at least check errno for ENOSPC on this one, because some
minimalistic libcs (or older ones) properly report EAFNOSUPPORT when
passed AF_INET6 addresses that they do not support, and we don't want
to crash at runtime when an address is updated..

Maybe such a variant instead ?

-   inet_ntop(addr->ss_family, ptr, buffer, sizeof(buffer));
+   if (inet_ntop(addr->ss_family, ptr, buffer, sizeof(buffer)) == NULL) {
+   BUG_ON(erro == ENOSPC);
+   return NULL;
+   }

Willy



Re: [PATCH 1/2] BUG/MEDIUM: tools: Fix `inet_ntop` usage in sa2str

2022-05-23 Thread Willy Tarreau
On Sun, May 22, 2022 at 01:06:27PM +0200, Tim Duesterhus wrote:
> The given size must be the size of the destination buffer, not the size of the
> (binary) address representation.
> 
> This fixes GitHub issue #1599.
> 
> The bug was introduced in 92149f9a82a9b55c598f1cc815bc330c555f3561 which is in
> 2.4+. The fix must be backported there.

Ah good catch, merged, thanks Tim!
Willy



Re: [PATCH] CLEANUP: tools: Clean up non-QUIC error message handling in str2sa_range()

2022-05-23 Thread Willy Tarreau
On Sun, May 22, 2022 at 12:40:58PM +0200, Tim Duesterhus wrote:
> If QUIC support is enabled both branches of the ternary conditional are
> identical, upsetting Coverity. Move the full conditional into the non-QUIC
> preprocessor branch to make the code more clear.
> 
> This resolves GitHub issue #1710.

OK that's fine like this. Now merged, thank you Tim!
Willy



[ANNOUNCE] haproxy-2.6-dev11

2022-05-21 Thread Willy Tarreau
eeBSD 13.1

David Carlier (2):
  BUILD: fix build warning on solaris based systems with __maybe_unused.
  MINOR: tools: add get_exec_path implementation for solaris based systems.

Frédéric Lécaille (15):
  MINOR: quic: Dump initial derived secrets
  MINOR: quic_tls: Add quic_tls_derive_retry_token_secret()
  MINOR: quic_tls: Add quic_tls_decrypt2() implementation
  MINOR: quic: Retry implementation
  MINOR: cfgparse: Update for "cluster-secret" keyword for QUIC Retry
  MINOR: quic: Move quic_lstnr_dgram_dispatch() out of xprt_quic.c
  BUILD: stats: Missing headers inclusions from stats.h
  MINOR: quic_stats: Add a new stats module for QUIC
  MINOR: quic: Attach proxy QUIC stats counters to the QUIC connection
  BUG/MINOR: quic: Fix potential memory leak during QUIC connection 
allocations
  MINOR: quic: QUIC stats counters handling
  MINOR: quic: Add tune.quic.retry-threshold keyword
  MINOR: quic: Dynamic Retry implementation
  BUG/MINOR: quic: Fixe a typo in qc_idle_timer_task()
  BUG/MINOR: quic: Missing  stats counter decrementation

Ilya Shipitsin (2):
  CI: determine actual LibreSSL version dynamically
  CI: determine actual OpenSSL version dynamically

Maciej Zdeb (2):
  MINOR: peers: Track number of applets run by thread
  MEDIUM: peers: Balance applets across threads

Remi Tricot-Le Breton (5):
  MEDIUM: ssl: Delay random generator initialization after config parsing
  MINOR: ssl: Add 'ssl-propquery' global option
  MINOR: ssl: Add 'ssl-provider' global option
  BUG/MINOR: ssl: Fix crash when no private key is found in pem
  MINOR: ssl: Add 'ssl-provider-path' global option

Tim Duesterhus (4):
  CLEANUP: Add missing header to ssl_utils.c
  CLEANUP: Add missing header to hlua_fcn.c
  CLEANUP: Remove unused function hlua_get_top_error_string
  CLEANUP: http_ana: Make use of the return value of 
stream_generate_unique_id()

Willy Tarreau (20):
  BUG/MINOR: cfgparse: abort earlier in case of allocation error
  BUG/MINOR: peers: fix error reporting of "bind" lines
  CLEANUP: config: improve address parser error report for unmatched 
protocols
  CLEANUP: config: provide cleare hints about unsupported QUIC addresses
  MINOR: protocol: replace ctrl_type with xprt_type and clarify it
  MINOR: listener: provide a function to process all of a bind_conf's 
arguments
  MINOR: config: use the new bind_parse_args_list() to parse a "bind" line
  CLEANUP: listener: add a comment about what the BC_SSL_O_* flags are for
  MINOR: listener: add a new "options" entry in bind_conf
  CLEANUP: listener: replace all uses of bind_conf->is_ssl with BC_O_USE_SSL
  CLEANUP: listener: replace bind_conf->generate_cers with 
BC_O_GENERATE_CERTS
  CLEANUP: listener: replace bind_conf->quic_force_retry with 
BC_O_QUIC_FORCE_RETRY
  CLEANUP: listener: store stream vs dgram at the bind_conf level
  MINOR: listener: detect stream vs dgram conflict during parsing
  MINOR: listener: set the QUIC xprt layer immediately after parsing the 
args
  MINOR: listener/ssl: set the SSL xprt layer only once the whole config is 
known
  MINOR: connection: add flag MX_FL_FRAMED to mark muxes relying on framed 
xprt
  MINOR: config: detect and report mux and transport incompatibilities
  MINOR: listener: automatically select a QUIC mux with a QUIC transport
  MINOR: listener: automatically enable SSL if a QUIC transport is found

---



Re: [PATCH] CI: determine actual OpenSSL version dynamically

2022-05-20 Thread Willy Tarreau
On Fri, May 20, 2022 at 11:10:28PM +0500,  ??? wrote:
> Hello,
> 
> another small improvement, this change introduce "OPENSSL_VERSION=latest"
> semantic.

Applied, thank you Ilya!
Willy



Re: Increase SSL Key Generation after upgrade from 2.4.15 to 2.4.17

2022-05-20 Thread Willy Tarreau
Hi Tomasz,

On Fri, May 20, 2022 at 05:17:19PM +0200, Tomasz Ludwiczak wrote:
> Hi,
> 
> I am seeing an increase in SSL Key Generation after upgrading from 2.4.15
> to 2.4.17. I have not changed the openssl version. Does anyone have an idea
> what this could be related to?
> I have looked at the changes from 2.4.16 and 2.4.17 and nothing obvious
> pointing to changes around TLS reuse.

Interesting, I've reviewed the fixes merged between the two and cannot
find anything relevant. Do you have copies of the "show info" output
before the upgrade to compare before and after ? There are SSL lookups
and misses there. These could give some hints about what is happening.
Have you tried reverting to 2.4.15 to see if the problem disappears ?
We could for example imagine that it's concommittant with another change
that happened during the same upgrade (e.g. openssl lib upgrade), even
if I would find it unlikely as well. Are you certain you didn't change
any tuning option in the config between the two versions ? For example
reducing the size of the SSL session cache could make a difference.

It would be useful if you could also test with 2.4.16 to help figure if
that's related to a change between 2.4.15->16 or 2.4.16->17.

Regards,
Willy



Re: Paid feature development: TCP stream compression

2022-05-20 Thread Willy Tarreau
On Fri, May 20, 2022 at 04:20:45PM +0500,  ??? wrote:
> yes, it was I meant actually. haproxy currently is not suitable for
> compressing tcp streams. even if such feature will be considered as useful,
> it will take time.

Compression is not done on TCP but since it's done using a filter that
deals with HTTP compression, I imagine that it wouldn't be too hard to
modify the filter not to emit HTTP chunks and to work on top of plain
TCP. My real concern is for the decompression. At 256 kB per stream,
that's quite a no-go :-(

Willy



Re: Paid feature development: TCP stream compression

2022-05-20 Thread Willy Tarreau
On Fri, May 20, 2022 at 12:16:07PM +0100, Mark Zealey wrote:
> Thanks, we may use this for a very rough proof-of-concept. However we are
> dealing with millions of concurrent connections, 10-100 million connections
> per day, so we'd prefer to pay someone to develop (+ test!) something for
> haproxy which will work at this scale

That's a big problem with gzip. While compression can be done stateless
(which is what we're doing with SLZ), decompression uses roughly 256 kB
of permanent RAM per *stream*. That's 256 GB of RAM for just one million
connections, 1 TB of RAM for just 4 million connections. Sadly, that's a
perfect example of use case that requires extreme horizontal scalability
and that's better kept away from the LB and performed on servers.

Isn't there any way to advertise support or lack of for compression ?
Because if your real need is to extract contents to perform any form of
processing, we could imagine that instead of having to decompress the
stream, it would be better if we could interfere with the connection
setup to disable compression.

I'm just trying to figure a reasonable alternative, because like this in
addition to being extremely unscalable, it makes your LBs a trivial DoS
target.

Regards,
Willy



Re: GitHub Issue Tracker: New "Close Reason" feature

2022-05-20 Thread Willy Tarreau
Hi Tim,

On Thu, May 19, 2022 at 09:18:16PM +0200, Tim Düsterhus wrote:
> Hi!
> 
> as a heads up for the folks with issue tracker access:
> 
> https://github.blog/changelog/2022-05-19-the-new-github-issues-may-19th-update/
> 
> GitHub updated the issue tracker to basically" allow specifying whether an
> issue is closed as "resolved" or as "won't do". The selection is available
> in the dropdown of the close button.
> 
> As of now this feature does little more than using a slightly different icon
> + color for the closed issue, but I think it would be useful selecting the
> correct variant going forward, in case GitHub extends this feature in the
> future.

Indeed, I've already felt that something like this was missing. Now whether
we'll think about it or not, and it will be convenient, we'll see.

Thanks for the heads up!
Willy



Re: [PATCH 1/1] : BUILD/MINOR cpuset build fix for FreeBSD 13.1

2022-05-20 Thread Willy Tarreau
Hi David,

On Wed, May 18, 2022 at 03:50:04PM +0100, David CARLIER wrote:
> Hi,
> 
> FreeBSD 13.1 had been released this week and here a little fix for the
> cpuset part.

Merged, thank you!
Willy



Re: [PATCH] CLEANUP: http_ana: Make use of the return value of stream_generate_unique_id()

2022-05-17 Thread Willy Tarreau
On Wed, May 18, 2022 at 12:22:15AM +0200, Tim Duesterhus wrote:
> Even if `unique_id` and `s->unique_id` are identical it is a bit odd to
> `isttest()` `unique_id` and then use `s->unique_id` in the call to 
> `http_add_header()`.

Agreed, better be consistent. Now applied, thank you Tim!
Willy



Re: [PATCH 1/1]: BUILD/MINOR: solaris based oses build fix/get_exe_path implementation.

2022-05-17 Thread Willy Tarreau
Both patches merged, thanks David!
Willy



Re: [PATCH v2 2/3] CLEANUP: Add missing header to hlua_fcn.c

2022-05-17 Thread Willy Tarreau
On Sat, May 14, 2022 at 10:17:25PM +0200, Tim Duesterhus wrote:
> Found with -Wmissing-prototypes:
(...)

All the series merged (with v2), thank you Tim!
Willy



Re: [PATCH 1/1]: BUILD/MINOR: solaris based oses build fix/get_exe_path implementation.

2022-05-14 Thread Willy Tarreau
Hi David,

> From 5b175adfa5ef9ab52ce69f7eb6775efe8a828974 Mon Sep 17 00:00:00 2001
> From: David Carlier 
> Date: Fri, 13 May 2022 20:16:15 +0100
> Subject: [PATCH] BUILD/MINOR: few solaris updates.
> 
> - get_exec_path using getexecname, fetching AT_SUN_EXECNAME from the
>   auxiliary vectors.
> - __maybe_unused already defined.

Please split it in two, as it really addresses two completely independent
things. It seems that the former improves error reporting while the
second one is a build fix.

Thanks,
Willy



Re: [PATCH] CI: determine actual LibreSSL version dynamically

2022-05-14 Thread Willy Tarreau
> From da2b295f45ecc6d99559ef147569514816ad6f7c Mon Sep 17 00:00:00 2001
> From: Ilya Shipitsin 
> Date: Fri, 13 May 2022 21:59:38 +0500
> Subject: [PATCH] CI: determine actual LibreSSL version dynamically
> 
> this change introduce "LIBRESSL_VERSION=latest" semantic, which scans
> http://ftp.openbsd.org/pub/OpenBSD/LibreSSL/ and detects latest release.
> 
> LIBRESSL_VERSION=2.9.2 is removed from the matrix.

OK, applied, thanks Ilya!
Willy



[ANNOUNCE] haproxy-2.6-dev10

2022-05-14 Thread Willy Tarreau
   Code reports : http://www.haproxy.org/l/code-reports
   Latest builds: http://www.haproxy.org/l/dev-packages

Willy
---
Complete changelog :
Amaury Denoyelle (13):
  MINOR: ncbuf: define non-contiguous buffer
  MINOR: ncbuf: complete API and define block interal abstraction
  MINOR: ncbuf: optimize storage for the last gap
  MINOR: ncbuf: implement insertion
  MINOR: ncbuf: define various insertion modes
  MINOR: ncbuf: implement advance
  MINOR: ncbuf: write unit tests
  BUG/MINOR: ncbuf: fix coverity warning on uninit sz_data
  MINOR: xprt_quic: adjust flow-control according to bufsize
  MEDIUM: mux-quic/h3/hq-interop: use ncbuf for bidir streams
  MEDIUM: mux-quic/h3/qpack: use ncbuf for uni streams
  CLEANUP: mux-quic: remove unused fields for Rx
  CLEANUP: quic: remove unused quic_rx_strm_frm

Boyang Li (2):
  BUG/MEDIUM: lua: fix argument handling in data removal functions
  DOC/MINOR: fix typos in the lua-api document

Christopher Faulet (2):
  MINOR: mux-h1: Add global option accpet payload for any HTTP/1.0 requests
  CLEANUP: mux-h1: Fix comments and error messages for global options

Emeric Brun (1):
  BUG/MAJOR: dns: multi-thread concurrency issue on UDP socket

Frédéric Lécaille (16):
  MINOR: quic: Add a debug counter for sendto() errors
  BUG/MINOR: quic: Dropped peer transport parameters
  BUG/MINOR: quic: Wrong unit for ack delay for incoming ACK frames
  MINOR: quic: Congestion controller event trace fix (loss)
  MINOR: quic: Add correct ack delay values to ACK frames
  MINOR: config: Add "cluster-secret" new global keyword
  MINOR: quic-tls: Add quic_hkdf_extract_and_expand() for HKDF
  MINOR: quic: new_quic_cid() code moving
  MINOR: quic: Initialize stateless reset tokens with HKDF secrets
  MINOR: qc_new_conn() rework for stateless reset
  MINOR: quic: Stateless reset token copy to transport parameters
  MINOR: quic: Send stateless reset tokens
  MINOR: quic: Short packets always embed a trailing AEAD TAG
  CLEANUP: quic: wrong use of eb*entry() macro
  CLEANUP: quic: Useless use of pointer for quic_hkdf_extract()
  CLEANUP: quic_tls: QUIC_TLS_IV_LEN defined two times

Remi Tricot-Le Breton (1):
  BUG/MINOR: ssl: Fix typos in crl-file related CLI commands

William Lallemand (4):
  MINOR: ssl: ignore dotfiles when loading a dir w/ ca-file
  MEDIUM: ssl: ignore dotfiles when loading a dir w/ crt
  DOC: configuration: add the httpclient keywords to the global keywords 
index
      BUG/MEDIUM: wdt: don't trigger the watchdog when p is unitialized

Willy Tarreau (34):
  MINOR: compiler: add a new macro to set an attribute on an enum when 
possible
  BUILD: stats: conditionally mark obsolete stats states as deprecated
  BUILD: ssl: work around bogus warning in gcc 12's -Wformat-truncation
  BUILD: debug: work around gcc-12 excessive -Warray-bounds warnings
  BUILD: listener: shut report of possible null-deref in listener_accept()
  BUG/MEDIUM: ssl: fix the gcc-12 broken fix :-(
  DOC: install: update gcc version requirements
  BUILD: makefile: add -Wfatal-errors to the default flags
  BUG/MINOR: mux-h2: mark the stream as open before processing it not after
  MINOR: mux-h2: report a trace event when failing to create a new stream
  MINOR: conn_stream: make cs_set_error() work on the endpoint instead
  CLEANUP: mux-h1: always take the endp from the h1s not the cs
  CLEANUP: mux-h2: always take the endp from the h2s not the cs
  CLEANUP: mux-pt: always take the endp from the context not the cs
  CLEANUP: mux-fcgi: always take the endp from the fstrm not the cs
  CLEANUP: mux-quic: always take the endp from the qcs not the cs
  CLEANUP: applet: use the appctx's endp instead of cs->endp
  MINOR: conn_stream: add a pointer back to the cs from the endpoint
  MINOR: mux-h1: remove the now unneeded h1s->cs
  MINOR: mux-h2: make sure any h2s always has an endpoint
  MINOR: mux-h2: remove the now unneeded conn_stream from the h2s
  MINOR: mux-fcgi: make sure any stream always has an endpoint
  MINOR: mux-fcgi: remove the now unneeded conn_stream from the fcgi_strm
  MINOR: mux-quic: remove the now unneeded conn_stream from the qcs
  MINOR: mux-pt: remove the now unneeded conn_stream from the context
  CLEANUP: muxes: make mux->attach/detach take a conn_stream endpoint
  MINOR: applet: replace cs_applet_shut() with appctx_shut()
  MINOR: applet: add appctx_strm() and appctx_cs() to access common fields
  CLEANUP: applet: remove the unneeded appctx->owner
  CLEANUP: conn_stream: merge cs_new_from_{mux,applet} into 
cs_new_from_endp()
  MINOR: ext-check: indicate the transport and protocol of a server
  BUG/MEDIUM: mux-quic: fix a thinko in the latest cs/endpoint cleanup
  MINOR: tools: improve 

Re: Fwd: Set environment variables

2022-05-13 Thread Willy Tarreau
Hi Valerio,

On Mon, May 09, 2022 at 10:14:09AM +0200, Valerio Pachera wrote:
> Unfortunately I'm not a developer so it will take too much time form me to
> contribute to the code.

I've just implemented it in 2.6-dev as this commit:

https://github.com/haproxy/haproxy/commit/973cf90714

Once 2.6 is released we may backport it to 2.5 as it's expected to be
trivial and with no side effect.

Willy



Re: Fwd: Set environment variables

2022-05-11 Thread Willy Tarreau
On Mon, May 09, 2022 at 10:14:09AM +0200, Valerio Pachera wrote:
> Thank you very much willy for your reply.
> Unfortunately I'm not a developer so it will take too much time form me to
> contribute to the code.

No problem, do not worry. I've added an issue for this one:

   https://github.com/haproxy/haproxy/issues/1692

Thanks!
Willy



Re: 2.5: Possibility to upgrade http/1.0 clients to http/1.1?

2022-05-11 Thread Willy Tarreau
On Wed, May 11, 2022 at 08:43:38AM +, Froehlich, Dominik wrote:
> Hi Willy,
> 
> Thanks for the fruitful discussion!
> 
> I've opened https://github.com/haproxy/haproxy/issues/1691 to track this 
> feature request.

Thanks for this, Dominik!

Willy



Re: Patch for GitHub Issue 1530

2022-05-11 Thread Willy Tarreau
Hi Vignesh,

On Mon, May 09, 2022 at 05:38:44PM +, Vig Nesh wrote:
> Hello Team Haproxy,
> 
> Thanks for providing an opportunity to work with the product, I have
> submitted a patch for issue 1530
>  along with this email.

Now applied, thank you!
Willy



Re: 2.5: Possibility to upgrade http/1.0 clients to http/1.1?

2022-05-09 Thread Willy Tarreau
Hi Dominik,

On Mon, May 09, 2022 at 08:46:20AM +, Froehlich, Dominik wrote:
> Hi Willy,
> 
> Thanks for your response.
> 
> Yes, I agree an option that can be turned on would be the most feasible
> solution for us.
>
> I can think of a similar option like we have for "option
> h1-case-adjust-bogus-server"
> that allows you to tell HAProxy whether to touch header casing or not.

Yes I think it would make sense because overall when you know you have
to relax this, it's more based on the LB's location than on a specific
frontend or backend, so it could just be a global option.

> Could be called "option h1-reject-get-head-delete-with-payload-bogus-client" 
> or something.

Note, we want to leave it disabled by default so that only those with
this unusual requirement would have to set the option, so the option's
name should translate that it has to accept the requests instead, so it
could be something like "h1-no-method-restriction-for-payload" or
"h1-accept-payload-with-any-method", or something like this.

> As for your question how the clients ended up sending a body with such 
> requests:
> 
> The API the client is using allows them to send a DELETE request with a JSON
> document that lists all the resources to be deleted.

Ah I see, that kind of makes sense. I'm wondering if Elastic Search does
not do something similar, I seem to remember some discussions around this
when we were working on tightening the rule in the spec.

> So instead of
> 
> DELETE /books/1
> 
> They would send
> 
> DELETE /books
> {
>   "delete": [1,2,3]
> }
> 
> Weird, I know but it allowed them to delete more than one resource at a time.
> (of course, this could also be put on the path, but well... that's how they
> did it back then)

It's possible that it's easier to specify complex sets. It definitely
helps anticipate possible trouble to be expected at some places. Thanks
for the explanation!

Willy



[ANNOUNCE] haproxy-2.6-dev9

2022-05-08 Thread Willy Tarreau
ssion for closed streams

Ilya Shipitsin (1):
  CI: dynamically determine actual version of h2spec

William Lallemand (20):
  MEDIUM: httpclient: remove url2sa to use a more flexible parser
  MEDIUM: httpclient: http-request rules for resolving
  MEDIUM: httpclient: allow address and port change for resolving
  CLEANUP: httpclient: remove the comment about resolving
  MINOR: httpclient: handle unix and other socket types in dst
  MINOR: httpclient: rename dash by dot in global option
  MINOR: init: exit() after pre-check upon error
  MINOR: httpclient: cleanup the error handling in init
  MEDIUM: httpclient: hard-error when SSL is configured
  MINOR: httpclient: allow to configure the ca-file
  MINOR: httpclient: configure the resolvers section to use
  MINOR: httpclient: allow ipv4 or ipv6 preference for resolving
  DOC: configuration: httpclient global option
  MINOR: resolvers: cleanup alert/warning in parse-resolve-conf
  MINOR: resolvers: move the resolv.conf parser in parse_resolv_conf()
  MINOR: resolvers: resolvers_new() create a resolvers with default values
  BUG/MINOR: tcp/http: release the expr of set-{src,dst}[-port]
  MEDIUM: resolvers: create a "default" resolvers section at startup
  DOC: resolvers: default resolvers section
  BUG/MINOR: startup: usage() when no -cc arguments

Willy Tarreau (89):
  CLEANUP: backend: make alloc_{bind,dst}_address() idempotent
  MEDIUM: stream: remove the confusing SF_ADDR_SET flag
  MINOR: conn_stream: remove the now unused CS_FL_ADDR_*_SET flags
  CLEANUP: protocol: make sure the connect_* functions always receive a dst
  MINOR: connection: get rid of the CO_FL_ADDR_*_SET flags
  MINOR: session: get rid of the now unused SESS_FL_ADDR_*_SET flags
  BUILD: debug: unify the definition of ha_backtrace_to_stderr()
  BUG/MEDIUM: resolvers: make "show resolvers" properly yield
  BUG/MEDIUM: cli: make "show cli sockets" really yield
  BUG/MINOR: proxy/cli: don't enumerate internal proxies on "show backend"
  BUG/MINOR: map/cli: protect the backref list during "show map" errors
  BUG/MINOR: map/cli: make sure patterns don't vanish under "show map"'s 
init
  BUG/MINOR: ssl/cli: fix "show ssl ca-file/crl-file" not to mix cli+ssl 
contexts
  BUG/MINOR: ssl/cli: fix "show ssl ca-file " not to mix cli+ssl 
contexts
  BUG/MINOR: ssl/cli: fix "show ssl crl-file" not to mix cli+ssl contexts
  BUG/MINOR: ssl/cli: fix "show ssl cert" not to mix cli+ssl contexts
  CLEANUP: ssl/cli: do not loop on unknown states in "add ssl crt-list" 
handler
  MINOR: applet: reserve some generic storage in the applet's context
  CLEANUP: applet: make appctx_new() initialize the whole appctx
  CLEANUP: stream/cli: take the "show sess" context definition out of the 
appctx
  CLEANUP: stream/cli: stop using appctx->st2 for the dump state
  CLEANUP: stream/cli: remove the unneeded init state from "show sess"
  CLEANUP: stream/cli: remove the unneeded STATE_FIN state from "show sess"
  CLEANUP: stream/cli: remove the now unneeded dump state from "show sess"
  CLEANUP: proxy/cli: take the "show errors" context definition out of the 
appctx
  CLEANUP: stick-table/cli: take the "show table" context definition out of 
the appctx
  CLEANUP: stick-table/cli: stop using appctx->st2 for the dump state
  CLEANUP: stick-table/cli: remove the unneeded STATE_INIT for "show table"
  CLEANUP: map/cli: take the "show map" context definition out of the appctx
  CLEANUP: map/cli: stop using cli.i0/i1 to store the generation numbers
  CLEANUP: map/cli: stop using appctx->st2 for the dump state
  CLEANUP: map/cli: always detach the backref from the list after "show map"
  CLEANUP: peers/cli: take the "show peers" context definition out of the 
appctx
  CLEANUP: peers/cli: stop using appctx->st2 for the dump state
  CLEANUP: peers/cli: remove unneeded state STATE_INIT
  CLEANUP: cli: initialize the whole appctx->ctx, not just the stats part
  CLEANUP: promex: make the applet use its own context
  CLEANUP: promex: stop using appctx->st2
  CLEANUP: stats/cli: take the "show stat" context definition out of the 
appctx
  CLEANUP: stats/cli: stop using appctx->st2
  CLEANUP: hlua/cli: take the hlua_cli context definition out of the appctx
  CLEANUP: ssl/cli: use a local context for "show cafile"
  CLEANUP: ssl/cli: use a local context for "show crlfile"
  CLEANUP: ssl/cli: use a local context for "show ssl cert"
  CLEANUP: ssl/cli: use a local context for "commit ssl cert"
  CLEANUP: ssl/cli: 

Re: [PATCH 1/1: BUILD/MINOR: TCP_KEEPIDLE macos equivalence

2022-05-08 Thread Willy Tarreau
On Sun, May 08, 2022 at 12:39:13PM +0200, Vincent Bernat wrote:
>  ?  8 May 2022 10:57 +02, Willy Tarreau:
> 
> > After edition (still minimal and possibly inaccurate but the best I
> > could do):
> >
> > On Linux the interval before starting to send TCP keep-alive packets
> > is defined by TCP_KEEPIDLE. MacOS has an equivalent with TCP_KEEPIDLE,
> > which also uses seconds as a unit, so it's possible to simply remap the
> > definition of TCP_KEEPIDLE to TCP_KEEPALIVE there and get it to 
> > seamlessly
> > work. The other settings (interval and count) are not present,
> > though.
> 
> First instance of TCP_KEEPIDLE should be TCP_KEEPALIVE. ;-)

Oops, thanks, that's exactly why I hate having to document others' work
myself. It's so easy to get something wrong :-/

Willy



Re: [PATCH 1/1: BUILD/MINOR: TCP_KEEPIDLE macos equivalence

2022-05-08 Thread Willy Tarreau
On Sun, May 08, 2022 at 10:21:28AM +0100, David CARLIER wrote:
> On Sun, 8 May 2022 at 09:57, Willy Tarreau  wrote:
> >
> > On Sun, May 01, 2022 at 03:33:17PM +0100, David CARLIER wrote:
> > > Hi here a little patch to set idle time for SO_KEEPALIVE socket option.
> >
> > Now merged, thanks.
> >
> > David, one comment though, your commit messages keep missing a lot of
> > crucial information for reviewers and debuggers, and I had to spend time
> > documenting myself on keep-alive on MacOS to figure the differences and/or
> > what the impacts or limitations of the patch could be. I finally found and
> > put that information into the commit message, but it would be much easier
> > if you could put it yourself after your development since you actually
> > have access to the information I had to seek, and know the reasoning
> > behind your choice.
> >
> 
> Thanks, fair point :-) I ll take it in account

Thanks ;-)

> even tough I was certain for this one it would not break anything.

Quite frankly, experience tells us we really never know. If MacOS in
the future version would define the TCP_KEEPIDLE macro, that would be
sufficient to break the build for example.

> > The new choice was explained again. Finally, two weeks later I got a
> > report of breakage when using external code linked at run time, the
> > keyword registration didn't work anymore due to a mistake in this last
> > patch. Fortunately, the design decision was explained and I could
> > restart from this, figure my mistake and make sure that the 3rd attempt
> > at a fix this time addressed all 3 identified use cases:
> 
> Do you have a reasonable numbers of macOs users or is it a rare occurrence ?

I really don't know, here it was on the CI. Outside of the CI, breakage
is usually reported in this order:
  - Linux
  - FreeBSD
  - MacOS
  - others, in random order

Thus I guess it translates the number of occurrences in field. I suspect
that a number of admins have macs and are used to build it locally to test
their configs, and that it's probably a non-negligible part of the MacOS
reports. But as usual, I could be totally wrong ;-)

Willy




Re: 2.5: Possibility to upgrade http/1.0 clients to http/1.1?

2022-05-08 Thread Willy Tarreau
Hello Dominik,

On Thu, May 05, 2022 at 07:55:06AM +, Froehlich, Dominik wrote:
> Hello everyone,
> 
> We recently bumped our HAproxy deployment to 2.5 and are now getting hit by 
> this fix:
> 
> MEDIUM: mux-h1: Reject HTTP/1.0 GET/HEAD/DELETE requests with a payload
> 
> 
> http://git.haproxy.org/?p=haproxy-2.5.git;a=blob_plain;f=CHANGELOG
> 
> The issue is we have many legacy customers using very old systems and we
> can't tell all of them to rewrite their clients to http/1.1.

Of course... Do you have an idea how they ended up sending a body with
such requests ? Maybe adding a comment on the issue below for posterity
could be useful for future versions of the HTTP spec:

  https://github.com/httpwg/http-core/issues/904

> I get the security fix to prevent request smuggling where some servers ignore
> the body and treat it as another request, I'm not arguing that.
> 
> However, I was wondering if it was possible to intercept HTTP/1.0 client
> requests and upgrade them to HTTP/1.1 without hitting the rejection code of
> the commit here:
> https://github.com/haproxy/haproxy/commit/e136bd12a32970bc90d862d5fe09ea1952b62974

"Upgrading" the version must really never ever be done, as a lot of HTTP
semantics changed between 1.0 and 1.1, and by telling a server that the
client is 1.1, the server will be allowed to use chunking, trailers,
100-continue and a lot of other stuff that 1.0 clients are unable to
understand.

For your use case, I guess the best solution would be to add an option
(possibly a global one) to relax that rule. It's self-contained inside
an "if" statement so it should be quite easy to add such an option and
be done with it, because when you need it, you definitely know that you
need it, you'll find the keyword in the doc and the accompanying warning
about the security impacts. Also the HTTP spec now says "unless support
is indicated via other means", so the intent clearly is to make this
configurable on a case-by-case basis.

> This way we would not have to downgrade to HAproxy 2.4 again - which would be
> very unfortunate as we need many of the nice features of 2.5.

We'll discuss this with Christopher tomorrow, but I'm not worried about
this for now.

Thanks!
Willy



Re: Latest http/3 info

2022-05-08 Thread Willy Tarreau
On Sat, May 07, 2022 at 09:11:30AM -0600, Shawn Heisey wrote:
> If you look closely at the tcpdump output, you'll notice that when haproxy
> replies, it replies from the actual IP address of the machine (.200) rather
> than the ucarp VIP (.170) where it received the request.  Is this something
> that can be fixed?

Oh that's an excellent point! I remember having faced that in the past
on other services (e.g. DNS). That's why most UDP-based services require
that you enforce the exact address on the interface (e.g. 10.2.3.4 instead
of 0.0.0.0).

There's no good solution to this, except by forcing the exact address
yourself. The BSD socket API doesn't permit to send UDP packets from a
specific source, so the commonly used approach for clients is to bind
while sending the first packet, but that doesn't work for a server that
faces many clients, as it would restrict the traffic to the first IP
used.

Note that in order for the two haproxy nodes to bind to the virtual
address you'll likely have to enable ip_nonlocal_bind, but I guess you
already have it.

I think that for the short term we could forbid binding on 0.0.0.0 but
that would annoy most users by complicating config deployments. Thus
maybe the best we can do is document this limitation.

Thanks for reporting this!
Willy



Re: [PATCH] CI: dynamically determine actual h2spec version

2022-05-08 Thread Willy Tarreau
On Thu, May 05, 2022 at 03:17:07PM +0500,  ??? wrote:
> Hi,
> 
> small improvement, no need to use hardcoded version.

Merged, thank you Ilya
Willy



Re: Fwd: Set environment variables

2022-05-08 Thread Willy Tarreau
Hi Valerio,

On Fri, May 06, 2022 at 04:25:23PM +0200, Valerio Pachera wrote:
> Hi, I have several backend configuration that make use of a custom script:
> 
> external-check command 'custom-script.sh'
> 
> The script read uses the environment variables such as $HAPROXY_PROXY_NAME.
> I would like to be able to set and environment variable in the backend
> declaration, before running the external check.
> This environment variable will change the behavior of custom-script.sh.
> 
> Is it possible to declare environment variables in haproxy 1.9 or later?
> 
> What I need is to make custom-script.sh aware if SSL is used or not.
> If there's another way to achieve that, please tell me.

You can only set variables in the global section using "setenv" and
"presetenv", and they are in effect at the moment they're parsed (thus
they're even usable later in the config file). But there's nothing to
set per-backend variables that would be used by the external-check
commands. If you know that your whole haproxy process is either with
or without SSL, that might still work for you. Otherwise maybe you
should use two different scripts for SSL a clear servers.

You could possibly also rely on the server's name and/or port as a hint
to your script.

With that said, I admit that it could be useful to have another variable
such as HAPROXY_SERVER_SSL to indicate if SSL is being used, and possibly
another one such as HAPROXY_SERVER_PROTO to indicate when a protocol is
being forced (e.g. h2).

If you think your only solution would be to have such variables, feel
free to have a look at the code to see how to add them. That should be
simple enough for a first contribution. That's something we can merge
late in the cycle, and possibly even backport to 2.5.

Willy



Re: DOC/MINOR: Typo in INSTALL doc

2022-05-08 Thread Willy Tarreau
On Mon, May 02, 2022 at 11:02:11PM +, Tom?s Zubiri wrote:
> Line 227/581 Col 53/75 char 9913/27467
> 
> Section 4.5 cryptography
> "is known to build ant work with branches"
> 
> Release Branch 2.5.0

Now fixed, thank you Tomas :-)

Willy



Re: [PATCH 1/1: BUILD/MINOR: TCP_KEEPIDLE macos equivalence

2022-05-08 Thread Willy Tarreau
On Sun, May 01, 2022 at 03:33:17PM +0100, David CARLIER wrote:
> Hi here a little patch to set idle time for SO_KEEPALIVE socket option.

Now merged, thanks.

David, one comment though, your commit messages keep missing a lot of
crucial information for reviewers and debuggers, and I had to spend time
documenting myself on keep-alive on MacOS to figure the differences and/or
what the impacts or limitations of the patch could be. I finally found and
put that information into the commit message, but it would be much easier
if you could put it yourself after your development since you actually
have access to the information I had to seek, and know the reasoning
behind your choice.

Just an indication, your patch mentioned this:

   TCP_KEEPALIVE has the same semantic.

After edition (still minimal and possibly inaccurate but the best I
could do):
   
On Linux the interval before starting to send TCP keep-alive packets
is defined by TCP_KEEPIDLE. MacOS has an equivalent with TCP_KEEPIDLE,
which also uses seconds as a unit, so it's possible to simply remap the
definition of TCP_KEEPIDLE to TCP_KEEPALIVE there and get it to seamlessly
work. The other settings (interval and count) are not present, though.

A good rule of thumb to write a useful commit message is to think about
these 3 steps:
  - first, convince a project maintainer that your work is worth being
merged and is very unlikely to cause trouble elsewhere ;

  - second, provide helpful information about why it's done like this and
what it provides, to someone who would see a bisect session finish on
this patch, so that this person doesn't simply think "what's this mess,
let's revert it" without figuring what use case a revert could break ;

  - third, by explaining the design decisions, choices and tradeoffs (and
sometimes even alternate solutions that were ditched), you'll help the
person that discovers breakage find a different solution that will
still preserve the original intent while addressing the problem.

For example recently I broke MacOS build while trying to fix a clang
warning. The two patches that caused the breakage were these ones:

  commit b12966af1006be8d4438ee1ca39c2541a1f2a4f9
  Author: Willy Tarreau 
  Date:   Wed Apr 13 17:09:45 2022 +0200

BUILD: debug: mark the __start_mem_stats/__stop_mem_stats symbols as weak

Building with clang and DEBUG_MEM_STATS shows the following warnings:

  warning: __start_mem_stats changed binding to STB_WEAK [-Wsource-mgr]
  warning: __stop_mem_stats changed binding to STB_WEAK [-Wsource-mgr]

The reason is that the symbols are declared using ".globl" while they
are also referenced as __attribute__((weak)) elsewhere. It turns out
that a weak symbol is implicitly a global one and that the two classes
are exclusive, thus it may confuse the linker. Better fix this.

This may be backported where the patch applies.

  commit 2a06e248f5c8b7c86c7dd48eed7f6d5e87288457
  Author: Willy Tarreau 
  Date:   Wed Apr 13 17:12:20 2022 +0200

BUILD: initcall: mark the __start_i_* symbols as weak, not global

Just like for previous fix, these symbols are marked ".globl" during
their declaration, but their later mention uses __attribute__((weak)),
so it's better to only use ".weak" during the declaration so that the
symbol's class does not change.

No need to backport this unless someone reports build issues.

There's an explanation of the problem, when it's encountered, and the
reasoning behind the proposed solution. The follwing day the CI broke
on MacOS, I could restart from the elements above to try to design
another solution that still follows the same spirit (and in the worst
case it could possibly have been accepted to just revert and keep that
warning when debugging):

  commit fb1b6f5bc0e8eac116e2cafe8716a7f16d95b58e
  Author: Willy Tarreau 
  Date:   Thu Apr 14 16:57:12 2022 +0200

BUILD: compiler: use a more portable set of asm(".weak") statements

The two recent patches b12966af1 ("BUILD: debug: mark the
__start_mem_stats/__stop_mem_stats symbols as weak") and 2a06e248f
("BUILD: initcall: mark the __start_i_* symbols as weak, not global")
aimed at fixing a build warning and resulted in a build breakage on
MacOS which doesn't have a ".weak" asm statement.

We've already had MacOS-specific asm() statements for section names, so
this patch continues on this trend by moving HA_GLOBL() to compiler.h
and using ".globl" on MacOS since apparently nobody complains there.

It is debatable whether to expose this only when !USE_OBSOLETE_LINKER
or all the time, but since these are just macroes it's no big deal to
let them be available when needed and let the caller decide on the
build conditions.

If any of the patches above i

[ANNOUNCE] haproxy-2.6-dev8

2022-04-30 Thread Willy Tarreau
faults section
  BUG/MINOR: rules: Fix check_capture() function to use the right rule 
arguments
  REGTESTS: fix the race conditions in be2dec.vtc ad field.vtc
  BUG/MEDIUM: http-ana: Fix memleak in redirect rules with ignore-empty 
option
  BUG/MEDIUM: conn-stream: Don't erase endpoint flags on reset
  BUG/MEDIUM: httpclient: Fix loop consuming HTX blocks from the response 
channel
  BUG/MINOR: httpclient: Count metadata in size to transfer via 
htx_xfer_blks()
  MINOR: httpclient: Don't use co_set_data() to decrement output

Frédéric Lécaille (25):
  MINOR: quic: Improve qc_prep_pkts() flexibility
  MINOR: quic: Prepare quic_frame struct duplication
  MINOR: quic: Do not retransmit frames from coalesced packets
  MINOR: quic: Add traces about TX frame memory releasing
  MINOR: quic: process_timer() rework
  MEDIUM: quic: New functions for probing rework
  MEDIUM: quic: Retransmission functions rework
  MEDIUM: quic: qc_requeue_nacked_pkt_tx_frms() rework
  MINOR: quic: old data distinction for qc_send_app_pkt()
  MINOR: quic: Mark packets as probing with old data
  MEDIUM: quic: Mark copies of acknowledged frames as acknowledged
  MEDIUM: quic: Enable the new datagram probing process
  MINOR: quic: Do not send ACK frames when probing
  BUG/MINOR: quic: Wrong returned status by qc_build_frms()
  BUG/MINOR: quic: Avoid sending useless PADDING frame
  BUG/MINOR: quic: Traces fix about remaining frames upon packet build 
failure
  MINOR: quic: Wake up the mux to probe with new data
  BUG/MEDIUM: quic: Possible crash on STREAM frame loss
  BUG/MINOR: quic: Missing Initial packet length check
  CLEANUP: quic: Rely on the packet length set by qc_lstnr_pkt_rcv()
  MINOR: quic: Drop 0-RTT packets if not allowed
  MINOR: quic: Drop 0-RTT packets without secrets
  CLEANUP: quic: Remaining fprintf() debug trace
  MINOR: quic: moving code for QUIC loss detection
  BUG/MINOR: quic: Missing time threshold multiplifier for loss delay 
computation

Ilya Shipitsin (1):
  CI: github actions: update LibreSSL to 3.5.2

Remi Tricot-Le Breton (2):
  BUG/MINOR: connection: "connection:close" header added despite 
'close-spread-time'
  MINOR: connection: Add way to disable active connection closing during 
soft-stop

Thomas Prückl (1):
  MINOR: ssl: add a new global option "tune.ssl.hard-maxrecord"

Tim Duesterhus (3):
  CLEANUP: Destroy `http_err_chunks` members during deinit
  BUG/MINOR: resolvers: Fix memory leak in resolvers_deinit()
  MINOR: Call deinit_and_exit(0) for `haproxy -vv`

William Lallemand (8):
  REGTESTS: webstats: remove unused stats socket in /tmp
  MEDIUM: httpclient: disable SSL when the ca-file couldn't be loaded
  BUG/MINOR: httpclient/lua: error when the httpclient_start() fails
  BUG/MINOR: ssl: free the cafile entries on deinit
  BUG/MINOR: ssl: memory leak when trying to load a directory with ca-file
  MEDIUM: httpclient: re-enable the verify by default
  BUG/MEDIUM: ssl/cli: fix yielding in show_cafile_detail
  BUG/MINOR: httpclient/ssl: use the correct verify constant

Willy Tarreau (26):
  BUG/MINOR: http-act: make release_http_redir() more robust
  BUG/MINOR: sample: add missing use_backend/use-server contexts in 
smp_resolve_args
  MINOR: sample: don't needlessly call c_none() in sample_fetch_as_type()
  MINOR: sample: make the bool type cast to bin
  MEDIUM: backend: add new "balance hash " algorithm
  MINOR: init: add global setting "fd-hard-limit" to bound system limits
  BUILD: pollers: use an initcall to register the pollers
  BUILD: xprt: use an initcall to register the transport layers
  BUILD: thread: use initcall instead of a constructor
  BUILD: http: remove the two unused constructors in rules and ana
  CLEANUP: compression: move the default setting of maxzlibmem to defaults
  MINOR: tree-wide: always consider EWOULDBLOCK in addition to EAGAIN
  MINOR: fd: add functions to set O_NONBLOCK and FD_CLOEXEC
  CLEANUP: tree-wide: use fd_set_nonblock() and fd_set_cloexec()
  CLEANUP: tree-wide: remove 25 occurrences of unneeded fcntl.h
  BUILD: compiler: properly distinguish weak and global symbols
  BUILD: fd: disguise the fd_set_nonblock/cloexec result
  BUG/MINOR: pools: make sure to also destroy shared pools in 
pool_destroy_all()
  CLEANUP: errors: also call deinit_errors_buffers() on deinit()
  CLEANUP: chunks: release trash also in deinit
  CLEANUP: deinit: release the pre-check callbacks
  CLEANUP: deinit: release the config postparsers
  CLEANUP: listeners/deinit: release accept queue tasklets on deinit
  CLEANUP: connections/deinit: destroy the idle_conns tasks
  BUG/MINOR: conn_stream: do not confirm a connection from the frontend path
  SCRIPTS: announce-release: add URL of dev packages

---



Re: Set environment variables

2022-04-30 Thread Willy Tarreau
On Tue, Apr 26, 2022 at 03:34:32PM +0200, Aleksandar Lazic wrote:
> On Tue, 26 Apr 2022 15:03:51 +0200
> Valerio Pachera  wrote:
> 
> > Hi, I have several backend configuration that make use of a custom script:
> > 
> > external-check command 'custom-script.sh'
> > 
> > The script read uses the environment variables such as $HAPROXY_PROXY_NAME.
> > I would like to be able to set and environment variable in the backend
> > declaration, before running the external check.
> > This environment variable will change the behavior of custom-script.sh.
> > 
> > Is it possible to declare environment variables in haproxy 1.9 or later?
> > 
> > What I need is to make custom-script.sh aware if SSL is used or not.
> > If there's another way to achieve that, please tell me.
> 
> Well you can put it in the name of the server as I don't see any other option
> to add extra variables into the external check.

It's possible to set variables in the global section using "setenv"
and "presetenv", please refer to the doc for these.

Willy



Re: [PATCH] CI: minor LibreSSL update 3.5.1 --> 3.5.2

2022-04-30 Thread Willy Tarreau
On Thu, Apr 28, 2022 at 11:59:39AM +0500,  ??? wrote:
> Hello,
> 
> small patch to sync with current LibreSSL release

Merged, thank you Ilya!
Willy



  1   2   3   4   5   6   7   8   9   10   >