Re: [ANNOUNCE] haproxy-2.8-dev7

2023-04-11 Thread William Lallemand
On Sat, Apr 08, 2023 at 06:30:24PM +0200, Willy Tarreau wrote:
>   - William managed to improve acme.sh to directly manage haproxy at
> runtime via the CLI, and to start a config without certificates.
> This means that once his pull request will be merged, Let's Encrypt
> support will be totally seamless. In case it wouldn't be merged (there
> are many PRs pending there), we'd find a different way to host this
> change, either by integrating it as a "plugin" with haproxy, or by
> forking the repository and providing it under the haproxy github 
> organization. More info in the PR here:
>   https://github.com/acmesh-official/acme.sh/pull/4581
> 

Regarding this, I also updated the documentation of the stateless mode
so people can easily configure haproxy for it, no more proxying is
required:

https://github.com/acmesh-official/acme.sh/wiki/Stateless-Mode#haproxy

-- 
William Lallemand



[ANNOUNCE] haproxy-2.8-dev7

2023-04-08 Thread Willy Tarreau
Hi,

HAProxy 2.8-dev7 was released on 2023/04/08. It added 147 new commits
after version 2.8-dev6.

Freezing features has good virtues. This version attracted a number of
pending commits that will need a bit of testing before the release. Among
the changes:
  - Christopher could finally merge a significant part of his rework of
the status flags that moved from the channel to the stream connector.
This was an important change that we've been hoping to do since 2.5
or so, that will greatly help avoid some of the obscure bugs that are
occasionally reported after a minor change that causes never-ending
streams or ignored aborts. It may possibly bring a bug or two caused
by mistakes during the conversion, and it may possibly wake up some
such older bugs, though the risk is quite low and the patches were
carefully reviewed. But testing and reports are welcome. There's a
second part to this that we're still wondering whether a part can
still be merged or not, we'll see soon.

  - Olivier implemented support for compressing requests. While it can
sound surprising at first, it's very useful when the front LB nodes
are selected by DNS and need to route large requests to a remote
server that is reached over a limited link or when outgoing traffic
is expensive.

  - Aurélien merged his event handler API. This is publish/subscribe
mechanism used to broadcast notifications about state changes to
internal subsystems. For now, only server state change is supported,
and such events may be subscribed to from Lua. This means that some
Lua code can be notified about added/removed/up/down servers. A mid-
term goal will be to reimplement the mailers entirely in Lua, allowing
unlimited customizing.

  - Aurélien addressed a CPU usage issue when stopping; pool_gc() was
called too often, resulting in stick-tables possibly consuming a lot
of CPU during such periods, which was even worse when malloc_trim()
was called to free memory as this call involves locking inside the
libc, hence severe contention.

  - Thierry's changes to make Lua support dynamic servers were finally
merged. In the past, Lua could take a long time to start when dealing
with very large configs as it was learning the servers, and wouldn't
be aware of servers added on the fly. This is no more a limitation.

  - Fred and Amaury improved the QUIC stack in various areas: less memory
consumption by removing one tree lookup, fixed some RTT calculations
and corner cases in congestion control algorithms, added more info to
"show quic", added a dummy congestion control algorithm with a fixed
window that may help us diagnose some issues in field.

  - William managed to improve acme.sh to directly manage haproxy at
runtime via the CLI, and to start a config without certificates.
This means that once his pull request will be merged, Let's Encrypt
support will be totally seamless. In case it wouldn't be merged (there
are many PRs pending there), we'd find a different way to host this
change, either by integrating it as a "plugin" with haproxy, or by
forking the repository and providing it under the haproxy github 
organization. More info in the PR here:
  https://github.com/acmesh-official/acme.sh/pull/4581

  - Martin, our intern for one week, implemented the support for case
insensitive argument match in urlp/url_param (issue 1633). It was a
one year-old feature request that we hoped for 2.6. Not bad at all
for a first contribution ;-)

  - the rest is essentially bug fixes, doc and cleanups.

Regarding the pending stuff I'm aware of at the moment:
  - Christopher still has changes aiming at making applets much more
reliable by no more stepping over the stream's feet. It's really
desired but possibly risky at the same time. We'll check this
together next week, and possibly take some of it, or maybe insert
just a smaller step towards it in order to ease backports of post-2.9
fixes to 2.8 later.

  - Amaury managed to make the QUIC connections behave exactly like
the TCP connections at the lower layers, in that they are migrable
to other threads, giving back the control to listener_accept() like
for TCP. It's only a matter of writing commit messages now.

  - My changes to make bind lines support multiple thread groups are
now working, but I was blocked on the QUIC part, which is no more
a problem after Amaury's change above. I've rebased my code on top
of his and will merge it once it's in.

  - Aurélien managed to add a working timeout for Lua instructions
that are not yieldable (i.e. busy loops called from sample fetch
functions). The patches were reviewed yesterday afternoon and
will be merged early next week after some cosmetic updates.

And with all this done we'll have to focus more on testing, debugging,
documentation, cleanups and small