Re: [ANNOUNCE] haproxy-2.4-dev19

2021-05-10 Thread Willy Tarreau
Hi Tim,

On Mon, May 10, 2021 at 10:48:38AM +0200, Tim Düsterhus wrote:
> As this is expected to be the last "release candidate" I just threw
> 2.4-dev19 onto one of our HAProxy machines:
> 
> > # 
> >   3932master  0   *snip*
> > *snip*  2.4~dev19-1# workers
> > 15082   worker  1   0
> > 0d00h00m12s 2.4~dev19-1# old workers
> > 10384   worker  [was: 1]1   *snip*  
> > 2.3.10-1~bpo10+1
> > # programs
> 
> Let's see how that goes :-)

Hehe, thanks for joining the effort!

> > So if you still have a comment to formulate, hurry up!
> 
> I dropped the ball on the normalizers a bit due to other work, I'm sorry
> about that. I plan to send two patches for 'fragment-strip' and
> 'fragment-encode' tonight (based off our private exchange). I expect these
> to be safe, as the feature is marked experimental and clearly separated.

OK, thanks for letting me know.

Willy



Re: [ANNOUNCE] haproxy-2.4-dev19

2021-05-10 Thread Tim Düsterhus

Willy,

On 5/10/21 8:50 AM, Willy Tarreau wrote:

I really don't expect any more significant changes being applied before
the release now, so we'll focus on cosmetic updates, doc and build tests
on various platforms. I'll add a few CPU entries in the Makefile to ease
building on modern ARM platforms and recheck the docs aimed at newcomers
(contributing, reporting issues etc).

Unless a huge bug falls in front of us blocking any progress, I think
we'll release this week, ideally on Friday morning so that those in search
of a distraction for their boring Friday afternoon have something to play
with :-)


As this is expected to be the last "release candidate" I just threw 
2.4-dev19 onto one of our HAProxy machines:


#   
3932master  0   *snip*  *snip*  2.4~dev19-1
# workers
15082   worker  1   0   0d00h00m12s 2.4~dev19-1
# old workers

10384   worker  [was: 1]1   *snip*  
2.3.10-1~bpo10+1
# programs


Let's see how that goes :-)


So if you still have a comment to formulate, hurry up!


I dropped the ball on the normalizers a bit due to other work, I'm sorry 
about that. I plan to send two patches for 'fragment-strip' and 
'fragment-encode' tonight (based off our private exchange). I expect 
these to be safe, as the feature is marked experimental and clearly 
separated.


Best regards
Tim Düsterhus



[ANNOUNCE] haproxy-2.4-dev19

2021-05-09 Thread Willy Tarreau
Hi,

HAProxy 2.4-dev19 was released on 2021/05/10. It added 155 new commits
after version 2.4-dev18.

It took longer than I expected because of late discoveries of some
monstrosities with the includes which are responsible for the long
build times. When you see that the task scheduler or arguments
parser implicitly include openssl and resulted in 41k lines having
to be compiled, that's quite annoying. At least now I figured the
root cause. It's because openssl uses typedefs to define types, and
doing so prevents one from using forward declaration of incomplete
types, forcing all struct members that require a pointer to include
the file (these are in struct server and struct listener, so virtually
everywhere). On the forthcoming week I'll still look if a cleaner cut
is possible between the include files, othrewise we'll defer this to
2.5.

With that aside, I think we're done, with everything planed having been
completed by the end of last week.

Among the user-visible changes, here's what I'm having in mind in random
order:
  - CLI: fixed a memory leak on write errors. Also the "help" output was
re-aligned and sorted for better legibility. Larger lines allowed to
report the usage syntax that most commands were missing.

  - experimental commands/directives: using experimental directives in
the configuration will require an opt-in by placing the directive
"expose-experimental-directives" in the global section. The goal is
to make sure that random copy-paste from blogs doesn't result in users
being exposed without being aware of it. For now this affects the
"tune.fd.edge-triggered" global option and "http-request normalize-uri"
action.

  - use of declaration and execution of experimental or expert features
is reported with a new "tainted" flag reported in "show info", that
should primarily help developers know if anything dangerous or less
known was done on the process, but could also possibly help an admin
figure that someone used some tricky commands on the CLI.

  - the spelling inconsistencies between "haproxy/Haproxy/HAproxy/HA-Proxy"
etc were (hopefully) all addressed, including the one that appears in
"haproxy -v" that I already forgot to fix before previous release.

  - the "abortonclose" option was fixed for HTTP/1, as in some cases the
mux wouldn't try to read anymore thus wouldn't detect a client close.

  - an optional memory profiler was added to detect where memory is used.
It's enabled by "set profiling memory on" on the CLI and consulted
using "show profiling memory". It shows the number of allocations and
releases per caller as well as their size. It's interesting to watch
when using external libs like openssl or Lua. It works well on Linux
and glibc, I managed to get it to build and work on FreeBSD as well,
though with a bit less visibility for now. It uses very little
resources so I wanted to have it merged so that in the future we
could ask some issue reporters to enable it.

  - idle frontend connections will now be closed on reload. Till now they
could only be closed when after receiving and processing a new request,
forcing many users to add a hard-stop-after directive. Now for the vast
majority of users this should be cleaner and the old process should
disappear much faster.

  - on reload, idle backend connections were not explicitly closed,
resulting in a "normal" death when the process exited. This used to
cause some TIME_WAIT sockets to be left, consuming source ports for
a long time. In some environments it's a problem. Now there's an
explicit kill of these connections when quitting so as not to leave
such entries lying around.

  - the "ALERT/WARNING/NOTICE/DIAG" output messages in foreground were
cleaned up to drop the unused time code and to align them so that
they read better.

  - the .if/.elif config directives now support a few predicates to isolate
a config block. The strict minimum was done to help with migrations,
test configs for developers, and more portable regtests in the future.
We currently have "version_atleast()" and "version_before()" which test
for the current version being at least or before the mentioned one,
"feature()" to test for a feature to appear in the "Feature list" of
"haproxy -vv", "streq()" and "strneq()" to test for string equality or
difference, and "defined()" to test for the existence of an environment
variable. I think that some of us will stop modifying their test configs
10 times a day to enable/disable SSL or PROMEX for example, and that
we won't replace as many VTCs between 2.4 and 2.5 as we replaced between
2.3 and 2.4.

  - the diagnostic mode (-dD) will not complain anymore about peers having
a zero weight, and will also report annotations left by the admin using
the new ".diag" config directive.

  - "show info" now supports an optional "float" opti