Hi,
On Wed, Feb 16, 2022 at 10:26:08PM +0100, Willy Tarreau wrote:
> Maybe we'll figure reasonable ways to turn some options to dynamic
> in the future (thinking about what's done with pools, I'm pretty sure
> that would be possible for almost half of the options, this would
> solve the problem).
I managed to work on this to turn almost all pool-related debug options
to boot-time options.
For this I extended the "-dM" command-line argument to also support
enabling/disabling/listing debug settings. The code was arranged so
that disabled options have no impact and the most common options have
almost no measurable impact. There's a "help" option which lists the
current settings (which are still preset based on DEBUG_* so that it
remains possible to set the desired default settings at build time):
$ ./haproxy -dM,help
-dM alone enables memory poisonning with byte 0x50 on allocation. A numeric
value may be appended immediately after -dM to use another value (0
supported).
Then an optional list of comma-delimited keywords may be appended to set or
clear some debugging options ('*' marks the current setting):
set clear description
-----------------+-----------------+-----------------------------------------
fail |* no-fail | randomly fail allocations
* no-merge | merge | disable merging of similar pools
cold-first |* hot-first | pick cold objects first
integrity |* no-integrity | enable cache integrity checks
* no-global | global | disable global shared cache
no-cache |* cache | disable thread-local cache
caller |* no-caller | save caller information in cache
* tag | no-tag | add tag at end of allocated objects
* poison | no-poison | poison newly allocated objects
As it was easier to make measurements here, I could verify that, as
expected, poisonning, integrity checks and disabling the cache have an
important impact (halves the max req rate), and that the "tag" which
catches most misuses of pool_free() is almost non-measurable beyond
eating 8 extra bytes per object. I still need to mark the buffers
non-checkable (as there's no benefit in doing that and it's heavy).
Thus I updated the makefile to enable this by default (by setting
DEBUG_MEMORY_POOL), and DEBUG_STRICT (which enables BUG_ON()). I have
plans to improve the pool debugging, which may result in less options
but easier ones (i.e. instead of knowing how it's implemetend inside,
we would simply configure based on the use case). I don't know if this
will be done for 2.6. Regarding DEBUG_STRICT I would really like to
improve this to stuff many more of them in more sensitive areas but
they would not be enabled in default builds, only in CI and developers'.
Now asking reporters for more info will just be a matter of asking to
restart with "-dMno-merge,cold-first,integrity,caller,tag" instead of
rebuilding, and that will provide the highest level of detail we
currently support.
Unexpectedly, the main difficulty in this patch set was to re-arrange
the init code so that it was possible to move the command line parsing
earlier, and that we know pools configuration before they are created.
Thus, while I initially considered that it would be a childs game that
could be trivially backported, now I know it is not as trivial anymore.
I do think it remains trivially backportable to 2.5 and might eventually
be done, but at least I want to leave that under observation for some
time before engaging into this.
Thanks again for the discussion, it was useful ;-)
Willy
