Hi, On Wed, Feb 16, 2022 at 10:26:08PM +0100, Willy Tarreau wrote: > Maybe we'll figure reasonable ways to turn some options to dynamic > in the future (thinking about what's done with pools, I'm pretty sure > that would be possible for almost half of the options, this would > solve the problem).
I managed to work on this to turn almost all pool-related debug options to boot-time options. For this I extended the "-dM" command-line argument to also support enabling/disabling/listing debug settings. The code was arranged so that disabled options have no impact and the most common options have almost no measurable impact. There's a "help" option which lists the current settings (which are still preset based on DEBUG_* so that it remains possible to set the desired default settings at build time): $ ./haproxy -dM,help -dM alone enables memory poisonning with byte 0x50 on allocation. A numeric value may be appended immediately after -dM to use another value (0 supported). Then an optional list of comma-delimited keywords may be appended to set or clear some debugging options ('*' marks the current setting): set clear description -----------------+-----------------+----------------------------------------- fail |* no-fail | randomly fail allocations * no-merge | merge | disable merging of similar pools cold-first |* hot-first | pick cold objects first integrity |* no-integrity | enable cache integrity checks * no-global | global | disable global shared cache no-cache |* cache | disable thread-local cache caller |* no-caller | save caller information in cache * tag | no-tag | add tag at end of allocated objects * poison | no-poison | poison newly allocated objects As it was easier to make measurements here, I could verify that, as expected, poisonning, integrity checks and disabling the cache have an important impact (halves the max req rate), and that the "tag" which catches most misuses of pool_free() is almost non-measurable beyond eating 8 extra bytes per object. I still need to mark the buffers non-checkable (as there's no benefit in doing that and it's heavy). Thus I updated the makefile to enable this by default (by setting DEBUG_MEMORY_POOL), and DEBUG_STRICT (which enables BUG_ON()). I have plans to improve the pool debugging, which may result in less options but easier ones (i.e. instead of knowing how it's implemetend inside, we would simply configure based on the use case). I don't know if this will be done for 2.6. Regarding DEBUG_STRICT I would really like to improve this to stuff many more of them in more sensitive areas but they would not be enabled in default builds, only in CI and developers'. Now asking reporters for more info will just be a matter of asking to restart with "-dMno-merge,cold-first,integrity,caller,tag" instead of rebuilding, and that will provide the highest level of detail we currently support. Unexpectedly, the main difficulty in this patch set was to re-arrange the init code so that it was possible to move the command line parsing earlier, and that we know pools configuration before they are created. Thus, while I initially considered that it would be a childs game that could be trivially backported, now I know it is not as trivial anymore. I do think it remains trivially backportable to 2.5 and might eventually be done, but at least I want to leave that under observation for some time before engaging into this. Thanks again for the discussion, it was useful ;-) Willy