The rcutree.do_rcu_barrier hook currently waits for ordinary RCU callbacks, but objects may still be retained in kfree_rcu() batching or a partial per-CPU SLUB sheaf. This is consistent with the hook's documented rcu_barrier() operation, but incomplete for its intended use as a boundary between userspace tests.
The immediate trigger was a false allocation-leak failure in the bcachefs ktest suite while testing performance changes. Its end check writes the hook before reading /proc/allocinfo, assuming a complete deferred-free drain. Small objects remained visible after repeated hook writes and 20 seconds of waiting, so otherwise clean tests failed their leak check. Changing the hook to drain kvfree_rcu() work let the same unmodified bcachefs workload pass its allocation check. All eight checkpoints in one VM, after 50 through 400 option changes, reported zero retained reconcile_scan objects. The retained population on the original kernel eventually fell as a sheaf filled; there is no evidence here of unbounded growth or OOM. Calling kvfree_rcu_barrier() from rcu_barrier_throttled() was proposed and agreed during review of the former API in 2024, specifically to restore a clean baseline between userspace benchmark runs: https://lore.kernel.org/all/[email protected]/ This patch implements that follow-up and documents the expanded hook. It also removes the old ordinary-barrier completion shortcut: an unrelated rcu_barrier() does not establish that kvfree_rcu() work was drained. Four counterbalanced fresh-VM pairs with the full private-cache fixture reported 60 to 60 active objects on the unpatched kernel and 60 to 59 on the patched kernel. A separate ordinary-callback regression test passed on both kernels. The simplified reproducer below removes that separate regression machinery. One additional fresh control/treatment pair with this exact 41-line source confirmed the same 60 to 60 versus 60 to 59 split. These counts reflect the slab layout in the tested configuration. Save the source as rcu_barrier_sheaf_repro.c and create a Makefile containing: obj-m := rcu_barrier_sheaf_repro.o Build it with: make -C /lib/modules/$(uname -r)/build M="$PWD" modules Then, as root on a disposable test kernel: insmod rcu_barrier_sheaf_repro.ko awk '$1 == "rcu_barrier_sheaf_repro" { print $2 }' /proc/slabinfo cat /sys/kernel/slab/rcu_barrier_sheaf_repro/sheaf_capacity echo 1 > /sys/module/rcutree/parameters/do_rcu_barrier awk '$1 == "rcu_barrier_sheaf_repro" { print $2 }' /proc/slabinfo rmmod rcu_barrier_sheaf_repro The first and second slabinfo readings are 60 and 60 without the patch, and 60 and 59 with it. kmem_cache_destroy() performs per-cache deferred-free cleanup when the module is removed, after the measurement. // SPDX-License-Identifier: GPL-2.0 #include <linux/init.h> #include <linux/module.h> #include <linux/rcupdate.h> #include <linux/slab.h> struct repro_object { struct rcu_head rcu; unsigned long payload; }; static struct kmem_cache *repro_cache; static int __init rcu_barrier_sheaf_repro_init(void) { struct repro_object *object; repro_cache = kmem_cache_create("rcu_barrier_sheaf_repro", sizeof(*object), 0, SLAB_NO_MERGE, NULL); if (!repro_cache) return -ENOMEM; object = kmem_cache_alloc(repro_cache, GFP_KERNEL); if (!object) { kmem_cache_destroy(repro_cache); return -ENOMEM; } kfree_rcu(object, rcu); return 0; } static void __exit rcu_barrier_sheaf_repro_exit(void) { kmem_cache_destroy(repro_cache); } module_init(rcu_barrier_sheaf_repro_init); module_exit(rcu_barrier_sheaf_repro_exit); MODULE_LICENSE("GPL"); MODULE_DESCRIPTION("Reproduce incomplete rcutree.do_rcu_barrier drains"); --- Changes since v1: - Add the motivating bcachefs failure and the successful unmodified workload result to both the cover letter and commit message. - Drop the incorrect sheaf Fixes: tag and regression framing; describe this as a strengthening of the existing test interface. - Credit the agreed 2024 proposal for this extension. - Broaden the subject and changelog from sheaves to kvfree_rcu work. - Hard-wrap the prose for text-based mail readers. The code diff is unchanged from v1. The results above are the existing validation results; no new kernel tests were run for this prose revision. v1: https://lore.kernel.org/all/[email protected]/ Matthias Goergens (1): rcu: make userspace barrier hook drain kvfree_rcu work .../admin-guide/kernel-parameters.txt | 7 ++--- kernel/rcu/tree.c | 27 ++++++++++++------- 2 files changed, 21 insertions(+), 13 deletions(-) base-commit: 50d05c7c76c96b90462f24debacca971d2e86713 -- 2.55.0
