On 9/24/26 13:45, Steven Rostedt wrote:
Just an FYI. My wife flew into Europe today and I'll be touring Europe
with her until Oct 3rd. I may get time to look at patches, but it will
be very limited. And then LPC starts and who knows when I'll get a
chance.
Thus I may not be very responsive for the next couple of weeks.
Thx for the reply Steve. I understand it is that time of the year with
travels and such.
No rush, just wanted to push it out of mailbox, to yours :-)
Besides it might all be moot as Peter's solution is much more elegant
and efficient (as always).
I will reply to that but don't feel pressed to respond.
Thx,
-Vineet
-- Steve
On Tue, 22 Sep 2026 15:55:24 -0700
Vineet Gupta <[email protected]> wrote:
Turning on function-graph tracing, or attaching a kprobe_multi return
probe, hands every thread a shadow stack. alloc_retstack_tasklist()
does that 32 tasks at a time, and since for_each_process_thread() has
no cursor, every sweep restarts from init_task and re-walks the tasks
already served. Total work is quadratic O(N^2) on thread count.
On a 60-core Sapphire Rapids machine with 400000 idle threads the 0 -> 1
transition takes 227 s, inside a single bpf() syscall for the kprobe_multi
case. On Meta fleet this showed up as RCU stalls and softlockup panics.
Patch 1 raises the batch to 1024, dividing the sweeps by 32:
227 s -> 7.2 s at 400000 threads. It helps on every preemption model.
Patch 2 adds a cond_resched() between sweeps. It is supplementary and
separable: a no-op on current x86 and arm64, but on !CONFIG_PREEMPTION
builds it takes soft lockups from 3-of-3 runs to 0-of-3. Dropping it
leaves patch 1 intact.
Neither changes the O(N^2) shape; a cursor-based walk would, but
task_struct lifetime makes that considerably more involved.
Vineet Gupta (2):
tracing: fgraph: Raise FTRACE_RETSTACK_ALLOC_SIZE to 1024
tracing: fgraph: Add a cond_resched() to the shadow stack retry loop
include/linux/ftrace.h | 7 ++++++-
kernel/trace/fgraph.c | 8 ++++++++
2 files changed, 14 insertions(+), 1 deletion(-)
