Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] bpf: per-CPU bpf_raw_tp_regs corruption in bpf_get_stack_raw_tp_sleepable() --
commit c66538ad7075b69839a1969f72a7ea7eff53e112 Author: Feng Yang <[email protected]> bpf: Fix suspicious RCU usage for sleepable bpf_get_stack() This patch routes sleepable tracepoint and raw tracepoint/tracing BPF programs calling bpf_get_stack() to bpf_get_stack_sleepable() so that classic RCU read lock is held when accessing callchain entries and the helper is marked might_sleep. > diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c > index 14adfb85fdb2b..7887d1c00be83 100644 > --- a/kernel/trace/bpf_trace.c > +++ b/kernel/trace/bpf_trace.c [ ... ] > @@ -1705,6 +1727,33 @@ static const struct bpf_func_proto > bpf_get_stack_proto_raw_tp = { > .arg4_type = ARG_ANYTHING, > }; > > +BPF_CALL_4(bpf_get_stack_raw_tp_sleepable, struct bpf_raw_tracepoint_args *, > args, > + void *, buf, u32, size, u64, flags) > +{ > + struct pt_regs *regs = get_bpf_raw_tp_regs(); > + int ret; > + > + if (IS_ERR(regs)) > + return PTR_ERR(regs); > + > + perf_fetch_caller_regs(regs); > + ret = bpf_get_stack_sleepable((unsigned long) regs, (unsigned long) buf, > + (unsigned long) size, flags, 0); > + put_bpf_raw_tp_regs(); > + return ret; > +} [Severity: High] Can holding the per-CPU bpf_raw_tp_regs slot across bpf_get_stack_sleepable() corrupt active register state or exhaust the per-CPU slots on the local CPU? In kernel/trace/bpf_trace.c, get_bpf_raw_tp_regs() and put_bpf_raw_tp_regs() manage a 3-slot per-CPU array indexed as a LIFO stack via bpf_raw_tp_nest_level: static struct pt_regs *get_bpf_raw_tp_regs(void) { struct bpf_raw_tp_regs *tp_regs = this_cpu_ptr(&bpf_raw_tp_regs); int nest_level = this_cpu_inc_return(bpf_raw_tp_nest_level); if (nest_level > ARRAY_SIZE(tp_regs->regs)) { this_cpu_dec(bpf_raw_tp_nest_level); return ERR_PTR(-EBUSY); } return &tp_regs->regs[nest_level - 1]; } static void put_bpf_raw_tp_regs(void) { this_cpu_dec(bpf_raw_tp_nest_level); } Sleepable BPF programs using raw_tp_prog_func_proto() or tracing_prog_func_proto() run under rcu_read_lock_trace() and migrate_disable() with preemption enabled. When such a program calls bpf_get_stack_raw_tp_sleepable() with BPF_F_USER_STACK | BPF_F_USER_BUILD_ID, bpf_get_stack_sleepable() -> __bpf_get_stack() only disables preemption around get_perf_callchain() and callchain_store(), and then re-enables preemption before calling callchain_finalize(): /* trace should not be dereferenced after this point */ if (may_fault) rcu_read_unlock(); preempt_enable(); return callchain_finalize(buf, size, trace_nr, elem_size, flags, may_fault); Inside callchain_finalize() -> stack_map_get_build_id_offset_sleepable() -> build_id_parse_file(), the task can sleep on page faults or filesystem reads while still holding the per-CPU bpf_raw_tp_regs slot. If task A on a CPU acquires regs[0] (nest_level becomes 1) and sleeps or is preempted inside callchain_finalize(), task B on the same CPU can call get_bpf_raw_tp_regs(), acquire regs[1] (nest_level becomes 2), populate regs[1] via perf_fetch_caller_regs(), and get preempted before __bpf_get_stack() disables preemption. If task A then wakes up on the same CPU and calls put_bpf_raw_tp_regs(), bpf_raw_tp_nest_level is decremented from 2 to 1 while task B is still about to use regs[1]. Any subsequent caller on that CPU (another task, or an IRQ/NMI raw tracepoint program calling bpf_get_stack_raw_tp(), bpf_get_stackid_raw_tp(), or bpf_perf_event_output_raw_tp()) will increment bpf_raw_tp_nest_level back to 2, receive the same &tp_regs->regs[1] pointer, and overwrite task B's saved registers before task B runs get_perf_callchain(). In addition, if three sleepable tasks on the same CPU block concurrently in build_id_parse_file(), bpf_raw_tp_nest_level stays at 3 for the duration of the I/O, causing all get_bpf_raw_tp_regs() calls on that CPU across task, IRQ, and NMI contexts to fail with -EBUSY. -- Sashiko AI review ยท https://sashiko.dev/#/patchset/[email protected]?part=1
