On Fri, Apr 04, 2025 at 01:36:13PM -0700, Andrii Nakryiko wrote: > On Thu, Mar 20, 2025 at 4:42 AM Jiri Olsa <jo...@kernel.org> wrote: > > > > hi, > > this patchset adds support to optimize usdt probes on top of 5-byte > > nop instruction. > > > > The generic approach (optimize all uprobes) is hard due to emulating > > possible multiple original instructions and its related issues. The > > usdt case, which stores 5-byte nop seems much easier, so starting > > with that. > > > > The basic idea is to replace breakpoint exception with syscall which > > is faster on x86_64. For more details please see changelog of patch 8. > > > > The run_bench_uprobes.sh benchmark triggers uprobe (on top of different > > original instructions) in a loop and counts how many of those happened > > per second (the unit below is million loops). > > > > There's big speed up if you consider current usdt implementation > > (uprobe-nop) compared to proposed usdt (uprobe-nop5): > > > > current: > > usermode-count : 152.604 ± 0.044M/s > > syscall-count : 13.359 ± 0.042M/s > > --> uprobe-nop : 3.229 ± 0.002M/s > > uprobe-push : 3.086 ± 0.004M/s > > uprobe-ret : 1.114 ± 0.004M/s > > uprobe-nop5 : 1.121 ± 0.005M/s > > uretprobe-nop : 2.145 ± 0.002M/s > > uretprobe-push : 2.070 ± 0.001M/s > > uretprobe-ret : 0.931 ± 0.001M/s > > uretprobe-nop5 : 0.957 ± 0.001M/s > > > > after the change: > > usermode-count : 152.448 ± 0.244M/s > > syscall-count : 14.321 ± 0.059M/s > > uprobe-nop : 3.148 ± 0.007M/s > > uprobe-push : 2.976 ± 0.004M/s > > uprobe-ret : 1.068 ± 0.003M/s > > --> uprobe-nop5 : 7.038 ± 0.007M/s > > uretprobe-nop : 2.109 ± 0.004M/s > > uretprobe-push : 2.035 ± 0.001M/s > > uretprobe-ret : 0.908 ± 0.001M/s > > uretprobe-nop5 : 3.377 ± 0.009M/s > > > > I see bit more speed up on Intel (above) compared to AMD. The big nop5 > > speed up is partly due to emulating nop5 and partly due to optimization. > > > > The key speed up we do this for is the USDT switch from nop to nop5: > > uprobe-nop : 3.148 ± 0.007M/s > > uprobe-nop5 : 7.038 ± 0.007M/s > > > > > > rfc v3 changes: > > - I tried to have just single syscall for both entry and return uprobe, > > but it turned out to be slower than having two separated syscalls, > > probably due to extra save/restore processing we have to do for > > argument reg, I see differences like: > > > > 2 syscalls: uprobe-nop5 : 7.038 ± 0.007M/s > > 1 syscall: uprobe-nop5 : 6.943 ± 0.003M/s > > > > - use instructions (nop5/int3/call) to determine the state of the > > uprobe update in the process > > - removed endbr instruction from uprobe trampoline > > - seccomp changes > > > > pending todo (or follow ups): > > - shadow stack fails for uprobe session setup, will fix it in next version > > - use PROCMAP_QUERY in tests > > - alloc 'struct uprobes_state' for mm_struct only when needed [Andrii] > > All the pending TODO stuff seems pretty minor. So is there anything > else holding your patch set from graduating out of RFC status? > > David's uprobe_write_opcode() patch set landed, so you should be ready > to rebase and post a proper v1 now, right? > > Performance wins are huge, looking forward to this making it into the > kernel soon!
I just saw notification that those changes are on the way to mm tree, I have the rebase ready, want to post it this week, could be v1 ;-) jirka > > > > > thanks, > > jirka > > > > > > Cc: Eyal Birger <eyal.bir...@gmail.com> > > Cc: k...@kernel.org > > --- > > Jiri Olsa (23): > > uprobes: Rename arch_uretprobe_trampoline function > > uprobes: Make copy_from_page global > > uprobes: Move ref_ctr_offset update out of uprobe_write_opcode > > uprobes: Add uprobe_write function > > uprobes: Add nbytes argument to uprobe_write_opcode > > uprobes: Add orig argument to uprobe_write and uprobe_write_opcode > > uprobes: Remove breakpoint in unapply_uprobe under mmap_write_lock > > uprobes/x86: Add uprobe syscall to speed up uprobe > > uprobes/x86: Add mapping for optimized uprobe trampolines > > uprobes/x86: Add support to emulate nop5 instruction > > uprobes/x86: Add support to optimize uprobes > > selftests/bpf: Use 5-byte nop for x86 usdt probes > > selftests/bpf: Reorg the uprobe_syscall test function > > selftests/bpf: Rename uprobe_syscall_executed prog to > > test_uretprobe_multi > > selftests/bpf: Add uprobe/usdt syscall tests > > selftests/bpf: Add hit/attach/detach race optimized uprobe test > > selftests/bpf: Add uprobe syscall sigill signal test > > selftests/bpf: Add optimized usdt variant for basic usdt test > > selftests/bpf: Add uprobe_regs_equal test > > selftests/bpf: Change test_uretprobe_regs_change for uprobe and > > uretprobe > > selftests/bpf: Add 5-byte nop uprobe trigger bench > > seccomp: passthrough uprobe systemcall without filtering > > selftests/seccomp: validate uprobe syscall passes through seccomp > > > > arch/arm/probes/uprobes/core.c | 2 +- > > arch/x86/entry/syscalls/syscall_64.tbl | 1 + > > arch/x86/include/asm/uprobes.h | 7 ++ > > arch/x86/kernel/uprobes.c | 540 > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- > > include/linux/syscalls.h | 2 + > > include/linux/uprobes.h | 19 +++- > > kernel/events/uprobes.c | 141 > > +++++++++++++++++------- > > kernel/fork.c | 1 + > > kernel/seccomp.c | 32 ++++-- > > kernel/sys_ni.c | 1 + > > tools/testing/selftests/bpf/bench.c | 12 +++ > > tools/testing/selftests/bpf/benchs/bench_trigger.c | 42 ++++++++ > > tools/testing/selftests/bpf/benchs/run_bench_uprobes.sh | 2 +- > > tools/testing/selftests/bpf/prog_tests/uprobe_syscall.c | 453 > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------ > > tools/testing/selftests/bpf/prog_tests/usdt.c | 38 ++++--- > > tools/testing/selftests/bpf/progs/uprobe_syscall.c | 4 +- > > tools/testing/selftests/bpf/progs/uprobe_syscall_executed.c | 41 ++++++- > > tools/testing/selftests/bpf/sdt.h | 9 +- > > tools/testing/selftests/bpf/test_kmods/bpf_testmod.c | 11 +- > > tools/testing/selftests/seccomp/seccomp_bpf.c | 107 > > ++++++++++++++---- > > 20 files changed, 1338 insertions(+), 127 deletions(-)
