ping, thanks
On Mon, Jul 28, 2025 at 11:34:56PM +0200, Jiri Olsa wrote:
> On Fri, Jul 25, 2025 at 07:13:18PM +0900, Masami Hiramatsu wrote:
> > On Sun, 20 Jul 2025 13:21:20 +0200
> > Jiri Olsa <jo...@kernel.org> wrote:
> >
> > > Putting together all the previously added pieces to support optimized
> > > uprobes on top of 5-byte nop instruction.
> > >
> > > The current uprobe execution goes through following:
> > >
> > > - installs breakpoint instruction over original instruction
> > > - exception handler hit and calls related uprobe consumers
> > > - and either simulates original instruction or does out of line single
> > > step
> > > execution of it
> > > - returns to user space
> > >
> > > The optimized uprobe path does following:
> > >
> > > - checks the original instruction is 5-byte nop (plus other checks)
> > > - adds (or uses existing) user space trampoline with uprobe syscall
> > > - overwrites original instruction (5-byte nop) with call to user space
> > > trampoline
> > > - the user space trampoline executes uprobe syscall that calls related
> > > uprobe
> > > consumers
> > > - trampoline returns back to next instruction
> > >
> > > This approach won't speed up all uprobes as it's limited to using nop5 as
> > > original instruction, but we plan to use nop5 as USDT probe instruction
> > > (which currently uses single byte nop) and speed up the USDT probes.
> > >
> > > The arch_uprobe_optimize triggers the uprobe optimization and is called
> > > after
> > > first uprobe hit. I originally had it called on uprobe installation but
> > > then
> > > it clashed with elf loader, because the user space trampoline was added
> > > in a
> > > place where loader might need to put elf segments, so I decided to do it
> > > after
> > > first uprobe hit when loading is done.
> > >
> > > The uprobe is un-optimized in arch specific set_orig_insn call.
> > >
> > > The instruction overwrite is x86 arch specific and needs to go through 3
> > > updates:
> > > (on top of nop5 instruction)
> > >
> > > - write int3 into 1st byte
> > > - write last 4 bytes of the call instruction
> > > - update the call instruction opcode
> > >
> > > And cleanup goes though similar reverse stages:
> > >
> > > - overwrite call opcode with breakpoint (int3)
> > > - write last 4 bytes of the nop5 instruction
> > > - write the nop5 first instruction byte
> > >
> > > We do not unmap and release uprobe trampoline when it's no longer needed,
> > > because there's no easy way to make sure none of the threads is still
> > > inside the trampoline. But we do not waste memory, because there's just
> > > single page for all the uprobe trampoline mappings.
> > >
> > > We do waste frame on page mapping for every 4GB by keeping the uprobe
> > > trampoline page mapped, but that seems ok.
> > >
> > > We take the benefit from the fact that set_swbp and set_orig_insn are
> > > called under mmap_write_lock(mm), so we can use the current instruction
> > > as the state the uprobe is in - nop5/breakpoint/call trampoline -
> > > and decide the needed action (optimize/un-optimize) based on that.
> > >
> > > Attaching the speed up from benchs/run_bench_uprobes.sh script:
> > >
> > > current:
> > > usermode-count : 152.604 ± 0.044M/s
> > > syscall-count : 13.359 ± 0.042M/s
> > > --> uprobe-nop : 3.229 ± 0.002M/s
> > > uprobe-push : 3.086 ± 0.004M/s
> > > uprobe-ret : 1.114 ± 0.004M/s
> > > uprobe-nop5 : 1.121 ± 0.005M/s
> > > uretprobe-nop : 2.145 ± 0.002M/s
> > > uretprobe-push : 2.070 ± 0.001M/s
> > > uretprobe-ret : 0.931 ± 0.001M/s
> > > uretprobe-nop5 : 0.957 ± 0.001M/s
> > >
> > > after the change:
> > > usermode-count : 152.448 ± 0.244M/s
> > > syscall-count : 14.321 ± 0.059M/s
> > > uprobe-nop : 3.148 ± 0.007M/s
> > > uprobe-push : 2.976 ± 0.004M/s
> > > uprobe-ret : 1.068 ± 0.003M/s
> > > --> uprobe-nop5 : 7.038 ± 0.007M/s
> > > uretprobe-nop : 2.109 ± 0.004M/s
> > > uretprobe-push : 2.035 ± 0.001M/s
> > > uretprobe-ret : 0.908 ± 0.001M/s
> > > uretprobe-nop5 : 3.377 ± 0.009M/s
> > >
> > > I see bit more speed up on Intel (above) compared to AMD. The big nop5
> > > speed up is partly due to emulating nop5 and partly due to optimization.
> > >
> > > The key speed up we do this for is the USDT switch from nop to nop5:
> > > uprobe-nop : 3.148 ± 0.007M/s
> > > uprobe-nop5 : 7.038 ± 0.007M/s
> > >
> >
> > This also looks good to me.
> >
> > Acked-by: Masami Hiramatsu (Google) <mhira...@kernel.org>
>
> thanks!
>
> Peter, do you have more comments?
>
> thanks,
> jirka
