Re: [PATCHv6 0/4] perf: Add mmap2 build id support
On Tue, Jan 12, 2021 at 1:19 AM Jiri Olsa wrote: > > On Mon, Jan 11, 2021 at 06:49:58PM -0800, Alexei Starovoitov wrote: > > On Mon, Jan 11, 2021 at 10:38:19PM +0100, Jiri Olsa wrote: > > > hi, > > > adding the support to have buildid stored in mmap2 event, > > > so we can bypass the final perf record hunt on build ids. > > > > > > This patchset allows perf to record build ID in mmap2 event, > > > and adds perf tooling to store/download binaries to .debug > > > cache based on these build IDs. > > > > > > Note that the build id retrieval code is stolen from bpf > > > code, where it's been used (together with file offsets) > > > to replace IPs in user space stack traces. It's now added > > > under lib directory. > > > > > > v6 changes: > > > - last 4 patches rebased Arnaldo's perf/core > > > > There were no issues with v5 as far as I can remember. > > This is just a resubmit to get it landed ? > > yes, exactly > > > Last time we couldn't quite figure out which tree to go through. > > I think the recommend path was to go via bpf-next. > > Is it still the case? > > bpf-next would be best for kernel changes, > perf: Add build id data in mmap2 event > bpf: Add size arg to build_id_parse function > bpf: Move stack_map_get_build_id into lib Then please cc them to bpf@vger and add [PATCH bpf-next] otherwise it's all very confusing. > the 'perf buildid-cache' change needs to go through Arnaldo's tree, > because it depends on changes he already pulled in Also don't include the 4th patch in the series if it isn't meant for bpf-next.
Re: [PATCHv6 0/4] perf: Add mmap2 build id support
On Mon, Jan 11, 2021 at 06:49:58PM -0800, Alexei Starovoitov wrote: > On Mon, Jan 11, 2021 at 10:38:19PM +0100, Jiri Olsa wrote: > > hi, > > adding the support to have buildid stored in mmap2 event, > > so we can bypass the final perf record hunt on build ids. > > > > This patchset allows perf to record build ID in mmap2 event, > > and adds perf tooling to store/download binaries to .debug > > cache based on these build IDs. > > > > Note that the build id retrieval code is stolen from bpf > > code, where it's been used (together with file offsets) > > to replace IPs in user space stack traces. It's now added > > under lib directory. > > > > v6 changes: > > - last 4 patches rebased Arnaldo's perf/core > > There were no issues with v5 as far as I can remember. > This is just a resubmit to get it landed ? yes, exactly > Last time we couldn't quite figure out which tree to go through. > I think the recommend path was to go via bpf-next. > Is it still the case? bpf-next would be best for kernel changes, perf: Add build id data in mmap2 event bpf: Add size arg to build_id_parse function bpf: Move stack_map_get_build_id into lib the 'perf buildid-cache' change needs to go through Arnaldo's tree, because it depends on changes he already pulled in thanks, jirka
Re: [PATCHv6 0/4] perf: Add mmap2 build id support
On Mon, Jan 11, 2021 at 10:38:19PM +0100, Jiri Olsa wrote: > hi, > adding the support to have buildid stored in mmap2 event, > so we can bypass the final perf record hunt on build ids. > > This patchset allows perf to record build ID in mmap2 event, > and adds perf tooling to store/download binaries to .debug > cache based on these build IDs. > > Note that the build id retrieval code is stolen from bpf > code, where it's been used (together with file offsets) > to replace IPs in user space stack traces. It's now added > under lib directory. > > v6 changes: > - last 4 patches rebased Arnaldo's perf/core There were no issues with v5 as far as I can remember. This is just a resubmit to get it landed ? Last time we couldn't quite figure out which tree to go through. I think the recommend path was to go via bpf-next. Is it still the case?
[PATCHv6 0/4] perf: Add mmap2 build id support
hi, adding the support to have buildid stored in mmap2 event, so we can bypass the final perf record hunt on build ids. This patchset allows perf to record build ID in mmap2 event, and adds perf tooling to store/download binaries to .debug cache based on these build IDs. Note that the build id retrieval code is stolen from bpf code, where it's been used (together with file offsets) to replace IPs in user space stack traces. It's now added under lib directory. v6 changes: - last 4 patches rebased Arnaldo's perf/core v5 changes: - rebased on latest perf/core - several patches already pulled in - fixed trace+probe_vfs_getname.sh output redirection - fixed changelogs [Arnaldo] - renamed BUILD_ID_SIZE to BUILD_ID_SIZE_MAX [Song] v4 changes: - fixed typo in changelog [Namhyung] - removed force_download bool from struct dso_store_data, because it's not used [Namhyung] v3 changes: - added acks - removed forgotten debug code [Arnaldo] - fixed readlink termination [Ian] - fixed doc for --debuginfod=URLs [Ian] - adopted kernel's memchr_inv function and used it in build_id__is_defined function [Arnaldo] On recording server: - on the recording server we can run record with --buildid-mmap option to store build ids in mmap2 events: # perf record --buildid-mmap ^C[ perf record: Woken up 2 times to write data ] [ perf record: Captured and wrote 0.836 MB perf.data ] - it stores nothing to ~/.debug cache: # find ~/.debug find: ‘/root/.debug’: No such file or directory - and still reports properly: # perf report --stdio ... 99.82% swapper [kernel.kallsyms] [k] native_safe_halt 0.03% swapper [kernel.kallsyms] [k] finish_task_switch 0.02% swapper [kernel.kallsyms] [k] __softirqentry_text_start 0.01% kcompactd0 [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore 0.01% ksoftirqd/6 [kernel.kallsyms] [k] slab_free_freelist_hook 0.01% kworker/17:1H-x [kernel.kallsyms] [k] slab_free_freelist_hook - display used/hit build ids: # perf buildid-list | head -5 5dcec522abf136fcfd3128f47e131f2365834dd7 /proc/kcore 589e403a34f55486bcac848a45e00bcdeedd1ca8 /usr/lib64/libcrypto.so.1.1.1g 94569566d4eac7e9c87ba029d43d4e2158f9527e /usr/lib64/libpthread-2.30.so 559b9702bebe31c6d132c8dc5cc887673d65d5b5 /usr/lib64/libc-2.30.so 40da7abe89f631f60538a17686a7d65c6a02ed31 /usr/lib64/ld-2.30.so - store build id binaries into build id cache: # perf buildid-cache -a perf.data OK 5dcec522abf136fcfd3128f47e131f2365834dd7 /proc/kcore OK 589e403a34f55486bcac848a45e00bcdeedd1ca8 /usr/lib64/libcrypto.so.1.1.1g OK 94569566d4eac7e9c87ba029d43d4e2158f9527e /usr/lib64/libpthread-2.30.so OK 559b9702bebe31c6d132c8dc5cc887673d65d5b5 /usr/lib64/libc-2.30.so OK 40da7abe89f631f60538a17686a7d65c6a02ed31 /usr/lib64/ld-2.30.so OK a674f7a47c78e35a088104647b9640710277b489 /usr/sbin/sshd OK e5cb4ca25f46485bdbc691c3a92e7e111dac3ef2 /usr/bin/bash OK 9bc8589108223c944b452f0819298a0c3cba6215 /usr/bin/find # find ~/.debug | head -5 /root/.debug /root/.debug/proc /root/.debug/proc/kcore /root/.debug/proc/kcore/5dcec522abf136fcfd3128f47e131f2365834dd7 /root/.debug/proc/kcore/5dcec522abf136fcfd3128f47e131f2365834dd7/kallsyms - run debuginfod daemon to provide binaries to another server (below) (the initialization could take some time) # debuginfod -F / On another server: - copy perf.data from 'record' server and run: $ find ~/.debug/ find: ‘/home/jolsa/.debug/’: No such file or directory $ perf buildid-list | head -5 No kallsyms or vmlinux with build-id 5dcec522abf136fcfd3128f47e131f2365834dd7 was found 5dcec522abf136fcfd3128f47e131f2365834dd7 [kernel.kallsyms] 5784f813b727a50cfd3363234aef9fcbab685cc4 /lib/modules/5.10.0-rc2speed+/kernel/fs/xfs/xfs.ko 589e403a34f55486bcac848a45e00bcdeedd1ca8 /usr/lib64/libcrypto.so.1.1.1g 94569566d4eac7e9c87ba029d43d4e2158f9527e /usr/lib64/libpthread-2.30.so 559b9702bebe31c6d132c8dc5cc887673d65d5b5 /usr/lib64/libc-2.30.so - report does not show anything (kernel build id does not match): $ perf report --stdio ... 76.73% swapper [kernel.kallsyms][k] 0x81aa8ebe 1.89% find [kernel.kallsyms][k] 0x810f2167 0.93% sshd [kernel.kallsyms][k] 0x8153380c 0.83% swapper [kernel.kallsyms][k] 0x81104b0b 0.71% kworker/u40:2-e [kernel.kallsyms][k] 0x810f3850 0.70% kworker/u40:0-e [kernel.kallsyms][k] 0x810f3850 0.64% find [kernel.kallsyms][k] 0x81a9ba0a 0.63% find [kernel.kallsyms][k] 0x81aa93b0 - add build ids does not work, because existing binaries (on another server) have different build ids: $ perf buildid-cache
