[Bug 1675949] Re: perf report can't annotate kernel and dbgsym package has wrong addresses
Ubuntu 18.10 (Cosmic Cuttlefish) has reached end of life, so this bug will not be fixed for that specific release. ** Changed in: linux (Ubuntu Cosmic) Status: New => Won't Fix -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675949 Title: perf report can't annotate kernel and dbgsym package has wrong addresses To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1675949/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1675949] Re: perf report can't annotate kernel and dbgsym package has wrong addresses
** Changed in: linux (Ubuntu Disco) Status: Confirmed => Won't Fix -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675949 Title: perf report can't annotate kernel and dbgsym package has wrong addresses To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1675949/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1675949] Re: perf report can't annotate kernel and dbgsym package has wrong addresses
** Tags added: cscc -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675949 Title: perf report can't annotate kernel and dbgsym package has wrong addresses To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1675949/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1675949] Re: perf report can't annotate kernel and dbgsym package has wrong addresses
** Also affects: linux (Ubuntu Disco) Importance: Medium Assignee: Paolo Pisati (p-pisati) Status: Confirmed ** Also affects: linux (Ubuntu Cosmic) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675949 Title: perf report can't annotate kernel and dbgsym package has wrong addresses To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1675949/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1675949] Re: perf report can't annotate kernel and dbgsym package has wrong addresses
** Also affects: linux (Ubuntu Bionic) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675949 Title: perf report can't annotate kernel and dbgsym package has wrong addresses To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1675949/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1675949] Re: perf report can't annotate kernel and dbgsym package has wrong addresses
** Description changed: + Impact: + + Under some condition, 'perf report' is unable to show the assembly code + of a kernel function (perf report -> select a kernel function -> + annotate). + + This is not an issue of the kernel itself or the perf tool, instead it's + your environment that is missing the objdump binary (and perf, + unfortunately, doesn't complain about it). + + Make sure binutils is not installed, or move away the objdump binary: + + # mv /usr/bin/objdump /usr/bin/objdump.foo + + Run a perf session and write down all events: + + # perf record -a -- sleep 10 + + # perf report + + selct a kernel function, press 'Annotate', a blank screen will appears + (instead of the assembly code). + + Now put the objdump binary back (or install binutils): + + # mv /usr/bin/objdump.foo /usr/bin/objdump + + and run again perf report: + + # perf report + + select a kernel function, annotate, the disassembly will show up. + + Fix: + + Make binutils a Depends in SRCPKGNAME-tools-PKGVER-ABINUM - see the + attached patch. + + How to test: + + Install a patched linux-tools-PKGVER-ABINUM.deb package: it will require + binutils. + + Regression: + + We are adding a new dependency on a package, so code wise there's no + regression potential - on the other hand, image creation, or development + environment will experience a slight increase in size (1.8MB on x86-64 + and 2.1M on arm64) if they didn't install binutils already. + + -- + Running zesty 4.10.0-13-generic kernel and running: - $ perf record + $ perf record followed by $ perf report results in output that can't be annotated (even when run as root). The following message appears: Couldn't annotate do_io_submit: No vmlinux file with build id 81ba79d482fa9e3ea58486de8f119b27fe6db55e was found in the path. Note that annotation using /proc/kcore requires CAP_SYS_RAWIO capability. Please use: perf buildid-cache -vu vmlinux Note this is being run as root and /proc/kcore is accessible with dd from the same shell: $ dd if=/proc/kcore bs=8 count=16 | hexdump 000 457f 464c 0102 0001 010 0004 00b7 0001 020 0040 030 0040 0038 0003 040 0004 00e8 Additionally, installing the linux-image-4.10.0-13-generic-dbgsym actually hurts the situation because the symbol addresses in the dbgsym don't match the kernel addresses. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675949 Title: perf report can't annotate kernel and dbgsym package has wrong addresses To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1675949/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1675949] Re: perf report can't annotate kernel and dbgsym package has wrong addresses
On a fresh Xenial/arm64 installation: $ uname -r 4.4.0-141-generic $ uname -m aarch64 $ dpkg -S $(which perf) linux-tools-common: /usr/bin/perf $ dpkg --get-selections | grep linux-tools linux-tools-4.4.0-141 install linux-tools-4.4.0-141-generic install linux-tools-common install linux-tools-generic install $ sudo perf record -a -- sleep 10 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.131 MB perf.data (626 samples) ] followed by: $ sudo perf report i see the captured samples: 42.22% swapper[kernel.kallsyms] [k] arch_cpu_idle ▒ 5.10% tmux [kernel.kallsyms] [k] copy_page ◆ 1.37% tmux libc-2.23.so [.] 0x0006e680 ▒ 1.37% tmux tmux [.] 0x00028684 ▒ 1.37% tmux libc-2.23.so [.] strcmp ▒ 1.35% swapper[kernel.kallsyms] [k] tick_nohz_idle_enter ▒ 1.23% tmux tmux [.] 0x000355dc ▒ 1.05% sh [kernel.kallsyms] [k] handle_mm_fault ▒ 1.01% tmux [kernel.kallsyms] [k] filemap_map_pages ▒ 1.01% swapper[kernel.kallsyms] [k] _raw_spin_unlock_irqrestore ▒ 0.96% tmux libc-2.23.so [.] free but when i try to inspect a kernel function: Annotate filemap_map_pages ◆ Zoom into sh(5304) thread ▒ Zoom into [kernel.kallsyms] DSO ▒Annotate filemap_map_pages ◆ Zoom into sh(5304) thread ▒ Zoom into [kernel.kallsyms] DSO ▒ Browse map details ▒ Zoom into Processor Socket 0 ▒ Run scripts for samples of thread [sh] ▒ Run scripts for samples of symbol [filemap_map_pages] ▒ Run scripts for all samples ▒ Switch to another data file in PWD ▒ Exit Browse map details ▒ Zoom into Processor Socket 0 ▒ Run scripts for samples of thread [sh] ▒ Run scripts for samples of symbol [filemap_map_pages] ▒ Run scripts for all samples ▒ Switc
[Bug 1675949] Re: perf report can't annotate kernel and dbgsym package has wrong addresses
** Changed in: linux (Ubuntu) Assignee: (unassigned) => Paolo Pisati (p-pisati) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675949 Title: perf report can't annotate kernel and dbgsym package has wrong addresses To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1675949/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1675949] Re: perf report can't annotate kernel and dbgsym package has wrong addresses
This issue fell off my radar, and I never followed up on it. I'll see if we can get someone to take a look. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675949 Title: perf report can't annotate kernel and dbgsym package has wrong addresses To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1675949/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1675949] Re: perf report can't annotate kernel and dbgsym package has wrong addresses
Have we fixed this issue for aarch64? We experienced the same thing with 4.10, 4.13 and 4.15 kernels on xenial, artful and bionic uname -a Linux 4.13.0-46-generic #51-Ubuntu SMP Tue Jun 12 12:39:10 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux dpkg --get-selections | grep linux-tools linux-tools-4.13.0-46 install linux-tools-4.13.0-46-generic install linux-tools-common install linux-tools-generic install -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675949 Title: perf report can't annotate kernel and dbgsym package has wrong addresses To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1675949/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1675949] Re: perf report can't annotate kernel and dbgsym package has wrong addresses
[Expired for linux (Ubuntu) because there has been no activity for 60 days.] ** Changed in: linux (Ubuntu) Status: Incomplete => Expired -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675949 Title: perf report can't annotate kernel and dbgsym package has wrong addresses To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1675949/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1675949] Re: perf report can't annotate kernel and dbgsym package has wrong addresses
** Changed in: linux (Ubuntu) Importance: Undecided => Medium ** Tags added: kernel-da-key -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675949 Title: perf report can't annotate kernel and dbgsym package has wrong addresses To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1675949/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1675949] Re: perf report can't annotate kernel and dbgsym package has wrong addresses
Everything continues to work fine for me, but I see now that you are on aarch64. I don't have any 64-bit ARM platforms in my posession, but I should be able to get access to something. I'll see what I can find out. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675949 Title: perf report can't annotate kernel and dbgsym package has wrong addresses To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1675949/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1675949] Re: perf report can't annotate kernel and dbgsym package has wrong addresses
root@host1:~/fio# uname -a Linux host1 4.10.0-14-generic #16-Ubuntu SMP Fri Mar 17 15:20:17 UTC 2017 aarch64 aarch64 aarch64 GNU/Linux root@host1:~/fio# dpkg -S $(which perf) linux-tools-common: /usr/bin/perf root@host1:~/fio# dpkg --get-selections | grep linux-tools linux-tools-4.10.0-11 install linux-tools-4.10.0-13 install linux-tools-4.10.0-13-generic install linux-tools-4.10.0-14 install linux-tools-4.10.0-14-generic install linux-tools-4.10.0-9install linux-tools-4.10.0-9-genericinstall linux-tools-common install linux-tools-generic install Running perf like: perf record -F 99 -g -- sleep 10 also has the same issue I've been running: perf record -a -- sleep 10 Annotation works on kernel modules and user code, but non-module kernel code has the issue where annotation doesn't work on it. After running record, run perf report, select one of the kernel symbols, hit enter, select Annotate and hit enter again. Thanks, Ali -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675949 Title: perf report can't annotate kernel and dbgsym package has wrong addresses To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1675949/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1675949] Re: perf report can't annotate kernel and dbgsym package has wrong addresses
I'm going to need more information. Can you give me the output of these commands? $ uname -a $ dpkg -S $(which perf) $ dpkg --get-selections | grep linux-tools Can you proived the exact commands you're running to produce this behavior? For me, running "perf record" with no additional arguments only generates help text. Is this only with one specific way of invoking "perf record"? Have you tried sampling other events to see if you get the same problem? For example, does this produce a similar error? # perf record -F 99 -g -- sleep 10 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675949 Title: perf report can't annotate kernel and dbgsym package has wrong addresses To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1675949/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1675949] Re: perf report can't annotate kernel and dbgsym package has wrong addresses
Hi Seth, Thanks for taking a look. After upgrading to 4.10.0-14-generic I'm still seeing the same issue. The build-id has changed, but the symptoms are the same: -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675949 Title: perf report can't annotate kernel and dbgsym package has wrong addresses To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1675949/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1675949] Re: perf report can't annotate kernel and dbgsym package has wrong addresses
I tried this with 4.10.0-14-generic and could not reproduce your issue. Please try updating and see if you still experience this issue. Run-time addresses are expected to be unpredictable due to ASLR. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675949 Title: perf report can't annotate kernel and dbgsym package has wrong addresses To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1675949/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs