[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
With the last update (and the patches already in eoan) this ticket can be closed as Fix Released. ** Changed in: linux (Ubuntu) Status: Fix Committed => Fix Released ** Changed in: ubuntu-z-systems Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
The fixes for Bionic and Disco have been released, as stated on the stable update bugs mentioned on comment #50. Therefore I'm marking this bug report manually as 'Fix Released'. Thank you. ** Changed in: linux (Ubuntu Disco) Status: Fix Committed => Fix Released ** Changed in: linux (Ubuntu Bionic) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
** Description changed: SRU Justification: == [Impact] * The perf top tool hangs and shows error messages, like 'Not enough memory for annotating' [Fix] * b9c0a64901d5bdec6eafd38d1dc8fa0e2974fccb b9c0a64 "perf annotate: Fix s390 gap between kernel end and module start" * 12a6d2940b5f02b4b9f71ce098e3bb02bc24a9ea 12a6d29 "perf record: Fix module size on s390" Disco needs also as prereq: - * 6738028dd57df064b969d8392c943ef3b3ae705d 6738028 perf record: Fix s390 - missing module symbol and warning for non-root users - + * 6738028dd57df064b969d8392c943ef3b3ae705d 6738028 "perf record: Fix + s390 missing module symbol and warning for non-root users" [Test Case] * start a benchmark (mem_alloc, but it doesn't really matter what) * execute perf top in a second terminal * the output of perf top is correct * now stop the benchmark * and perf top shows an error message, like "Not enough memory for annotating '__irf_end' symbol!)" * and perf top can't be exited anymore [Regression Potential] * The regression potential can be considered as medium since this happens only while using the perf top tool * and just 3 files are changed, and one of them is arch/s390/util/machine.c * but symbol and machine header in /tools/perf/util modified and several loc added [Other Info] * cherry-pick was possible to bionic-master-next and to disco-master- next (but used '--strategy=recursive -X theirs' for disco) * adding the patches to disco is to avoid regressions _ perf top hangs and shows error messages ---uname output--- Linux weather 4.15.0-46-generic #49-Ubuntu SMP Wed Feb 6 09:32:27 UTC 2019 s390x s390x s390x GNU/Linux ---Steps to Reproduce--- I start a benchmark (mem_alloc, but it really doesn't matter) and then issue perf top in a second terminal, the output from perf top is correct. Now I stop the benchmark: perf top shows a error message (Not enough memory for annotating '__irf_end' symbol!) and I can't quit from perf top anymore Following analyse took place: No problem with current kernel . Bi-Secting of perf tool took place and following commit was found: commit edeb0c90df3581b821a764052d185df985f8b8dc (HEAD, refs/bisect/bad) Author: Arnaldo Carvalho de Melo Date: Tue Oct 16 17:08:29 2018 -0300 perf tools: Stop fallbacking to kallsyms for vdso symbols lookup When you apply this patch the issue is gone, however it is contained in these versions: git tag --contains edeb0c90df3581b821 v4.19 v4.20 The level I was debugging was kernel 4.15 which does not contain this patch. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
** Changed in: ubuntu-z-systems Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Note: The fixes for this problem were applied under a different bug number via stable updates: For Bionic: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1840520 For Disco: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1841994 ** Changed in: linux (Ubuntu Bionic) Status: In Progress => Fix Committed ** Changed in: linux (Ubuntu Disco) Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
SRU request: https://lists.ubuntu.com/archives/kernel-team/2019-August/103323.html -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
These patches have already been committed to Eoan (LP: #1841110). ** Changed in: linux (Ubuntu) Status: Confirmed => Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
** Description changed: SRU Justification: == [Impact] * The perf top tool hangs and shows error messages, like 'Not enough memory for annotating' [Fix] * b9c0a64901d5bdec6eafd38d1dc8fa0e2974fccb b9c0a64 "perf annotate: Fix s390 gap between kernel end and module start" * 12a6d2940b5f02b4b9f71ce098e3bb02bc24a9ea 12a6d29 "perf record: Fix module size on s390" + + Disco needs also as prereq: + + * 6738028dd57df064b969d8392c943ef3b3ae705d 6738028 perf record: Fix s390 + missing module symbol and warning for non-root users + [Test Case] * start a benchmark (mem_alloc, but it doesn't really matter what) * execute perf top in a second terminal * the output of perf top is correct * now stop the benchmark * and perf top shows an error message, like "Not enough memory for annotating '__irf_end' symbol!)" * and perf top can't be exited anymore [Regression Potential] * The regression potential can be considered as medium since this happens only while using the perf top tool * and just 3 files are changed, and one of them is arch/s390/util/machine.c * but symbol and machine header in /tools/perf/util modified and several loc added [Other Info] * cherry-pick was possible to bionic-master-next and to disco-master- next (but used '--strategy=recursive -X theirs' for disco) * adding the patches to disco is to avoid regressions _ perf top hangs and shows error messages ---uname output--- Linux weather 4.15.0-46-generic #49-Ubuntu SMP Wed Feb 6 09:32:27 UTC 2019 s390x s390x s390x GNU/Linux ---Steps to Reproduce--- I start a benchmark (mem_alloc, but it really doesn't matter) and then issue perf top in a second terminal, the output from perf top is correct. Now I stop the benchmark: perf top shows a error message (Not enough memory for annotating '__irf_end' symbol!) and I can't quit from perf top anymore Following analyse took place: No problem with current kernel . Bi-Secting of perf tool took place and following commit was found: commit edeb0c90df3581b821a764052d185df985f8b8dc (HEAD, refs/bisect/bad) Author: Arnaldo Carvalho de Melo Date: Tue Oct 16 17:08:29 2018 -0300 perf tools: Stop fallbacking to kallsyms for vdso symbols lookup When you apply this patch the issue is gone, however it is contained in these versions: git tag --contains edeb0c90df3581b821 v4.19 v4.20 The level I was debugging was kernel 4.15 which does not contain this patch. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Hello IBM, Thank you for the feedback. We will submit the patches for integration into the next SRU cycle. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Hello IBM, I have built test kernels for Bionic and Disco with the patches mentioned: b9c0a64 perf annotate: Fix s390 gap between kernel end and module start 12a6d29 perf record: Fix module size on s390 For Disco, I have also backported the following, which seemed to be a pre-req: 3376b6b7fb57 perf record: Fix s390 missing module symbol and warning for non-root users The debian packages can be found at: Bionic (4.15.0-59.66+lp1828166.20190814): https://people.canonical.com/~ksouza/lp1828166/20190814/bionic/ Disco (5.0.0-26.27+lp1828166.20190814): https://people.canonical.com/~ksouza/lp1828166/20190814/disco/ I have tested these kernels locally and I was not able to reproduce the issue anymore. Could you please test them to confirm the issue is fixed? Thank you. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
** Changed in: ubuntu-z-systems Status: Triaged => In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
** Also affects: linux (Ubuntu Disco) Importance: Undecided Status: New ** Changed in: linux (Ubuntu Disco) Status: New => In Progress ** Changed in: linux (Ubuntu Bionic) Importance: Undecided => High ** Changed in: linux (Ubuntu Disco) Importance: Undecided => High -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
I tried if the two commits can be cleanly applied by cherry-picking them from bionic-master-next (and disco-master-next), and it looks good (with a little hick-up on disco). I updated the SRU justification in the bug description and new test packages will be built soon (thx klebers). ** Description changed: SRU Justification: == [Impact] * The perf top tool hangs and shows error messages, like 'Not enough memory for annotating' [Fix] - * edeb0c90df3581b821a764052d185df985f8b8dc edeb0c9 "perf tools: Stop - fallbacking to kallsyms for vdso symbols lookup" + * b9c0a64901d5bdec6eafd38d1dc8fa0e2974fccb b9c0a64 "perf annotate: Fix + s390 gap between kernel end and module start" + + * 12a6d2940b5f02b4b9f71ce098e3bb02bc24a9ea 12a6d29 "perf record: Fix + module size on s390" [Test Case] * start a benchmark (mem_alloc, but it doesn't really matter what) * execute perf top in a second terminal * the output of perf top is correct * now stop the benchmark * and perf top shows an error message, like "Not enough memory for annotating '__irf_end' symbol!)" * and perf top can't be exited anymore [Regression Potential] - * The regression potential can be considered as low since this happens - only while using the perf top tool + * The regression potential can be considered as medium since this + happens only while using the perf top tool - * and it is known that the commit (above) fixes the problem + * and just 3 files are changed, and one of them is + arch/s390/util/machine.c - * and the fix is upstream since 4.19 + * but symbol and machine header in /tools/perf/util modified and several + loc added [Other Info] - * current disco and eoan kernels don't show that problem + * cherry-pick was possible to bionic-master-next and to disco-master- + next (but used '--strategy=recursive -X theirs' for disco) - * bisecting result points to above commit - - * applies cleanly on cosmic, but has a little conflict on bionic (both master-next) + * adding the patches to disco is to avoid regressions _ perf top hangs and shows error messages ---uname output--- Linux weather 4.15.0-46-generic #49-Ubuntu SMP Wed Feb 6 09:32:27 UTC 2019 s390x s390x s390x GNU/Linux ---Steps to Reproduce--- I start a benchmark (mem_alloc, but it really doesn't matter) and then issue perf top in a second terminal, the output from perf top is correct. Now I stop the benchmark: perf top shows a error message (Not enough memory for annotating '__irf_end' symbol!) and I can't quit from perf top anymore Following analyse took place: No problem with current kernel . Bi-Secting of perf tool took place and following commit was found: commit edeb0c90df3581b821a764052d185df985f8b8dc (HEAD, refs/bisect/bad) Author: Arnaldo Carvalho de Melo Date: Tue Oct 16 17:08:29 2018 -0300 perf tools: Stop fallbacking to kallsyms for vdso symbols lookup When you apply this patch the issue is gone, however it is contained in these versions: git tag --contains edeb0c90df3581b821 v4.19 v4.20 The level I was debugging was kernel 4.15 which does not contain this patch. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
** Changed in: ubuntu-z-systems Status: Incomplete => Triaged -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Hi Thomas, thanks for looking into this and doing this analysis. Sounds reasonable to bring it upstream first and do a backport afterwards. Since 18.10/cosmic with kernel 4.18 is EOL in between, we just need to get it into bionic 4.15 and disco 5.0 - but I assume you've meant that anyway ... ** Changed in: ubuntu-z-systems Status: Confirmed => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Hi Thomas, The tarball with the sources I used to build linux 4.15.0-55.60+lp1828166.2 can be found here: https://people.canonical.com/~ksouza/lp1828166/bionic/linux- source-4.15.0.tar.bz2 md5sum: 58e725a4c20dcf3fb0b209e3d6155e74 linux-source-4.15.0.tar.bz2 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
'apt-get source' should work - just double checked it here: # just these two deb-src lines should be sufficient: $ grep ^deb-src /etc/apt/sources.list deb-src http://us.ports.ubuntu.com/ubuntu-ports/ bionic main deb-src http://us.ports.ubuntu.com/ubuntu-ports/ bionic-updates main $ apt-get source linux-image-$(uname -r) Reading package lists... Done Picking 'linux' as source package instead of 'linux-image-4.15.0-54-generic' NOTICE: 'linux' packaging is maintained in the 'Git' version control system at: git://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/bionic Please use: git clone git://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/bionic to retrieve the latest (possibly unreleased) updates to the package. Skipping already downloaded file 'linux_4.15.0-54.58.dsc' Skipping already downloaded file 'linux_4.15.0.orig.tar.gz' Skipping already downloaded file 'linux_4.15.0-54.58.diff.gz' Need to get 0 B of source archives. Skipping unpack of already unpacked source in linux-4.15.0 $ Did you do the required "apt update" after modifying the sources.list file - to update the local archive package index? But 'klebers' will share the sources he used ... -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
And afterwards you can get the package that contains perf like this: $ dpkg -S $(which perf) linux-tools-common: /usr/bin/perf $ apt-get source linux-tools-common For the linux-tools source package that fits to your running kernel, do: apt-get source linux-tools-$(uname -r) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Sorry, I missed comment #31, just saw now #32. Please have a look here on how to install the build env. and the kernel sources: https://wiki.ubuntu.com/Kernel/BuildYourOwnKernel Just notice that the deb line in the sources.list file is different for non-x86 (like s390x) It must be: deb-src http://us.ports.ubuntu.com/ubuntu-ports/ bionic main deb-src http://us.ports.ubuntu.com/ubuntu-ports/ bionic-updates main instead of: deb-src http://archive.ubuntu.com/ubuntu bionic main deb-src http://archive.ubuntu.com/ubuntu bionic-updates main -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
I can also confirm that the issue still exists, but only if I run perf top as root or with sudo. Running perf top it as regular user is fine - even w/o the patches packages! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Hi Klaus, The same version of perf is expected. I was able to reproduce the issue and I can confirm that I see the same behavior as you described even with the patched test kernel. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Hello IBM, I have built a kernel for tests with the patches mentioned in the previous comment: perf record: Fix s390 missing module symbol and warning for non-root users perf machine: Read also the end of the kernel perf machine: Update kernel map address and re-order properly perf machine: Set main kernel end address properly perf machine: Don't search for active kernel start in __machine__create_kernel_maps perf machine: Generalize machine__set_kernel_mmap() The packages for s390x can be found at: https://people.canonical.com/~ksouza/lp1828166/bionic/ Could you please test the kernel from the link above if it fixes the problem? Thank you. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
** Changed in: linux (Ubuntu Cosmic) Status: In Progress => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Hello Frank, On 6/19/19 7:13 AM, Frank Heimes wrote: > Just tried to cherry-pick 1) 6738028 and 2) ed9adb2 from 5.3-rc3 to bionic's > master-next tree: > git://kernel.ubuntu.com/ubuntu/ubuntu-bionic --branch master-next > 1) applied cleanly, but 2) did not - I got 3 conflicts while cherry-picking > ed9adb2 > Is there something still missing? > (I used bionic to check, because it's the oldest release this should SRUed > to, and master-next is needed, because things progress and some patches might > already got accepted on top of master, even if not yet released.) > Sorry I am late getting back to you. Yes, Kleber set out to do what you tried and found the same. he was hoping to finish this before his vacation but was unable to. I asked Khaled to try and finish that trello card and of course he ran into problems and ran out of time due to impact on our SRU from the SACK Panic release and having to get new kernels ready for July 1 SRU release in the two weeks we had left. Khaled is hoping to get back to this card tomorrow, Friday, June 21. Will keep you updated but hope this is resolved and fix-committed for SRU 2019.07.01 Terry -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Just tried to cherry-pick 1) 6738028 and 2) ed9adb2 from 5.3-rc3 to bionic's master-next tree: git://kernel.ubuntu.com/ubuntu/ubuntu-bionic --branch master-next 1) applied cleanly, but 2) did not - I got 3 conflicts while cherry-picking ed9adb2 Is there something still missing? (I used bionic to check, because it's the oldest release this should SRUed to, and master-next is needed, because things progress and some patches might already got accepted on top of master, even if not yet released.) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
** Changed in: linux (Ubuntu) Status: Fix Released => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Thx for the confirmation ** Changed in: ubuntu-z-systems Status: Triaged => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Thx for the analysis and the sharing of the commit IDs for the two patches from kernel 5.2-rc3 in comment #19. Since you mentioned three patches in comment #18 I assume the 3rd one became obsolete? With that I assume that: "perf record: Fix s390 missing module symbol and warning for non-root users" 6738028dd57df064b969d8392c943ef3b3ae705d and "perf machine: Read also the end of the kernel" ed9adb2035b5be5896d465b19040262be5f4a824 need to be applied to Disco, (Cosmic) and Bionic. They will automatically land in Eoan, since kernel 5.2 is Eoan's target kernel. Is it correct that: edeb0c90df3581b821a764052d185df985f8b8dc "perf tools: Stop fallbacking to kallsyms for vdso symbols lookup" does not need to be considered anymore (neither for cosmic nor for bionic)? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
** Changed in: ubuntu-z-systems Status: Incomplete => Triaged -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Changed to Incomplete while waiting for new analysis results. ** Changed in: ubuntu-z-systems Status: In Progress => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Hi, Can you please provide the output of 'uname -a'? Also, on the original bug report it states that the issue is fixed with the mentioned patch applied. On top of which kernel was this patch applied and tested? Thank you. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Klaus, Please download the new packages from the following URL: https://people.canonical.com/~ksouza/lp1828166/bionic/lp/ Additionally to the linux-image and linux-modules packages, please install also: linux-tools-common_4.15.0-49.53+lp1828166_all.deb linux-tools-4.15.0-49-generic_4.15.0-49.53+lp1828166_s390x.deb linux-tools-4.15.0-49_4.15.0-49.53+lp1828166_s390x.deb -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Hello Klaus, The packages I provided indeed don't ship the perf binaries with it, my apologies. I'm building the packages again in another build environment which will produce the perf files we need, I will post a link to the new packages soon. Thank you. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
@ Klaus: Did you tried the linux-tools-common from klebers link ? https://people.canonical.com/~ksouza/lp1828166/bionic/ especially: https://people.canonical.com/~ksouza/lp1828166/bionic/linux-tools-common_4.15.0-49.53+lp1828166_all.deb -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Looks like a dedicated linux-tool-common package is needed, because perf too tight to the kernel ... -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Hi, perf should be in package linux-tools-common and each kernel versions has a dedicated linux-tool-common) package version. $ dpkg -S $(which perf) linux-tools-common: /usr/bin/perf But on disk it's usually in /usr/bin. Please can you check if it's in with: $ dpkg -L linux-tools-common | grep perf$ /usr/bin/perf In case it's not can you just take it temporarily from the last recent bionic package? wget http://launchpadlibrarian.net/417515745/linux-tools-common_4.15.0-48.51_all.deb dpkg-deb -x ./linux-tools-common_4.15.0-48.51_all.deb I think that would be okay for this pre-test, since the perf tool wasn't touched, but just the kernel. The verification during the SRU process later of course needs to be with the proper package. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
** Changed in: ubuntu-z-systems Status: Triaged => In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
** Changed in: linux (Ubuntu Bionic) Status: New => In Progress ** Changed in: linux (Ubuntu Cosmic) Status: New => In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Hello IBM, I have built test kernels with the requested patch applied which can be found at: Bionic: https://people.canonical.com/~ksouza/lp1828166/bionic/ Cosmic: https://people.canonical.com/~ksouza/lp1828166/cosmic/ Please note that at least linux-image- and linux-modules- need to be installed to get a functional kernel. Please let us know if these test kernels fix the issue reported. Thank you. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
** Also affects: linux (Ubuntu Cosmic) Importance: Undecided Status: New ** Also affects: linux (Ubuntu Bionic) Importance: Undecided Status: New ** Changed in: linux (Ubuntu) Status: New => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
** Description changed: + SRU Justification: + == + + [Impact] + + * The perf top tool hangs and shows error messages, like 'Not enough + memory for annotating' + + [Fix] + + * edeb0c90df3581b821a764052d185df985f8b8dc edeb0c9 "perf tools: Stop + fallbacking to kallsyms for vdso symbols lookup" + + [Test Case] + + * start a benchmark (mem_alloc, but it doesn't really matter what) + + * execute perf top in a second terminal + + * the output of perf top is correct + + * now stop the benchmark + + * and perf top shows an error message, like "Not enough memory for + annotating '__irf_end' symbol!)" + + * and perf top can't be exited anymore + + [Regression Potential] + + * The regression potential can be considered as low since this happens + only while using the perf top tool + + * and it is known that the commit (above) fixes the problem + + * and the fix is upstream since 4.19 + + [Other Info] + + * current disco and eoan kernels don't show that problem + + * bisecting result points to above commit + + * applies cleanly on cosmic, but has a little conflict on bionic (both master-next) + _ + perf top hangs and shows error messages - + ---uname output--- Linux weather 4.15.0-46-generic #49-Ubuntu SMP Wed Feb 6 09:32:27 UTC 2019 s390x s390x s390x GNU/Linux - ---Steps to Reproduce--- - I start a benchmark (mem_alloc, but it really doesn't matter) and then issue perf top in a second terminal, the output from perf top is correct. Now I stop the benchmark: perf top shows a error message (Not enough memory for annotating '__irf_end' symbol!) and I can't quit from perf top anymore - + I start a benchmark (mem_alloc, but it really doesn't matter) and then issue perf top in a second terminal, the output from perf top is correct. Now I stop the benchmark: perf top shows a error message (Not enough memory for annotating '__irf_end' symbol!) and I can't quit from perf top anymore Following analyse took place: No problem with current kernel . Bi-Secting of perf tool took place and following commit was found: commit edeb0c90df3581b821a764052d185df985f8b8dc (HEAD, refs/bisect/bad) Author: Arnaldo Carvalho de Melo Date: Tue Oct 16 17:08:29 2018 -0300 - perf tools: Stop fallbacking to kallsyms for vdso symbols lookup + perf tools: Stop fallbacking to kallsyms for vdso symbols lookup When you apply this patch the issue is gone, however it is contained in these versions: git tag --contains edeb0c90df3581b821 v4.19 v4.20 The level I was debugging was kernel 4.15 which does not contain this patch. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1828166] Re: perf top problem on z with Ubuntu 18.04
Hence, this needs to land in bionic (and cosmic) - disco (and eoan) kernels are already > 4.20 ** Also affects: ubuntu-z-systems Importance: Undecided Status: New ** Changed in: ubuntu-z-systems Status: New => Triaged ** Changed in: ubuntu-z-systems Importance: Undecided => High ** Changed in: ubuntu-z-systems Assignee: (unassigned) => Canonical Kernel Team (canonical-kernel-team) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1828166 Title: perf top problem on z with Ubuntu 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1828166/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
