[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
** Tags added: cscc -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
** Changed in: linux (Ubuntu) Assignee: Joseph Salisbury (jsalisbury) => (unassigned) ** Changed in: linux (Ubuntu Bionic) Assignee: Joseph Salisbury (jsalisbury) => (unassigned) ** Changed in: linux (Ubuntu) Status: In Progress => Confirmed ** Changed in: linux (Ubuntu Bionic) Status: In Progress => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
I think our best bet is to perform the bisect. When you have a chance, can you test the kernel posted in comment #34? I'll build the next test kernel based on that test result. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
Tested the 4.17 kernel, and this is the issue I now see: [...] [ 17.340040] raid6: sse2x1 xor() 2828 MB/s [ 17.408036] raid6: sse2x2 gen() 7445 MB/s [ 17.472037] raid6: sse2x2 xor() 5425 MB/s [ 17.540032] raid6: sse2x4 gen() 10259 MB/s [ 17.604037] raid6: sse2x4 xor() 6286 MB/s [ 17.668034] raid6: avx2x1 gen() 14089 MB/s [ 17.732034] raid6: avx2x1 xor() 9346 MB/s [ 17.796035] raid6: avx2x2 gen() 16758 MB/s [ 17.860033] raid6: avx2x2 xor() 9779 MB/s [ 17.924034] raid6: avx2x4 gen() 19079 MB/s [ 17.988036] raid6: avx2x4 xor() 11383 MB/s [ 18.007313] raid6: using algorithm avx2x4 gen() 19079 MB/s [ 18.031519] raid6: xor() 11383 MB/s, rmw enabled [ 18.054148] raid6: using avx2x2 recovery algorithm [ 18.076927] xor: automatically using best checksumming function avx [ 18.109754] async_tx: api initialized (async) done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... [ 18.199553] Btrfs loaded, crc32c=crc32c-intel Scanning for Btrfs filesystems done. Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... mdadm: No devices listed in conf file were found. done. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: error opening /dev/md?*: No such file or directory mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. mdadm: No devices listed in conf file were found. done. Gave up waiting for root file system device. Common problems: - Boot args (cat /proc/cmdline) - Check rootdelay= (did the system wait long enough?) - Missing modules (cat /proc/modules; ls /dev) ALERT! UUID=bd0dbca3-60b8-474e-ac82-96f1b9ff9cc2 does not exist. Dropping to a shell! BusyBox v1.27.2 (Ubuntu 1:1.27.2-2ubuntu3) built-in shell (ash) Enter 'help' for a list of built-in commands. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
Thanks for testing. The panic was not caused from anything you did. That is a new bug in the 4.18 kernel. We will have to monitor this new bug, since Cosmic will eventually be 4.18 based. We should also test the latest Cosmic kernel, which is 4.17 based. It is available from: https://launchpad.net/~canonical-kernel- team/+archive/ubuntu/bootstrap/+build/15227897 ** Changed in: linux (Ubuntu) Status: Incomplete => In Progress ** Changed in: linux (Ubuntu Bionic) Status: Incomplete => In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
Maybe I did something wrong, but this kernel (4.18) causes a kernel panic. The process I followed was same as always, to wget the *.deb, dpkg -i *.deb && update-grub at the end to ensure this kernel is used: [6.538003] evm: security.SMACK64 [6.539108] evm: security.SMACK64EXEC [6.540387] evm: security.SMACK64TRANSMUTE [6.541798] evm: security.SMACK64MMAP [6.543023] evm: security.apparmor [6.544209] evm: security.ima [6.545253] evm: security6.668405] usb 2-1: New USB device found, idVendor=8087, idProduct=8002, bcdDevice= 0.05 [6.885734] evm: HMAC attrs: 0x1 [6.886472] Magic number: 14:225:302 [7.048969] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [7.050067] tty tty31: hash matches [7.054948] memory memory160: hash matches [7.056460] rtc_cmos 00:00: setting system clock to 2018-08-14 23:16:59 UTC (1534288619) [7.056552] hub 2-1:1.0: USB hub found [7.060431] hub 2-1:1.0: 8 ports detected [7.063051] md: Waiting for all devices to be available before autodetect [7.093858] md: If you don't use raid, use raid=noautodetect [7.118684] md: Autodetecting RAID arrays. [7.137860] md: autorun ... [7.150409] md: ... autorun DONE. [7.165346] VFS: Cannot open root device "sda1" or unknown-block(0,0): error -6 [7.197757] Please append a correct "root=" boot option; here are the available partitions: [7.235391] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) [7.273379] CPU: 6 PID: 1 Comm: swapper/0 Not tainted 4.18.0-041800-generic #201808122131 [7.310122] Hardware name: HP ProLiant DL360 Gen9/ProLiant DL360 Gen9, BIOS P89 01/22/2018 [7.544633] Call Trace: [7.62] dump_stack+0x63/0x83 [7.570510] panic+0xe7/0x247 [7.583414] mount_block_root+0x231/0x2d8 [7.601354] ? do_early_param+0x95/0x95 [7.618598] mount_root+0x7c/0x7f [7.634231] prepare_namespace+0x139/0x171 [7.653611] kernel_init_freeable+0x236/0x261 [7.673689] ? rest_init+0xb0/0xb0 [7.64] kernel_init+0xe/0x104 [7.704165] ret_from_fork+0x35/0x40 [7.720293] Kernel Offset: 0x29e0 from 0x8100 (relocation range: 0x8000-0xbfff) [7.772913] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]--- [7.815580] [ cut here ] [7.836276] sched: Unexpected reschedule of offline CPU#0! [7.860432] WARNING: CPU: 6 PID: 1 at arch/x86/kernel/smp.c:128 native_smp_send_reschedule+0x3a/0x40 [7.902510] Modules linked in: [7.916373] CPU: 6 PID: 1 Comm: swapper/0 Not tainted 4.18.0-041800-generic #201808122131 [7.953258] Hardware name: HP ProLiant DL360 Gen9/ProLiant DL360 Gen9, BIOS P89 01/22/2018 [8.089242] RIP: 0010:native_smp_send_reschedule+0x3a/0x40 [8.113868] Code: cf 62 01 73 17 48 8b 05 94 8c 17 01 be fd 00 00 00 48 8b 40 30 e8 06 a1 ba 00 5d c3 89 fe 48 c7 c7 58 c1 ed ab e8 48 41 03 00 <0f> 0b 5d c3 66 90 0f 1f 44 00 00 55 48 89 e5 53 48 83 ec 20 65 48 [8.198752] RSP: :8cc61f383e08 EFLAGS: 00010086 [8.24] RAX: RBX: 0003 RCX: ac262608 [8.254674] RDX: 0001 RSI: 0096 RDI: 0046 [8.287087] RBP: 8cc61f383e08 R08: 000cc2b4 R09: 0358 [8.319325] R10: 000f R11: ac7837ed R12: [8.351381] R13: 0003 R14: 8cc61b474500 R15: aaf24bc0 [8.383431] FS: () GS:8cc61f38() knlGS: [8.419863] CS: 0010 DS: ES: CR0: 80050033 [8.446342] CR2: CR3: 0001b020a001 CR4: 003606e0 [8.479621] DR0: DR1: DR2: [8.516202] DR3: DR6: fffe0ff0 DR7: 0400 [8.550084] Call Trace: [8.560999] [8.570157] kick_ilb+0x85/0x90 [8.584223] trigger_load_balance+0xa3/0x190 [8.603397] ? tick_sched_do_timer+0x60/0x60 [8.623122] scheduler_tick+0xae/0xd0 [8.639704] update_process_times+0x42/0x60 [8.658527] tick_sched_handle+0x29/0x60 [8.676734] tick_sched_timer+0x3c/0x80 [8.694426] __hrtimer_run_queues+0x107/0x270 [8.714088] hrtimer_interrupt+0x101/0x220 [8.732405] smp_apic_timer_interrupt+0x67/0x130 [8.753195] apic_timer_interrupt+0xf/0x20 [8.771190] [8.780539] RIP: 0010:panic+0x201/0x247 [8.798007] Code: eb a6 83 3d 27 ee 8e 01 00 74 05 e8 10 6b 02 00 48 c7 c6 20 c1 77 ac 48 c7 c7 78 59 ee ab e8 a0 59 06 00 fb 66 0f 1f 44 00 00 <31> db e8 2f a7 0d 00 4c 39 eb 7c 1d 41 83 f4 01 48 8b 05 cf ed 8e [8.882850] RSP: :a274c3167d78 EFLAGS: 0246 ORIG_RAX: ff13 [8.917249] RAX: 005c RBX: 8cc619067000 RCX: ac262608 [8.949417] RDX: RSI:
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
The problem with having the kernels pre-built is the bisect needs to know whether a test kernel has the bug or not. Based on that input, the bisect decides the next SHA1 to test. Since it's been some time, maybe it would be best to first test the latest mainline kernel again. That will tell us if the issue has already been fixed upstream. The latest mainline kernel is: http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.18/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
Hi Joseph, Sorry for such a late reply, but with all the recent business travel I've been unable to continue working on this. That said, now that things are calmer I can continue, however, I don't have easy access the hardware in question and it is very time consuming getting access to this while blocking all the other work that happens with this hardware. Is there a way to have all those kernels available so I can test things in one go? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
** Changed in: linux (Ubuntu Bionic) Status: In Progress => Incomplete ** Changed in: linux (Ubuntu) Status: In Progress => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
I built the next test kernel, up to the following commit: abb62c46d4949d44979fa647740feff3f7538799 The test kernel can be downloaded from: http://kernel.ubuntu.com/~jsalisbury/lp1761796 Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results. Thanks in advance -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
@Joseph: Issue still exists: ubuntu@superb-whale:~$ uname -a Linux superb-whale 4.15.0-041500rc6-generic #201805171959 SMP Thu May 17 20:02:44 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux ubuntu@superb-whale:~$ sudo reboot === Stopping LVM2 metadata daemon... [14/5352] [ OK ] Stopped LVM2 metadata daemon. [ 107.570384] {1}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 1 [ 107.570386] {1}[Hardware Error]: event severity: fatal [ 107.570388] {1}[Hardware Error]: Error 0, type: fatal [ 107.570389] {1}[Hardware Error]: section_type: PCIe error [ 107.570391] {1}[Hardware Error]: port_type: 4, root port -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
** Changed in: linux (Ubuntu) Status: Confirmed => In Progress ** Changed in: linux (Ubuntu Bionic) Status: Confirmed => In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
I started a kernel bisect between v4.15-rc6 and v4.15-rc7. The kernel bisect will require testing of about 6 test kernels. I built the first test kernel, up to the following commit: d8887f1c7289848e74c92bd4322789a9cd7de699 The test kernel can be downloaded from: http://kernel.ubuntu.com/~jsalisbury/lp1761796 Can you test that kernel and report back if it has the bug or not? I will build the next test kernel based on your test results. Thanks in advance -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
So I've tested all the way from rc2 -> rc7, and confirm that rc7 is the issue. In two different machines: machine1: running kernel was rc3 machine2: running kernel was rc4 In both cases, after install of rc7, and reboot, I see the issue. Logs for the latest: https://pastebin.ubuntu.com/p/yVTWBRqXSC/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
Just to confirm, 4.15-rc1 does not have the issue, but 4.15 final does? If that is the case, we can try to narrow it down further. Could you test v4.15-rc9: http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.15-rc9/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
Finally got around to test this. ANd this are my findings: 1. Installed 4.15-rc1 above, rebooted the machine, and did not see the issue. ubuntu@just-pika:~$ uname -a Linux just-pika 4.15.0-041500rc1-generic #201711262030 SMP Mon Nov 27 01:32:09 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux ubuntu@just-pika:~$ sudo reboot 2. I rebooted for a second time to see if the issue was present, but did not see the issue. 3. On the third attempt. I installed [1]. After I installed, I proceeded to reboot the machine. ANd this is when I saw the issue, even though the [1] kernel wasn't the running kernel. [1]: http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.15/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
I'd like to perform a bisect to figure out what commit caused this regression. We need to identify the earliest kernel where the issue started happening as well as the latest kernel that did not have this issue. Can you test the following kernels and report back? We are looking for the first kernel version that exhibits this bug: 4.14-rc1: http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.14-rc1/ 4.14 Final: http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.14/ 4.15-rc1: http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.15-rc1/ You don't have to test every kernel, just up until the kernel that first has this bug. Thanks in advance! ** No longer affects: linux (Ubuntu Artful) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
This issue is not present in Artful. This issue is only present in Bionic. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
Can you see if this bug happens with the Artful kernel? It is available from: https://launchpad.net/~canonical-kernel-team/+archive/ubuntu/ppa/+build/14537150 Since this bug does not happen with Xenial, we can perform a kernel bisect to identify the commit that introduced the bug. We first need to identify the last kernel version that did not have bug and the first kernel version that does. ** Changed in: linux (Ubuntu Bionic) Assignee: (unassigned) => Joseph Salisbury (jsalisbury) ** Also affects: linux (Ubuntu Artful) Importance: Undecided Status: New ** Changed in: linux (Ubuntu Artful) Status: New => Confirmed ** Changed in: linux (Ubuntu Artful) Importance: Undecided => High ** Changed in: linux (Ubuntu Artful) Assignee: (unassigned) => Joseph Salisbury (jsalisbury) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error on ProLiant DL360 Gen9
collecting logs with mainline kernel. ** Summary changed: - Rebooting bionic fails with hardware error + Rebooting bionic fails with hardware error on ProLiant DL360 Gen9 ** Description changed: From an ephemeral environment in an HPGen9, the machine fails to reboot when the ephemeral environment is Bionic (it does not fail in Xenial). Full log from the moment the 'sudo reboot' was executed: https://pastebin.ubuntu.com/p/hyD6VY7CbW/ The actual failure is: [ OK ] Stopped Remount Root and Kernel File Systems. [ OK ] Reached target Shutdown. [ OK ] Reached target Final Step. Starting Reboot... [ OK ] Stopped Monitoring of LVM2 mirrors,…sing dmeventd or progress polling. Stopping LVM2 metadata daemon... [ OK ] Stopped LVM2 metadata daemon. [ 257.160527] {1}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 1 [ 257.160530] {1}[Hardware Error]: event severity: fatal [ 257.160533] {1}[Hardware Error]: Error 0, type: fatal [ 257.160534] {1}[Hardware Error]: section_type: PCIe error [ 257.160536] {1}[Hardware Error]: port_type: 4, root port [ 257.160541] {1}[Hardware Error]: version: 1.16 [ 257.160544] {1}[Hardware Error]: command: 0x6010, status: 0x0143 [ 257.160546] {1}[Hardware Error]: device_id: :00:01.0 [ 257.160547] {1}[Hardware Error]: slot: 0 [ 257.160548] {1}[Hardware Error]: secondary_bus: 0x03 [ 257.160550] {1}[Hardware Error]: vendor_id: 0x8086, device_id: 0x6f02 [ 257.160552] {1}[Hardware Error]: class_code: 040600 [ 257.160554] {1}[Hardware Error]: bridge: secondary_status: 0x2000, control: 0x0003 [ 257.160556] {1}[Hardware Error]: Error 1, type: fatal [ 257.160557] {1}[Hardware Error]: section_type: PCIe error [ 257.160559] {1}[Hardware Error]: port_type: 4, root port [ 257.160560] {1}[Hardware Error]: version: 1.16 [ 257.160562] {1}[Hardware Error]: command: 0x6010, status: 0x0143 [ 257.160564] {1}[Hardware Error]: device_id: :00:01.0 [ 257.160565] {1}[Hardware Error]: slot: 0 [ 257.160566] {1}[Hardware Error]: secondary_bus: 0x03 [ 257.160567] {1}[Hardware Error]: vendor_id: 0x8086, device_id: 0x6f02 [ 257.160568] {1}[Hardware Error]: class_code: 040600 [ 257.160569] {1}[Hardware Error]: bridge: secondary_status: 0x2000, control: 0x0003 [ 257.160570] Kernel panic - not syncing: Fatal hardware error! [ 257.200091] Kernel Offset: 0x2540 from 0x8100 (relocation range: 0x8000-0xbfff) [ 257.815850] ERST: [Firmware Warn]: Firmware does not respond in time. --- AlsaDevices: total 0 crw-rw 1 root audio 116, 33 Apr 6 17:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay' ApportVersion: 2.20.9-0ubuntu2 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A DistroRelease: Ubuntu 18.04 IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig' MachineType: HP ProLiant DL360 Gen9 NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair Package: linux (not installed) PciMultimedia: ProcEnviron: TERM=screen-256color PATH=(custom, no user) XDG_RUNTIME_DIR= LANG=C.UTF-8 SHELL=/bin/bash ProcFB: 0 EFI VGA ProcKernelCmdLine: BOOT_IMAGE=username/amd64/ga-18.04/bionic/daily/boot-kernel nomodeset ro root=squash:http://10.245.136.6:5248/images/username/amd64/ga-18.04/bionic/daily/squashfs ip=hostname:BOOTIF ip6=off overlayroot=tmpfs overlayroot_cfgdisk=disabled cc:{datasource_list: [MAAS]}end_cc cloud-config-url=http://10.245.136.6:5240/MAAS/metadata/latest/by-id/hqsxdc/?op=get_preseed apparmor=0 log_host=10.245.136.6 log_port=514 --- console=ttyS1,115200n8 BOOTIF=01-94:18:82:7a:82:c8 ProcVersionSignature: User Name 4.15.0-13.14-generic 4.15.10 RelatedPackageVersions: linux-restricted-modules-4.15.0-13-generic N/A linux-backports-modules-4.15.0-13-generic N/A linux-firmware N/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill' Tags: bionic uec-images Uname: Linux 4.15.0-13-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: adm audio cdrom dialout dip floppy lxd netdev plugdev sudo video _MarkForUpload: True dmi.bios.date: 09/13/2016 dmi.bios.vendor: HP dmi.bios.version: P89 dmi.board.name: ProLiant DL360 Gen9 dmi.board.vendor: HP dmi.chassis.type: 23 dmi.chassis.vendor: HP dmi.modalias: dmi:bvnHP:bvrP89:bd09/13/2016:svnHP:pnProLiantDL360Gen9:pvr:rvnHP:rnProLiantDL360Gen9:rvr:cvnHP:ct23:cvr: dmi.product.family: ProLiant dmi.product.name: ProLiant DL360 Gen9 dmi.sys.vendor: HP --- AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Apr 6 17:53
[Bug 1761796] Re: Rebooting bionic fails with hardware error
Test both -proposed & 4.16 kernels, and the issue persists. ** Tags added: kernel-bug-exists-upstream -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1761796 Title: Rebooting bionic fails with hardware error To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1761796/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1761796] Re: Rebooting bionic fails with hardware error
apport information ** Summary changed: - Rebooting bionic running on an ephemeral env (overlayfs) fails with hardware error + Rebooting bionic fails with hardware error ** Description changed: From an ephemeral environment in an HPGen9, the machine fails to reboot - with the following hardware error. - - This doesn't happen on xenial, it only happens in Bionic. + when the ephemeral environment is Bionic (it does not fail in Xenial). Full log from the moment the 'sudo reboot' was executed: https://pastebin.ubuntu.com/p/hyD6VY7CbW/ - + The actual failure is: [ OK ] Stopped Remount Root and Kernel File Systems. [ OK ] Reached target Shutdown. [ OK ] Reached target Final Step. Starting Reboot... [ OK ] Stopped Monitoring of LVM2 mirrors,…sing dmeventd or progress polling. Stopping LVM2 metadata daemon... [ OK ] Stopped LVM2 metadata daemon. [ 257.160527] {1}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 1 [ 257.160530] {1}[Hardware Error]: event severity: fatal [ 257.160533] {1}[Hardware Error]: Error 0, type: fatal [ 257.160534] {1}[Hardware Error]: section_type: PCIe error [ 257.160536] {1}[Hardware Error]: port_type: 4, root port [ 257.160541] {1}[Hardware Error]: version: 1.16 [ 257.160544] {1}[Hardware Error]: command: 0x6010, status: 0x0143 [ 257.160546] {1}[Hardware Error]: device_id: :00:01.0 [ 257.160547] {1}[Hardware Error]: slot: 0 [ 257.160548] {1}[Hardware Error]: secondary_bus: 0x03 [ 257.160550] {1}[Hardware Error]: vendor_id: 0x8086, device_id: 0x6f02 [ 257.160552] {1}[Hardware Error]: class_code: 040600 [ 257.160554] {1}[Hardware Error]: bridge: secondary_status: 0x2000, control: 0x0003 [ 257.160556] {1}[Hardware Error]: Error 1, type: fatal [ 257.160557] {1}[Hardware Error]: section_type: PCIe error [ 257.160559] {1}[Hardware Error]: port_type: 4, root port [ 257.160560] {1}[Hardware Error]: version: 1.16 [ 257.160562] {1}[Hardware Error]: command: 0x6010, status: 0x0143 [ 257.160564] {1}[Hardware Error]: device_id: :00:01.0 [ 257.160565] {1}[Hardware Error]: slot: 0 [ 257.160566] {1}[Hardware Error]: secondary_bus: 0x03 [ 257.160567] {1}[Hardware Error]: vendor_id: 0x8086, device_id: 0x6f02 [ 257.160568] {1}[Hardware Error]: class_code: 040600 [ 257.160569] {1}[Hardware Error]: bridge: secondary_status: 0x2000, control: 0x0003 [ 257.160570] Kernel panic - not syncing: Fatal hardware error! [ 257.200091] Kernel Offset: 0x2540 from 0x8100 (relocation range: 0x8000-0xbfff) [ 257.815850] ERST: [Firmware Warn]: Firmware does not respond in time. - --- + --- AlsaDevices: - total 0 - crw-rw 1 root audio 116, 33 Apr 6 17:40 timer + total 0 + crw-rw 1 root audio 116, 33 Apr 6 17:40 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay' ApportVersion: 2.20.9-0ubuntu2 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A DistroRelease: Ubuntu 18.04 IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig' MachineType: HP ProLiant DL360 Gen9 NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair Package: linux (not installed) PciMultimedia: - + ProcEnviron: - TERM=screen-256color - PATH=(custom, no user) - XDG_RUNTIME_DIR= - LANG=C.UTF-8 - SHELL=/bin/bash + TERM=screen-256color + PATH=(custom, no user) + XDG_RUNTIME_DIR= + LANG=C.UTF-8 + SHELL=/bin/bash ProcFB: 0 EFI VGA ProcKernelCmdLine: BOOT_IMAGE=username/amd64/ga-18.04/bionic/daily/boot-kernel nomodeset ro root=squash:http://10.245.136.6:5248/images/username/amd64/ga-18.04/bionic/daily/squashfs ip=hostname:BOOTIF ip6=off overlayroot=tmpfs overlayroot_cfgdisk=disabled cc:{datasource_list: [MAAS]}end_cc cloud-config-url=http://10.245.136.6:5240/MAAS/metadata/latest/by-id/hqsxdc/?op=get_preseed apparmor=0 log_host=10.245.136.6 log_port=514 --- console=ttyS1,115200n8 BOOTIF=01-94:18:82:7a:82:c8 ProcVersionSignature: User Name 4.15.0-13.14-generic 4.15.10 RelatedPackageVersions: - linux-restricted-modules-4.15.0-13-generic N/A - linux-backports-modules-4.15.0-13-generic N/A - linux-firmware N/A + linux-restricted-modules-4.15.0-13-generic N/A + linux-backports-modules-4.15.0-13-generic N/A + linux-firmware N/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill' Tags: bionic uec-images Uname: Linux 4.15.0-13-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: adm audio cdrom dialout dip floppy lxd netdev plugdev sudo video _MarkForUpload: True dmi.bios.date: 09/13/2016 dmi.bios.vendor: