[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
** Tags added: cscc -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel Status in The Ubuntu-power-systems project: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Bionic: Fix Released Bug description: Problem Description: === Host crashed & enters into xmon after updating to 4.15.0-15.16 kernel kernel. Steps to re-create: == 1. boslcp3 is up with BMC:118 & PNOR: 20180330 levels 2. Installed boslcp3 with latest kernel 4.15.0-13-generic 3. Enabled "-proposed" kernel in /etc/apt/sources.list file 4. Ran sudo apt-get update & apt-get upgrade 5. root@boslcp3:~# ls /boot abi-4.15.0-13-generic retpoline-4.15.0-13-generic abi-4.15.0-15-generic retpoline-4.15.0-15-generic config-4.15.0-13-generic System.map-4.15.0-13-generic config-4.15.0-15-generic System.map-4.15.0-15-generic grub vmlinux initrd.imgvmlinux-4.15.0-13-generic initrd.img-4.15.0-13-generic vmlinux-4.15.0-15-generic initrd.img-4.15.0-15-generic vmlinux.old initrd.img.old 6. Rebooted & booted with 4.15.0-15 kernel 7. Enabled xmon by editing file "vi /etc/default/grub" and ran update-grub 8. Rebooted host. 9. Booted with 4.15.0-15 & provided root/password credentials in login prompt 10. Host crashed & enters into XMON state with 'Unable to handle kernel paging request' root@boslcp3:~# [ 66.295233] Unable to handle kernel paging request for data at address 0x8882f6ed90e9151a [ 66.295297] Faulting instruction address: 0xc038a110 cpu 0x50: Vector: 380 (Data Access Out of Range) at [c692f650] pc: c038a110: kmem_cache_alloc_node+0x2f0/0x350 lr: c038a0fc: kmem_cache_alloc_node+0x2dc/0x350 sp: c692f8d0 msr: 90009033 dar: 8882f6ed90e9151a current = 0xc698fd00 paca= 0xcfab7000 softe: 0irq_happened: 0x01 pid = 1762, comm = systemd-journal Linux version 4.15.0-15-generic (buildd@bos02-ppc64el-002) (gcc version 7.3.0 (Ubuntu 7.3.0-14ubuntu1)) #16-Ubuntu SMP Wed Apr 4 13:57:51 UTC 2018 (Ubuntu 4.15.0-15.16-generic 4.15.15) enter ? for help [c692f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c692f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c692f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c692fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c692fae0] c0c5705c unix_dgram_sendmsg+0x15c/0x8f0 [c692fbc0] c0b1ec64 sock_sendmsg+0x64/0x90 [c692fbf0] c0b20abc ___sys_sendmsg+0x31c/0x390 [c692fd90] c0b221ec __sys_sendmsg+0x5c/0xc0 [c692fe30] c000b184 system_call+0x58/0x6c --- Exception: c00 (System Call) at 74826f6fa9c4 SP (75dc5510) is in userspace 50:mon> 50:mon> 10. Attached Host console logs I rebooted the host just to see if it would hit the issue again and this time I didn't even get to the login prompt but it crashed in the same location: 50:mon> r R00 = c0389fd4 R16 = c000200e0b20fdc0 R01 = c000200e0b20f8d0 R17 = 0048 R02 = c16eb400 R18 = 0001fe80 R03 = 0001 R19 = R04 = 0048ca1cff37803d R20 = R05 = 0688 R21 = R06 = 0001 R22 = 0048 R07 = 0687 R23 = 4882d6e3c8b7ab55 R08 = 48ca1cff37802b68 R24 = c000200e5851df01 R09 = R25 = 8882f6ed90e67454 R10 = R26 = c0b2ec6c R11 = c0d10f78 R27 = c00ff901ee00 R12 = 2000 R28 = R13 = cfab7000 R29 = 015004c0 R14 = c000200e4c973fc8 R30 = c000200e5851df01 R15 = c000200e4c974238 R31 = c00ff901ee00 pc = c038a110 kmem_cache_alloc_node+0x2f0/0x350 cfar= c0016e1c arch_local_irq_restore+0x1c/0x90 lr = c038a0fc kmem_cache_alloc_node+0x2dc/0x350 msr = 90009033 cr = 28002844 ctr = c061e1b0 xer = trap = 380 dar = 8882f6ed90e67454 dsisr = c000200e40bd8400 50:mon> t [c000200e0b20f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c000200e0b20f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c000200e0b20f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c000200e0b20fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c000200e0b20fae0] c0c56ae4 unix_stream_sendmsg+0x264/0x5c0 [c000200e0b20fbc0] c0b1ec64
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
The bionic-proposed kernel referred to in comment #268 has now been released. Marking as "Fix Released". -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel Status in The Ubuntu-power-systems project: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Bionic: Fix Released Bug description: Problem Description: === Host crashed & enters into xmon after updating to 4.15.0-15.16 kernel kernel. Steps to re-create: == 1. boslcp3 is up with BMC:118 & PNOR: 20180330 levels 2. Installed boslcp3 with latest kernel 4.15.0-13-generic 3. Enabled "-proposed" kernel in /etc/apt/sources.list file 4. Ran sudo apt-get update & apt-get upgrade 5. root@boslcp3:~# ls /boot abi-4.15.0-13-generic retpoline-4.15.0-13-generic abi-4.15.0-15-generic retpoline-4.15.0-15-generic config-4.15.0-13-generic System.map-4.15.0-13-generic config-4.15.0-15-generic System.map-4.15.0-15-generic grub vmlinux initrd.imgvmlinux-4.15.0-13-generic initrd.img-4.15.0-13-generic vmlinux-4.15.0-15-generic initrd.img-4.15.0-15-generic vmlinux.old initrd.img.old 6. Rebooted & booted with 4.15.0-15 kernel 7. Enabled xmon by editing file "vi /etc/default/grub" and ran update-grub 8. Rebooted host. 9. Booted with 4.15.0-15 & provided root/password credentials in login prompt 10. Host crashed & enters into XMON state with 'Unable to handle kernel paging request' root@boslcp3:~# [ 66.295233] Unable to handle kernel paging request for data at address 0x8882f6ed90e9151a [ 66.295297] Faulting instruction address: 0xc038a110 cpu 0x50: Vector: 380 (Data Access Out of Range) at [c692f650] pc: c038a110: kmem_cache_alloc_node+0x2f0/0x350 lr: c038a0fc: kmem_cache_alloc_node+0x2dc/0x350 sp: c692f8d0 msr: 90009033 dar: 8882f6ed90e9151a current = 0xc698fd00 paca= 0xcfab7000 softe: 0irq_happened: 0x01 pid = 1762, comm = systemd-journal Linux version 4.15.0-15-generic (buildd@bos02-ppc64el-002) (gcc version 7.3.0 (Ubuntu 7.3.0-14ubuntu1)) #16-Ubuntu SMP Wed Apr 4 13:57:51 UTC 2018 (Ubuntu 4.15.0-15.16-generic 4.15.15) enter ? for help [c692f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c692f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c692f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c692fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c692fae0] c0c5705c unix_dgram_sendmsg+0x15c/0x8f0 [c692fbc0] c0b1ec64 sock_sendmsg+0x64/0x90 [c692fbf0] c0b20abc ___sys_sendmsg+0x31c/0x390 [c692fd90] c0b221ec __sys_sendmsg+0x5c/0xc0 [c692fe30] c000b184 system_call+0x58/0x6c --- Exception: c00 (System Call) at 74826f6fa9c4 SP (75dc5510) is in userspace 50:mon> 50:mon> 10. Attached Host console logs I rebooted the host just to see if it would hit the issue again and this time I didn't even get to the login prompt but it crashed in the same location: 50:mon> r R00 = c0389fd4 R16 = c000200e0b20fdc0 R01 = c000200e0b20f8d0 R17 = 0048 R02 = c16eb400 R18 = 0001fe80 R03 = 0001 R19 = R04 = 0048ca1cff37803d R20 = R05 = 0688 R21 = R06 = 0001 R22 = 0048 R07 = 0687 R23 = 4882d6e3c8b7ab55 R08 = 48ca1cff37802b68 R24 = c000200e5851df01 R09 = R25 = 8882f6ed90e67454 R10 = R26 = c0b2ec6c R11 = c0d10f78 R27 = c00ff901ee00 R12 = 2000 R28 = R13 = cfab7000 R29 = 015004c0 R14 = c000200e4c973fc8 R30 = c000200e5851df01 R15 = c000200e4c974238 R31 = c00ff901ee00 pc = c038a110 kmem_cache_alloc_node+0x2f0/0x350 cfar= c0016e1c arch_local_irq_restore+0x1c/0x90 lr = c038a0fc kmem_cache_alloc_node+0x2dc/0x350 msr = 90009033 cr = 28002844 ctr = c061e1b0 xer = trap = 380 dar = 8882f6ed90e67454 dsisr = c000200e40bd8400 50:mon> t [c000200e0b20f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c000200e0b20f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c000200e0b20f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c000200e0b20fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c000200e0b20fae0] c0c56ae4
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
** Changed in: linux (Ubuntu Bionic) Status: Fix Committed => Fix Released ** Changed in: linux (Ubuntu) Status: Fix Committed => Fix Released ** Changed in: ubuntu-power-systems Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel Status in The Ubuntu-power-systems project: Fix Released Status in linux package in Ubuntu: Fix Released Status in linux source package in Bionic: Fix Released Bug description: Problem Description: === Host crashed & enters into xmon after updating to 4.15.0-15.16 kernel kernel. Steps to re-create: == 1. boslcp3 is up with BMC:118 & PNOR: 20180330 levels 2. Installed boslcp3 with latest kernel 4.15.0-13-generic 3. Enabled "-proposed" kernel in /etc/apt/sources.list file 4. Ran sudo apt-get update & apt-get upgrade 5. root@boslcp3:~# ls /boot abi-4.15.0-13-generic retpoline-4.15.0-13-generic abi-4.15.0-15-generic retpoline-4.15.0-15-generic config-4.15.0-13-generic System.map-4.15.0-13-generic config-4.15.0-15-generic System.map-4.15.0-15-generic grub vmlinux initrd.imgvmlinux-4.15.0-13-generic initrd.img-4.15.0-13-generic vmlinux-4.15.0-15-generic initrd.img-4.15.0-15-generic vmlinux.old initrd.img.old 6. Rebooted & booted with 4.15.0-15 kernel 7. Enabled xmon by editing file "vi /etc/default/grub" and ran update-grub 8. Rebooted host. 9. Booted with 4.15.0-15 & provided root/password credentials in login prompt 10. Host crashed & enters into XMON state with 'Unable to handle kernel paging request' root@boslcp3:~# [ 66.295233] Unable to handle kernel paging request for data at address 0x8882f6ed90e9151a [ 66.295297] Faulting instruction address: 0xc038a110 cpu 0x50: Vector: 380 (Data Access Out of Range) at [c692f650] pc: c038a110: kmem_cache_alloc_node+0x2f0/0x350 lr: c038a0fc: kmem_cache_alloc_node+0x2dc/0x350 sp: c692f8d0 msr: 90009033 dar: 8882f6ed90e9151a current = 0xc698fd00 paca= 0xcfab7000 softe: 0irq_happened: 0x01 pid = 1762, comm = systemd-journal Linux version 4.15.0-15-generic (buildd@bos02-ppc64el-002) (gcc version 7.3.0 (Ubuntu 7.3.0-14ubuntu1)) #16-Ubuntu SMP Wed Apr 4 13:57:51 UTC 2018 (Ubuntu 4.15.0-15.16-generic 4.15.15) enter ? for help [c692f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c692f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c692f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c692fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c692fae0] c0c5705c unix_dgram_sendmsg+0x15c/0x8f0 [c692fbc0] c0b1ec64 sock_sendmsg+0x64/0x90 [c692fbf0] c0b20abc ___sys_sendmsg+0x31c/0x390 [c692fd90] c0b221ec __sys_sendmsg+0x5c/0xc0 [c692fe30] c000b184 system_call+0x58/0x6c --- Exception: c00 (System Call) at 74826f6fa9c4 SP (75dc5510) is in userspace 50:mon> 50:mon> 10. Attached Host console logs I rebooted the host just to see if it would hit the issue again and this time I didn't even get to the login prompt but it crashed in the same location: 50:mon> r R00 = c0389fd4 R16 = c000200e0b20fdc0 R01 = c000200e0b20f8d0 R17 = 0048 R02 = c16eb400 R18 = 0001fe80 R03 = 0001 R19 = R04 = 0048ca1cff37803d R20 = R05 = 0688 R21 = R06 = 0001 R22 = 0048 R07 = 0687 R23 = 4882d6e3c8b7ab55 R08 = 48ca1cff37802b68 R24 = c000200e5851df01 R09 = R25 = 8882f6ed90e67454 R10 = R26 = c0b2ec6c R11 = c0d10f78 R27 = c00ff901ee00 R12 = 2000 R28 = R13 = cfab7000 R29 = 015004c0 R14 = c000200e4c973fc8 R30 = c000200e5851df01 R15 = c000200e4c974238 R31 = c00ff901ee00 pc = c038a110 kmem_cache_alloc_node+0x2f0/0x350 cfar= c0016e1c arch_local_irq_restore+0x1c/0x90 lr = c038a0fc kmem_cache_alloc_node+0x2dc/0x350 msr = 90009033 cr = 28002844 ctr = c061e1b0 xer = trap = 380 dar = 8882f6ed90e67454 dsisr = c000200e40bd8400 50:mon> t [c000200e0b20f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c000200e0b20f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c000200e0b20f9a0] c0b30b6c
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
** Tags removed: triage-a ** Tags added: triage-g -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel Status in The Ubuntu-power-systems project: Fix Committed Status in linux package in Ubuntu: Fix Committed Status in linux source package in Bionic: Fix Committed Bug description: Problem Description: === Host crashed & enters into xmon after updating to 4.15.0-15.16 kernel kernel. Steps to re-create: == 1. boslcp3 is up with BMC:118 & PNOR: 20180330 levels 2. Installed boslcp3 with latest kernel 4.15.0-13-generic 3. Enabled "-proposed" kernel in /etc/apt/sources.list file 4. Ran sudo apt-get update & apt-get upgrade 5. root@boslcp3:~# ls /boot abi-4.15.0-13-generic retpoline-4.15.0-13-generic abi-4.15.0-15-generic retpoline-4.15.0-15-generic config-4.15.0-13-generic System.map-4.15.0-13-generic config-4.15.0-15-generic System.map-4.15.0-15-generic grub vmlinux initrd.imgvmlinux-4.15.0-13-generic initrd.img-4.15.0-13-generic vmlinux-4.15.0-15-generic initrd.img-4.15.0-15-generic vmlinux.old initrd.img.old 6. Rebooted & booted with 4.15.0-15 kernel 7. Enabled xmon by editing file "vi /etc/default/grub" and ran update-grub 8. Rebooted host. 9. Booted with 4.15.0-15 & provided root/password credentials in login prompt 10. Host crashed & enters into XMON state with 'Unable to handle kernel paging request' root@boslcp3:~# [ 66.295233] Unable to handle kernel paging request for data at address 0x8882f6ed90e9151a [ 66.295297] Faulting instruction address: 0xc038a110 cpu 0x50: Vector: 380 (Data Access Out of Range) at [c692f650] pc: c038a110: kmem_cache_alloc_node+0x2f0/0x350 lr: c038a0fc: kmem_cache_alloc_node+0x2dc/0x350 sp: c692f8d0 msr: 90009033 dar: 8882f6ed90e9151a current = 0xc698fd00 paca= 0xcfab7000 softe: 0irq_happened: 0x01 pid = 1762, comm = systemd-journal Linux version 4.15.0-15-generic (buildd@bos02-ppc64el-002) (gcc version 7.3.0 (Ubuntu 7.3.0-14ubuntu1)) #16-Ubuntu SMP Wed Apr 4 13:57:51 UTC 2018 (Ubuntu 4.15.0-15.16-generic 4.15.15) enter ? for help [c692f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c692f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c692f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c692fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c692fae0] c0c5705c unix_dgram_sendmsg+0x15c/0x8f0 [c692fbc0] c0b1ec64 sock_sendmsg+0x64/0x90 [c692fbf0] c0b20abc ___sys_sendmsg+0x31c/0x390 [c692fd90] c0b221ec __sys_sendmsg+0x5c/0xc0 [c692fe30] c000b184 system_call+0x58/0x6c --- Exception: c00 (System Call) at 74826f6fa9c4 SP (75dc5510) is in userspace 50:mon> 50:mon> 10. Attached Host console logs I rebooted the host just to see if it would hit the issue again and this time I didn't even get to the login prompt but it crashed in the same location: 50:mon> r R00 = c0389fd4 R16 = c000200e0b20fdc0 R01 = c000200e0b20f8d0 R17 = 0048 R02 = c16eb400 R18 = 0001fe80 R03 = 0001 R19 = R04 = 0048ca1cff37803d R20 = R05 = 0688 R21 = R06 = 0001 R22 = 0048 R07 = 0687 R23 = 4882d6e3c8b7ab55 R08 = 48ca1cff37802b68 R24 = c000200e5851df01 R09 = R25 = 8882f6ed90e67454 R10 = R26 = c0b2ec6c R11 = c0d10f78 R27 = c00ff901ee00 R12 = 2000 R28 = R13 = cfab7000 R29 = 015004c0 R14 = c000200e4c973fc8 R30 = c000200e5851df01 R15 = c000200e4c974238 R31 = c00ff901ee00 pc = c038a110 kmem_cache_alloc_node+0x2f0/0x350 cfar= c0016e1c arch_local_irq_restore+0x1c/0x90 lr = c038a0fc kmem_cache_alloc_node+0x2dc/0x350 msr = 90009033 cr = 28002844 ctr = c061e1b0 xer = trap = 380 dar = 8882f6ed90e67454 dsisr = c000200e40bd8400 50:mon> t [c000200e0b20f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c000200e0b20f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c000200e0b20f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c000200e0b20fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c000200e0b20fae0] c0c56ae4 unix_stream_sendmsg+0x264/0x5c0 [c000200e0b20fbc0]
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
Looks like the bionic-proposed kernel works for IBM, and so marking this fix-committed. ** Changed in: linux (Ubuntu Bionic) Status: Incomplete => Fix Committed ** Changed in: linux (Ubuntu) Status: Incomplete => Fix Committed ** Changed in: ubuntu-power-systems Status: Incomplete => Fix Committed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel Status in The Ubuntu-power-systems project: Fix Committed Status in linux package in Ubuntu: Fix Committed Status in linux source package in Bionic: Fix Committed Bug description: Problem Description: === Host crashed & enters into xmon after updating to 4.15.0-15.16 kernel kernel. Steps to re-create: == 1. boslcp3 is up with BMC:118 & PNOR: 20180330 levels 2. Installed boslcp3 with latest kernel 4.15.0-13-generic 3. Enabled "-proposed" kernel in /etc/apt/sources.list file 4. Ran sudo apt-get update & apt-get upgrade 5. root@boslcp3:~# ls /boot abi-4.15.0-13-generic retpoline-4.15.0-13-generic abi-4.15.0-15-generic retpoline-4.15.0-15-generic config-4.15.0-13-generic System.map-4.15.0-13-generic config-4.15.0-15-generic System.map-4.15.0-15-generic grub vmlinux initrd.imgvmlinux-4.15.0-13-generic initrd.img-4.15.0-13-generic vmlinux-4.15.0-15-generic initrd.img-4.15.0-15-generic vmlinux.old initrd.img.old 6. Rebooted & booted with 4.15.0-15 kernel 7. Enabled xmon by editing file "vi /etc/default/grub" and ran update-grub 8. Rebooted host. 9. Booted with 4.15.0-15 & provided root/password credentials in login prompt 10. Host crashed & enters into XMON state with 'Unable to handle kernel paging request' root@boslcp3:~# [ 66.295233] Unable to handle kernel paging request for data at address 0x8882f6ed90e9151a [ 66.295297] Faulting instruction address: 0xc038a110 cpu 0x50: Vector: 380 (Data Access Out of Range) at [c692f650] pc: c038a110: kmem_cache_alloc_node+0x2f0/0x350 lr: c038a0fc: kmem_cache_alloc_node+0x2dc/0x350 sp: c692f8d0 msr: 90009033 dar: 8882f6ed90e9151a current = 0xc698fd00 paca= 0xcfab7000 softe: 0irq_happened: 0x01 pid = 1762, comm = systemd-journal Linux version 4.15.0-15-generic (buildd@bos02-ppc64el-002) (gcc version 7.3.0 (Ubuntu 7.3.0-14ubuntu1)) #16-Ubuntu SMP Wed Apr 4 13:57:51 UTC 2018 (Ubuntu 4.15.0-15.16-generic 4.15.15) enter ? for help [c692f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c692f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c692f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c692fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c692fae0] c0c5705c unix_dgram_sendmsg+0x15c/0x8f0 [c692fbc0] c0b1ec64 sock_sendmsg+0x64/0x90 [c692fbf0] c0b20abc ___sys_sendmsg+0x31c/0x390 [c692fd90] c0b221ec __sys_sendmsg+0x5c/0xc0 [c692fe30] c000b184 system_call+0x58/0x6c --- Exception: c00 (System Call) at 74826f6fa9c4 SP (75dc5510) is in userspace 50:mon> 50:mon> 10. Attached Host console logs I rebooted the host just to see if it would hit the issue again and this time I didn't even get to the login prompt but it crashed in the same location: 50:mon> r R00 = c0389fd4 R16 = c000200e0b20fdc0 R01 = c000200e0b20f8d0 R17 = 0048 R02 = c16eb400 R18 = 0001fe80 R03 = 0001 R19 = R04 = 0048ca1cff37803d R20 = R05 = 0688 R21 = R06 = 0001 R22 = 0048 R07 = 0687 R23 = 4882d6e3c8b7ab55 R08 = 48ca1cff37802b68 R24 = c000200e5851df01 R09 = R25 = 8882f6ed90e67454 R10 = R26 = c0b2ec6c R11 = c0d10f78 R27 = c00ff901ee00 R12 = 2000 R28 = R13 = cfab7000 R29 = 015004c0 R14 = c000200e4c973fc8 R30 = c000200e5851df01 R15 = c000200e4c974238 R31 = c00ff901ee00 pc = c038a110 kmem_cache_alloc_node+0x2f0/0x350 cfar= c0016e1c arch_local_irq_restore+0x1c/0x90 lr = c038a0fc kmem_cache_alloc_node+0x2dc/0x350 msr = 90009033 cr = 28002844 ctr = c061e1b0 xer = trap = 380 dar = 8882f6ed90e67454 dsisr = c000200e40bd8400 50:mon> t [c000200e0b20f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c000200e0b20f940]
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
** Tags added: triage-a -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel Status in The Ubuntu-power-systems project: Fix Committed Status in linux package in Ubuntu: Fix Committed Status in linux source package in Bionic: Fix Committed Bug description: Problem Description: === Host crashed & enters into xmon after updating to 4.15.0-15.16 kernel kernel. Steps to re-create: == 1. boslcp3 is up with BMC:118 & PNOR: 20180330 levels 2. Installed boslcp3 with latest kernel 4.15.0-13-generic 3. Enabled "-proposed" kernel in /etc/apt/sources.list file 4. Ran sudo apt-get update & apt-get upgrade 5. root@boslcp3:~# ls /boot abi-4.15.0-13-generic retpoline-4.15.0-13-generic abi-4.15.0-15-generic retpoline-4.15.0-15-generic config-4.15.0-13-generic System.map-4.15.0-13-generic config-4.15.0-15-generic System.map-4.15.0-15-generic grub vmlinux initrd.imgvmlinux-4.15.0-13-generic initrd.img-4.15.0-13-generic vmlinux-4.15.0-15-generic initrd.img-4.15.0-15-generic vmlinux.old initrd.img.old 6. Rebooted & booted with 4.15.0-15 kernel 7. Enabled xmon by editing file "vi /etc/default/grub" and ran update-grub 8. Rebooted host. 9. Booted with 4.15.0-15 & provided root/password credentials in login prompt 10. Host crashed & enters into XMON state with 'Unable to handle kernel paging request' root@boslcp3:~# [ 66.295233] Unable to handle kernel paging request for data at address 0x8882f6ed90e9151a [ 66.295297] Faulting instruction address: 0xc038a110 cpu 0x50: Vector: 380 (Data Access Out of Range) at [c692f650] pc: c038a110: kmem_cache_alloc_node+0x2f0/0x350 lr: c038a0fc: kmem_cache_alloc_node+0x2dc/0x350 sp: c692f8d0 msr: 90009033 dar: 8882f6ed90e9151a current = 0xc698fd00 paca= 0xcfab7000 softe: 0irq_happened: 0x01 pid = 1762, comm = systemd-journal Linux version 4.15.0-15-generic (buildd@bos02-ppc64el-002) (gcc version 7.3.0 (Ubuntu 7.3.0-14ubuntu1)) #16-Ubuntu SMP Wed Apr 4 13:57:51 UTC 2018 (Ubuntu 4.15.0-15.16-generic 4.15.15) enter ? for help [c692f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c692f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c692f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c692fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c692fae0] c0c5705c unix_dgram_sendmsg+0x15c/0x8f0 [c692fbc0] c0b1ec64 sock_sendmsg+0x64/0x90 [c692fbf0] c0b20abc ___sys_sendmsg+0x31c/0x390 [c692fd90] c0b221ec __sys_sendmsg+0x5c/0xc0 [c692fe30] c000b184 system_call+0x58/0x6c --- Exception: c00 (System Call) at 74826f6fa9c4 SP (75dc5510) is in userspace 50:mon> 50:mon> 10. Attached Host console logs I rebooted the host just to see if it would hit the issue again and this time I didn't even get to the login prompt but it crashed in the same location: 50:mon> r R00 = c0389fd4 R16 = c000200e0b20fdc0 R01 = c000200e0b20f8d0 R17 = 0048 R02 = c16eb400 R18 = 0001fe80 R03 = 0001 R19 = R04 = 0048ca1cff37803d R20 = R05 = 0688 R21 = R06 = 0001 R22 = 0048 R07 = 0687 R23 = 4882d6e3c8b7ab55 R08 = 48ca1cff37802b68 R24 = c000200e5851df01 R09 = R25 = 8882f6ed90e67454 R10 = R26 = c0b2ec6c R11 = c0d10f78 R27 = c00ff901ee00 R12 = 2000 R28 = R13 = cfab7000 R29 = 015004c0 R14 = c000200e4c973fc8 R30 = c000200e5851df01 R15 = c000200e4c974238 R31 = c00ff901ee00 pc = c038a110 kmem_cache_alloc_node+0x2f0/0x350 cfar= c0016e1c arch_local_irq_restore+0x1c/0x90 lr = c038a0fc kmem_cache_alloc_node+0x2dc/0x350 msr = 90009033 cr = 28002844 ctr = c061e1b0 xer = trap = 380 dar = 8882f6ed90e67454 dsisr = c000200e40bd8400 50:mon> t [c000200e0b20f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c000200e0b20f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c000200e0b20f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c000200e0b20fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c000200e0b20fae0] c0c56ae4 unix_stream_sendmsg+0x264/0x5c0 [c000200e0b20fbc0] c0b1ec64
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
** Tags added: p9 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel Status in The Ubuntu-power-systems project: Incomplete Status in linux package in Ubuntu: Incomplete Status in linux source package in Bionic: Incomplete Bug description: Problem Description: === Host crashed & enters into xmon after updating to 4.15.0-15.16 kernel kernel. Steps to re-create: == 1. boslcp3 is up with BMC:118 & PNOR: 20180330 levels 2. Installed boslcp3 with latest kernel 4.15.0-13-generic 3. Enabled "-proposed" kernel in /etc/apt/sources.list file 4. Ran sudo apt-get update & apt-get upgrade 5. root@boslcp3:~# ls /boot abi-4.15.0-13-generic retpoline-4.15.0-13-generic abi-4.15.0-15-generic retpoline-4.15.0-15-generic config-4.15.0-13-generic System.map-4.15.0-13-generic config-4.15.0-15-generic System.map-4.15.0-15-generic grub vmlinux initrd.imgvmlinux-4.15.0-13-generic initrd.img-4.15.0-13-generic vmlinux-4.15.0-15-generic initrd.img-4.15.0-15-generic vmlinux.old initrd.img.old 6. Rebooted & booted with 4.15.0-15 kernel 7. Enabled xmon by editing file "vi /etc/default/grub" and ran update-grub 8. Rebooted host. 9. Booted with 4.15.0-15 & provided root/password credentials in login prompt 10. Host crashed & enters into XMON state with 'Unable to handle kernel paging request' root@boslcp3:~# [ 66.295233] Unable to handle kernel paging request for data at address 0x8882f6ed90e9151a [ 66.295297] Faulting instruction address: 0xc038a110 cpu 0x50: Vector: 380 (Data Access Out of Range) at [c692f650] pc: c038a110: kmem_cache_alloc_node+0x2f0/0x350 lr: c038a0fc: kmem_cache_alloc_node+0x2dc/0x350 sp: c692f8d0 msr: 90009033 dar: 8882f6ed90e9151a current = 0xc698fd00 paca= 0xcfab7000 softe: 0irq_happened: 0x01 pid = 1762, comm = systemd-journal Linux version 4.15.0-15-generic (buildd@bos02-ppc64el-002) (gcc version 7.3.0 (Ubuntu 7.3.0-14ubuntu1)) #16-Ubuntu SMP Wed Apr 4 13:57:51 UTC 2018 (Ubuntu 4.15.0-15.16-generic 4.15.15) enter ? for help [c692f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c692f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c692f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c692fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c692fae0] c0c5705c unix_dgram_sendmsg+0x15c/0x8f0 [c692fbc0] c0b1ec64 sock_sendmsg+0x64/0x90 [c692fbf0] c0b20abc ___sys_sendmsg+0x31c/0x390 [c692fd90] c0b221ec __sys_sendmsg+0x5c/0xc0 [c692fe30] c000b184 system_call+0x58/0x6c --- Exception: c00 (System Call) at 74826f6fa9c4 SP (75dc5510) is in userspace 50:mon> 50:mon> 10. Attached Host console logs I rebooted the host just to see if it would hit the issue again and this time I didn't even get to the login prompt but it crashed in the same location: 50:mon> r R00 = c0389fd4 R16 = c000200e0b20fdc0 R01 = c000200e0b20f8d0 R17 = 0048 R02 = c16eb400 R18 = 0001fe80 R03 = 0001 R19 = R04 = 0048ca1cff37803d R20 = R05 = 0688 R21 = R06 = 0001 R22 = 0048 R07 = 0687 R23 = 4882d6e3c8b7ab55 R08 = 48ca1cff37802b68 R24 = c000200e5851df01 R09 = R25 = 8882f6ed90e67454 R10 = R26 = c0b2ec6c R11 = c0d10f78 R27 = c00ff901ee00 R12 = 2000 R28 = R13 = cfab7000 R29 = 015004c0 R14 = c000200e4c973fc8 R30 = c000200e5851df01 R15 = c000200e4c974238 R31 = c00ff901ee00 pc = c038a110 kmem_cache_alloc_node+0x2f0/0x350 cfar= c0016e1c arch_local_irq_restore+0x1c/0x90 lr = c038a0fc kmem_cache_alloc_node+0x2dc/0x350 msr = 90009033 cr = 28002844 ctr = c061e1b0 xer = trap = 380 dar = 8882f6ed90e67454 dsisr = c000200e40bd8400 50:mon> t [c000200e0b20f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c000200e0b20f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c000200e0b20f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c000200e0b20fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c000200e0b20fae0] c0c56ae4 unix_stream_sendmsg+0x264/0x5c0 [c000200e0b20fbc0] c0b1ec64 sock_sendmsg+0x64/0x90
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
** Tags removed: severity-critical ** Tags added: severity-high -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel Status in The Ubuntu-power-systems project: Incomplete Status in linux package in Ubuntu: Incomplete Status in linux source package in Bionic: Incomplete Bug description: Problem Description: === Host crashed & enters into xmon after updating to 4.15.0-15.16 kernel kernel. Steps to re-create: == 1. boslcp3 is up with BMC:118 & PNOR: 20180330 levels 2. Installed boslcp3 with latest kernel 4.15.0-13-generic 3. Enabled "-proposed" kernel in /etc/apt/sources.list file 4. Ran sudo apt-get update & apt-get upgrade 5. root@boslcp3:~# ls /boot abi-4.15.0-13-generic retpoline-4.15.0-13-generic abi-4.15.0-15-generic retpoline-4.15.0-15-generic config-4.15.0-13-generic System.map-4.15.0-13-generic config-4.15.0-15-generic System.map-4.15.0-15-generic grub vmlinux initrd.imgvmlinux-4.15.0-13-generic initrd.img-4.15.0-13-generic vmlinux-4.15.0-15-generic initrd.img-4.15.0-15-generic vmlinux.old initrd.img.old 6. Rebooted & booted with 4.15.0-15 kernel 7. Enabled xmon by editing file "vi /etc/default/grub" and ran update-grub 8. Rebooted host. 9. Booted with 4.15.0-15 & provided root/password credentials in login prompt 10. Host crashed & enters into XMON state with 'Unable to handle kernel paging request' root@boslcp3:~# [ 66.295233] Unable to handle kernel paging request for data at address 0x8882f6ed90e9151a [ 66.295297] Faulting instruction address: 0xc038a110 cpu 0x50: Vector: 380 (Data Access Out of Range) at [c692f650] pc: c038a110: kmem_cache_alloc_node+0x2f0/0x350 lr: c038a0fc: kmem_cache_alloc_node+0x2dc/0x350 sp: c692f8d0 msr: 90009033 dar: 8882f6ed90e9151a current = 0xc698fd00 paca= 0xcfab7000 softe: 0irq_happened: 0x01 pid = 1762, comm = systemd-journal Linux version 4.15.0-15-generic (buildd@bos02-ppc64el-002) (gcc version 7.3.0 (Ubuntu 7.3.0-14ubuntu1)) #16-Ubuntu SMP Wed Apr 4 13:57:51 UTC 2018 (Ubuntu 4.15.0-15.16-generic 4.15.15) enter ? for help [c692f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c692f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c692f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c692fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c692fae0] c0c5705c unix_dgram_sendmsg+0x15c/0x8f0 [c692fbc0] c0b1ec64 sock_sendmsg+0x64/0x90 [c692fbf0] c0b20abc ___sys_sendmsg+0x31c/0x390 [c692fd90] c0b221ec __sys_sendmsg+0x5c/0xc0 [c692fe30] c000b184 system_call+0x58/0x6c --- Exception: c00 (System Call) at 74826f6fa9c4 SP (75dc5510) is in userspace 50:mon> 50:mon> 10. Attached Host console logs I rebooted the host just to see if it would hit the issue again and this time I didn't even get to the login prompt but it crashed in the same location: 50:mon> r R00 = c0389fd4 R16 = c000200e0b20fdc0 R01 = c000200e0b20f8d0 R17 = 0048 R02 = c16eb400 R18 = 0001fe80 R03 = 0001 R19 = R04 = 0048ca1cff37803d R20 = R05 = 0688 R21 = R06 = 0001 R22 = 0048 R07 = 0687 R23 = 4882d6e3c8b7ab55 R08 = 48ca1cff37802b68 R24 = c000200e5851df01 R09 = R25 = 8882f6ed90e67454 R10 = R26 = c0b2ec6c R11 = c0d10f78 R27 = c00ff901ee00 R12 = 2000 R28 = R13 = cfab7000 R29 = 015004c0 R14 = c000200e4c973fc8 R30 = c000200e5851df01 R15 = c000200e4c974238 R31 = c00ff901ee00 pc = c038a110 kmem_cache_alloc_node+0x2f0/0x350 cfar= c0016e1c arch_local_irq_restore+0x1c/0x90 lr = c038a0fc kmem_cache_alloc_node+0x2dc/0x350 msr = 90009033 cr = 28002844 ctr = c061e1b0 xer = trap = 380 dar = 8882f6ed90e67454 dsisr = c000200e40bd8400 50:mon> t [c000200e0b20f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c000200e0b20f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c000200e0b20f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c000200e0b20fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c000200e0b20fae0] c0c56ae4 unix_stream_sendmsg+0x264/0x5c0
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
** Attachment removed: "boslcp3g3 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127826/+files/boslcp3g3.dumpxml ** Attachment removed: "boslcp3g3 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127711/+files/boslcp3g3.dumpxml ** Attachment removed: "boslcp3g3 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5126528/+files/boslcp3g3.dumpxml ** Attachment removed: "boslcp3g1 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134457/+files/boslcp3g1.dumpxml ** Attachment removed: "boslcp3g1 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5133950/+files/boslcp3g1.dumpxml ** Attachment removed: "boslcp3g1 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132669/+files/boslcp3g1.dumpxml ** Attachment removed: "boslcp3g1 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132339/+files/boslcp3g1.dumpxml ** Attachment removed: "boslcp3g1 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127941/+files/boslcp3g1.dumpxml ** Attachment removed: "boslcp3g1 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127825/+files/boslcp3g1.dumpxml ** Attachment removed: "boslcp3g1 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5126527/+files/boslcp3g1.dumpxml ** Attachment removed: "sol console log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132668/+files/boslcp3.0421.txt ** Attachment removed: "sol console log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127940/+files/boslcp3.0421.txt ** Attachment removed: "sol console log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127823/+files/boslcp3.0421.txt ** Attachment removed: "sol console log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5126526/+files/boslcp3.0421.txt ** Attachment removed: "sol console log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5126071/+files/boslcp3.0421.txt ** Attachment removed: "sol console log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5125483/+files/boslcp3.0421.txt ** Attachment removed: "sol console log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5125257/+files/boslcp3.0421.txt ** Attachment removed: "console log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132667/+files/boslcp3.0420.txt ** Attachment removed: "console log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127939/+files/boslcp3.0420.txt ** Attachment removed: "console log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127822/+files/boslcp3.0420.txt ** Attachment removed: "console log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5125482/+files/boslcp3.0420.txt ** Attachment removed: "console log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5124760/+files/boslcp3.0420.txt ** Attachment removed: "dmesg output 0418" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132666/+files/dmesg.201804181042 ** Attachment removed: "dmesg output 0418" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127938/+files/dmesg.201804181042 ** Attachment removed: "dmesg output 0418" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127821/+files/dmesg.201804181042 ** Attachment removed: "dmesg output 0418" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5125481/+files/dmesg.201804181042 ** Attachment removed: "dmesg output 0418" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5124759/+files/dmesg.201804181042 ** Attachment removed: "dmesg logs from xmon prompt_boslcp3" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132665/+files/dmesg_xmon_boslcp3.txt ** Attachment removed: "dmesg logs from xmon prompt_boslcp3" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127937/+files/dmesg_xmon_boslcp3.txt ** Attachment removed: "dmesg logs from xmon prompt_boslcp3" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127820/+files/dmesg_xmon_boslcp3.txt ** Attachment removed: "dmesg logs from xmon prompt_boslcp3" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5125480/+files/dmesg_xmon_boslcp3.txt ** Attachment removed: "dmesg logs from xmon prompt_boslcp3" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5124758/+files/dmesg_xmon_boslcp3.txt ** Attachment removed: "dmesg logs from xmon
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
** Attachment removed: "boslcp3 crashed_used X from xmon prompt" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134460/+files/boslcp3_usedX_xmonprompt.txt ** Attachment removed: "boslcp3 crashed_used X from xmon prompt" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5133953/+files/boslcp3_usedX_xmonprompt.txt ** Attachment removed: "boslcp3 crashed_used X from xmon prompt" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132690/+files/boslcp3_usedX_xmonprompt.txt ** Attachment removed: "boslcp3 crashed_used X from xmon prompt" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132672/+files/boslcp3_usedX_xmonprompt.txt ** Attachment removed: "boslcp3 crashed_used X from xmon prompt" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132342/+files/boslcp3_usedX_xmonprompt.txt ** Attachment removed: "boslcp3 crashed_used X from xmon prompt" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127944/+files/boslcp3_usedX_xmonprompt.txt ** Attachment removed: "boslcp3 crashed_used X from xmon prompt" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127892/+files/boslcp3_usedX_xmonprompt.txt ** Attachment removed: "boslcp3 crashed_used X from xmon prompt" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127828/+files/boslcp3_usedX_xmonprompt.txt ** Attachment removed: "boslcp3 crashed_used X from xmon prompt" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127713/+files/boslcp3_usedX_xmonprompt.txt ** Attachment removed: "boslcp3g4 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134459/+files/boslcp3g4.dumpxml ** Attachment removed: "boslcp3g4 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132341/+files/boslcp3g4.dumpxml ** Attachment removed: "boslcp3g4 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5133952/+files/boslcp3g4.dumpxml ** Attachment removed: "boslcp3g4 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132689/+files/boslcp3g4.dumpxml ** Attachment removed: "boslcp3g4 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132671/+files/boslcp3g4.dumpxml ** Attachment removed: "boslcp3g4 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127943/+files/boslcp3g4.dumpxml ** Attachment removed: "boslcp3g4 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127891/+files/boslcp3g4.dumpxml ** Attachment removed: "boslcp3g4 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127827/+files/boslcp3g4.dumpxml ** Attachment removed: "boslcp3g4 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127712/+files/boslcp3g4.dumpxml ** Attachment removed: "boslcp3g4 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5126845/+files/boslcp3g4.dumpxml ** Attachment removed: "boslcp3g4 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5126529/+files/boslcp3g4.dumpxml ** Attachment removed: "boslcp3g3 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134458/+files/boslcp3g3.dumpxml ** Attachment removed: "boslcp3g3 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5133951/+files/boslcp3g3.dumpxml ** Attachment removed: "boslcp3g3 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132688/+files/boslcp3g3.dumpxml ** Attachment removed: "boslcp3g3 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132670/+files/boslcp3g3.dumpxml ** Attachment removed: "boslcp3g3 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132340/+files/boslcp3g3.dumpxml ** Attachment removed: "boslcp3g3 xml" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127942/+files/boslcp3g3.dumpxml -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel Status in The Ubuntu-power-systems project: Incomplete Status in linux package in Ubuntu: Incomplete Status in linux source package in Bionic: Incomplete Bug description: Problem Description: === Host crashed & enters into xmon after updating to 4.15.0-15.16 kernel kernel. Steps to re-create: == 1. boslcp3 is up with BMC:118 & PNOR: 20180330 levels 2. Installed boslcp3 with latest kernel 4.15.0-13-generic 3.
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
** Attachment removed: "op-buid instructions for patched skiroot build" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134463/+files/file_166588.txt ** Attachment removed: "op-buid instructions for patched skiroot build" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5133956/+files/file_166588.txt ** Attachment removed: "op-buid instructions for patched skiroot build" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132693/+files/file_166588.txt ** Attachment removed: "op-buid instructions for patched skiroot build" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132675/+files/file_166588.txt ** Attachment removed: "op-buid instructions for patched skiroot build" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132349/+files/file_166588.txt ** Attachment removed: "op-buid instructions for patched skiroot build" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132345/+files/file_166588.txt ** Attachment removed: "op-buid instructions for patched skiroot build" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127947/+files/file_166588.txt ** Attachment removed: "op-buid instructions for patched skiroot build" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127895/+files/file_166588.txt ** Attachment removed: "qla2xxx version 10.00.00.04-k" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134462/+files/qla2xxx-10.00.00.04-k.tgz ** Attachment removed: "qla2xxx version 10.00.00.04-k" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5133955/+files/qla2xxx-10.00.00.04-k.tgz ** Attachment removed: "qla2xxx version 10.00.00.04-k" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132692/+files/qla2xxx-10.00.00.04-k.tgz ** Attachment removed: "qla2xxx version 10.00.00.04-k" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132674/+files/qla2xxx-10.00.00.04-k.tgz ** Attachment removed: "qla2xxx version 10.00.00.04-k" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132344/+files/qla2xxx-10.00.00.04-k.tgz ** Attachment removed: "qla2xxx version 10.00.00.04-k" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127946/+files/qla2xxx-10.00.00.04-k.tgz ** Attachment removed: "qla2xxx version 10.00.00.04-k" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127894/+files/qla2xxx-10.00.00.04-k.tgz ** Attachment removed: "qla2xxx version 10.00.00.04-k" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127830/+files/qla2xxx-10.00.00.04-k.tgz ** Attachment removed: "boslcp3_host reboots multiple times" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134461/+files/boslcp3_rebooting_muliptl_times_with_proper_kernel.txt ** Attachment removed: "boslcp3_host reboots multiple times" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5133954/+files/boslcp3_rebooting_muliptl_times_with_proper_kernel.txt ** Attachment removed: "boslcp3_host reboots multiple times" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132691/+files/boslcp3_rebooting_muliptl_times_with_proper_kernel.txt ** Attachment removed: "boslcp3_host reboots multiple times" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132673/+files/boslcp3_rebooting_muliptl_times_with_proper_kernel.txt ** Attachment removed: "boslcp3_host reboots multiple times" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132343/+files/boslcp3_rebooting_muliptl_times_with_proper_kernel.txt ** Attachment removed: "boslcp3_host reboots multiple times" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127945/+files/boslcp3_rebooting_muliptl_times_with_proper_kernel.txt ** Attachment removed: "boslcp3_host reboots multiple times" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127893/+files/boslcp3_rebooting_muliptl_times_with_proper_kernel.txt ** Attachment removed: "boslcp3_host reboots multiple times" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127829/+files/boslcp3_rebooting_muliptl_times_with_proper_kernel.txt ** Attachment removed: "boslcp3_host reboots multiple times" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5127714/+files/boslcp3_rebooting_muliptl_times_with_proper_kernel.txt -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
** Attachment removed: "dmesg log_boslcp3_latest for may1 run" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5135514/+files/dmesg_boslcp3_may3 ** Attachment removed: "dmesg log_boslcp3_latest for may1 run" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134466/+files/dmesg_boslcp3_may3 ** Attachment removed: "dmesg log_boslcp3_latest for may1 run" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134384/+files/dmesg_boslcp3_may3 ** Attachment removed: "dmesg log_boslcp3_latest for may1 run" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134081/+files/dmesg_boslcp3_may3 ** Attachment removed: "dmesg log_boslcp3_latest for may1 run" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134054/+files/dmesg_boslcp3_may3 ** Attachment removed: "dmesg log_boslcp3_latest for may1 run" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5133959/+files/dmesg_boslcp3_may3 ** Attachment removed: "dmesg log_boslcp3_latest for may1 run" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132801/+files/dmesg_boslcp3_may3 ** Attachment removed: "boslcp3 host console tee logs" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5135513/+files/boslcp3-host-may1.txt ** Attachment removed: "boslcp3 host console tee logs" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134465/+files/boslcp3-host-may1.txt ** Attachment removed: "boslcp3 host console tee logs" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134383/+files/boslcp3-host-may1.txt ** Attachment removed: "boslcp3 host console tee logs" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134080/+files/boslcp3-host-may1.txt ** Attachment removed: "boslcp3 host console tee logs" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5133958/+files/boslcp3-host-may1.txt ** Attachment removed: "boslcp3 host console tee logs" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132800/+files/boslcp3-host-may1.txt ** Attachment removed: "boslcp3 host console tee logs" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132695/+files/boslcp3-host-may1.txt ** Attachment removed: "dmesg log thus far, for May 1 run." https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5135512/+files/dmesg ** Attachment removed: "dmesg log thus far, for May 1 run." https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134464/+files/dmesg ** Attachment removed: "dmesg log thus far, for May 1 run." https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134378/+files/dmesg ** Attachment removed: "dmesg log thus far, for May 1 run." https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134079/+files/dmesg ** Attachment removed: "dmesg log thus far, for May 1 run." https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5133957/+files/dmesg ** Attachment removed: "dmesg log thus far, for May 1 run." https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132694/+files/dmesg ** Attachment removed: "dmesg log thus far, for May 1 run." https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132676/+files/dmesg ** Attachment removed: "dmesg log thus far, for May 1 run." https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132350/+files/dmesg -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel Status in The Ubuntu-power-systems project: Incomplete Status in linux package in Ubuntu: Incomplete Status in linux source package in Bionic: Incomplete Bug description: Problem Description: === Host crashed & enters into xmon after updating to 4.15.0-15.16 kernel kernel. Steps to re-create: == 1. boslcp3 is up with BMC:118 & PNOR: 20180330 levels 2. Installed boslcp3 with latest kernel 4.15.0-13-generic 3. Enabled "-proposed" kernel in /etc/apt/sources.list file 4. Ran sudo apt-get update & apt-get upgrade 5. root@boslcp3:~# ls /boot abi-4.15.0-13-generic retpoline-4.15.0-13-generic abi-4.15.0-15-generic retpoline-4.15.0-15-generic config-4.15.0-13-generic System.map-4.15.0-13-generic config-4.15.0-15-generic System.map-4.15.0-15-generic grub vmlinux initrd.imgvmlinux-4.15.0-13-generic initrd.img-4.15.0-13-generic vmlinux-4.15.0-15-generic
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
** Attachment removed: "fuller version of previous boslcp6 log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5135531/+files/kern-boslcp6-before-and-after-bz166588-patch-fuller.log ** Attachment removed: "Logs from crashes after SAN bring-up of boslcp6 and subsequent logs of success boot after install of bz166588 patch" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5135530/+files/kern-boslcp6-before-and-after-bz166588-patch.log ** Attachment removed: "fuller version of previous boslcp6 log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5135527/+files/kern-boslcp6-before-and-after-bz166588-patch-fuller.log ** Attachment removed: "Logs from crashes after SAN bring-up of boslcp6 and subsequent logs of success boot after install of bz166588 patch" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5135526/+files/kern-boslcp6-before-and-after-bz166588-patch.log ** Attachment removed: "boslcp3_host_reboot_consolelogs" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5135525/+files/boslcp3_reboot_console_logs.txt ** Attachment removed: "boslcp3_host_reboot_consolelogs" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134496/+files/boslcp3_reboot_console_logs.txt ** Attachment removed: "boslcp3_host_reboot_consolelogs" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134474/+files/boslcp3_reboot_console_logs.txt ** Attachment removed: "boslcp3_host_reboot_consolelogs" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134387/+files/boslcp3_reboot_console_logs.txt ** Attachment removed: "boslcp3_host_reboot_consolelogs" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134085/+files/boslcp3_reboot_console_logs.txt ** Attachment removed: "boslcp3_host_reboot_consolelogs" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134057/+files/boslcp3_reboot_console_logs.txt ** Attachment removed: "boslcp3_host_reboot_consolelogs" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5133969/+files/boslcp3_reboot_console_logs.txt ** Attachment removed: "/var/log/syslog1.file boslcp3" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5135524/+files/syslog.1 ** Attachment removed: "/var/log/syslog1.file boslcp3" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134495/+files/syslog.1 ** Attachment removed: "/var/log/syslog1.file boslcp3" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134473/+files/syslog.1 ** Attachment removed: "/var/log/syslog1.file boslcp3" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134386/+files/syslog.1 ** Attachment removed: "/var/log/syslog1.file boslcp3" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134084/+files/syslog.1 ** Attachment removed: "/var/log/syslog1.file boslcp3" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134056/+files/syslog.1 ** Attachment removed: "/var/log/syslog1.file boslcp3" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132814/+files/syslog.1 ** Attachment removed: "/var/log/syslog boslcp3 host" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5135523/+files/syslog ** Attachment removed: "/var/log/syslog boslcp3 host" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134472/+files/syslog ** Attachment removed: "/var/log/syslog boslcp3 host" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134385/+files/syslog ** Attachment removed: "/var/log/syslog boslcp3 host" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134082/+files/syslog ** Attachment removed: "/var/log/syslog boslcp3 host" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134055/+files/syslog ** Attachment removed: "/var/log/syslog boslcp3 host" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5133960/+files/syslog ** Attachment removed: "/var/log/syslog boslcp3 host" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5132802/+files/syslog ** Attachment removed: "fuller version of previous boslcp6 log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5135440/+files/kern-boslcp6-before-and-after-bz166588-patch-fuller.log ** Attachment removed: "fuller version of previous boslcp6 log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1762844/+attachment/5134610/+files/kern-boslcp6-before-and-after-bz166588-patch-fuller.log ** Attachment removed: "Logs from crashes after SAN bring-up of boslcp6 and subsequent logs of success boot after install of
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
--- Comment From indira.pr...@in.ibm.com 2018-05-02 23:21 EDT--- Attached host boslcp3 host console tee logs. ** Tags added: bugnameltc-166588 severity-critical -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel Status in The Ubuntu-power-systems project: Incomplete Status in linux package in Ubuntu: Incomplete Status in linux source package in Bionic: Incomplete Bug description: Problem Description: === Host crashed & enters into xmon after updating to 4.15.0-15.16 kernel kernel. Steps to re-create: == 1. boslcp3 is up with BMC:118 & PNOR: 20180330 levels 2. Installed boslcp3 with latest kernel 4.15.0-13-generic 3. Enabled "-proposed" kernel in /etc/apt/sources.list file 4. Ran sudo apt-get update & apt-get upgrade 5. root@boslcp3:~# ls /boot abi-4.15.0-13-generic retpoline-4.15.0-13-generic abi-4.15.0-15-generic retpoline-4.15.0-15-generic config-4.15.0-13-generic System.map-4.15.0-13-generic config-4.15.0-15-generic System.map-4.15.0-15-generic grub vmlinux initrd.imgvmlinux-4.15.0-13-generic initrd.img-4.15.0-13-generic vmlinux-4.15.0-15-generic initrd.img-4.15.0-15-generic vmlinux.old initrd.img.old 6. Rebooted & booted with 4.15.0-15 kernel 7. Enabled xmon by editing file "vi /etc/default/grub" and ran update-grub 8. Rebooted host. 9. Booted with 4.15.0-15 & provided root/password credentials in login prompt 10. Host crashed & enters into XMON state with 'Unable to handle kernel paging request' root@boslcp3:~# [ 66.295233] Unable to handle kernel paging request for data at address 0x8882f6ed90e9151a [ 66.295297] Faulting instruction address: 0xc038a110 cpu 0x50: Vector: 380 (Data Access Out of Range) at [c692f650] pc: c038a110: kmem_cache_alloc_node+0x2f0/0x350 lr: c038a0fc: kmem_cache_alloc_node+0x2dc/0x350 sp: c692f8d0 msr: 90009033 dar: 8882f6ed90e9151a current = 0xc698fd00 paca= 0xcfab7000 softe: 0irq_happened: 0x01 pid = 1762, comm = systemd-journal Linux version 4.15.0-15-generic (buildd@bos02-ppc64el-002) (gcc version 7.3.0 (Ubuntu 7.3.0-14ubuntu1)) #16-Ubuntu SMP Wed Apr 4 13:57:51 UTC 2018 (Ubuntu 4.15.0-15.16-generic 4.15.15) enter ? for help [c692f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c692f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c692f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c692fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c692fae0] c0c5705c unix_dgram_sendmsg+0x15c/0x8f0 [c692fbc0] c0b1ec64 sock_sendmsg+0x64/0x90 [c692fbf0] c0b20abc ___sys_sendmsg+0x31c/0x390 [c692fd90] c0b221ec __sys_sendmsg+0x5c/0xc0 [c692fe30] c000b184 system_call+0x58/0x6c --- Exception: c00 (System Call) at 74826f6fa9c4 SP (75dc5510) is in userspace 50:mon> 50:mon> 10. Attached Host console logs I rebooted the host just to see if it would hit the issue again and this time I didn't even get to the login prompt but it crashed in the same location: 50:mon> r R00 = c0389fd4 R16 = c000200e0b20fdc0 R01 = c000200e0b20f8d0 R17 = 0048 R02 = c16eb400 R18 = 0001fe80 R03 = 0001 R19 = R04 = 0048ca1cff37803d R20 = R05 = 0688 R21 = R06 = 0001 R22 = 0048 R07 = 0687 R23 = 4882d6e3c8b7ab55 R08 = 48ca1cff37802b68 R24 = c000200e5851df01 R09 = R25 = 8882f6ed90e67454 R10 = R26 = c0b2ec6c R11 = c0d10f78 R27 = c00ff901ee00 R12 = 2000 R28 = R13 = cfab7000 R29 = 015004c0 R14 = c000200e4c973fc8 R30 = c000200e5851df01 R15 = c000200e4c974238 R31 = c00ff901ee00 pc = c038a110 kmem_cache_alloc_node+0x2f0/0x350 cfar= c0016e1c arch_local_irq_restore+0x1c/0x90 lr = c038a0fc kmem_cache_alloc_node+0x2dc/0x350 msr = 90009033 cr = 28002844 ctr = c061e1b0 xer = trap = 380 dar = 8882f6ed90e67454 dsisr = c000200e40bd8400 50:mon> t [c000200e0b20f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c000200e0b20f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c000200e0b20f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c000200e0b20fa30] c0b247cc
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
--- Comment From kla...@br.ibm.com 2018-05-02 16:32 EDT--- I think next steps here are: 1) apply all the known firmware workarounds (GH 1158) 2) Bring up system with Doug's recommendations for log verbosity (comment 211 and 215). Also capture the console output to a separate file if possible. 3) re-start the test using this same kernel, but with no stress on the host: proceed to restart the 3 guests with stress, and have a 4th guest migrating between boslcp3 and 4. --- Comment From dougm...@us.ibm.com 2018-05-02 16:36 EDT--- (In reply to comment #218) > I think next steps here are: > > 1) apply all the known firmware workarounds (GH 1158) > 2) Bring up system with Doug's recommendations for log verbosity (comment > 211 and 215). Also capture the console output to a separate file if possible. > 3) re-start the test using this same kernel, but with no stress on the host: > proceed to restart the 3 guests with stress, and have a 4th guest migrating > between boslcp3 and 4. Klaus, let's hold off on making more changes right now. I'd like to let things run as-is a little longer. ** Tags removed: bugnameltc-166588 kernel-key severity-critical triage-g -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel Status in The Ubuntu-power-systems project: Incomplete Status in linux package in Ubuntu: Incomplete Status in linux source package in Bionic: Incomplete Bug description: Problem Description: === Host crashed & enters into xmon after updating to 4.15.0-15.16 kernel kernel. Steps to re-create: == 1. boslcp3 is up with BMC:118 & PNOR: 20180330 levels 2. Installed boslcp3 with latest kernel 4.15.0-13-generic 3. Enabled "-proposed" kernel in /etc/apt/sources.list file 4. Ran sudo apt-get update & apt-get upgrade 5. root@boslcp3:~# ls /boot abi-4.15.0-13-generic retpoline-4.15.0-13-generic abi-4.15.0-15-generic retpoline-4.15.0-15-generic config-4.15.0-13-generic System.map-4.15.0-13-generic config-4.15.0-15-generic System.map-4.15.0-15-generic grub vmlinux initrd.imgvmlinux-4.15.0-13-generic initrd.img-4.15.0-13-generic vmlinux-4.15.0-15-generic initrd.img-4.15.0-15-generic vmlinux.old initrd.img.old 6. Rebooted & booted with 4.15.0-15 kernel 7. Enabled xmon by editing file "vi /etc/default/grub" and ran update-grub 8. Rebooted host. 9. Booted with 4.15.0-15 & provided root/password credentials in login prompt 10. Host crashed & enters into XMON state with 'Unable to handle kernel paging request' root@boslcp3:~# [ 66.295233] Unable to handle kernel paging request for data at address 0x8882f6ed90e9151a [ 66.295297] Faulting instruction address: 0xc038a110 cpu 0x50: Vector: 380 (Data Access Out of Range) at [c692f650] pc: c038a110: kmem_cache_alloc_node+0x2f0/0x350 lr: c038a0fc: kmem_cache_alloc_node+0x2dc/0x350 sp: c692f8d0 msr: 90009033 dar: 8882f6ed90e9151a current = 0xc698fd00 paca= 0xcfab7000 softe: 0irq_happened: 0x01 pid = 1762, comm = systemd-journal Linux version 4.15.0-15-generic (buildd@bos02-ppc64el-002) (gcc version 7.3.0 (Ubuntu 7.3.0-14ubuntu1)) #16-Ubuntu SMP Wed Apr 4 13:57:51 UTC 2018 (Ubuntu 4.15.0-15.16-generic 4.15.15) enter ? for help [c692f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c692f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c692f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c692fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c692fae0] c0c5705c unix_dgram_sendmsg+0x15c/0x8f0 [c692fbc0] c0b1ec64 sock_sendmsg+0x64/0x90 [c692fbf0] c0b20abc ___sys_sendmsg+0x31c/0x390 [c692fd90] c0b221ec __sys_sendmsg+0x5c/0xc0 [c692fe30] c000b184 system_call+0x58/0x6c --- Exception: c00 (System Call) at 74826f6fa9c4 SP (75dc5510) is in userspace 50:mon> 50:mon> 10. Attached Host console logs I rebooted the host just to see if it would hit the issue again and this time I didn't even get to the login prompt but it crashed in the same location: 50:mon> r R00 = c0389fd4 R16 = c000200e0b20fdc0 R01 = c000200e0b20f8d0 R17 = 0048 R02 = c16eb400 R18 = 0001fe80 R03 = 0001 R19 = R04 = 0048ca1cff37803d R20 = R05 = 0688 R21 = R06 = 0001 R22 = 0048 R07 = 0687 R23 =
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
Marking as "incomplete" while the identification of a patch is in progress. ** Changed in: ubuntu-power-systems Status: Triaged => Incomplete ** Changed in: linux (Ubuntu Bionic) Status: Triaged => Incomplete -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel Status in The Ubuntu-power-systems project: Incomplete Status in linux package in Ubuntu: Incomplete Status in linux source package in Bionic: Incomplete Bug description: Problem Description: === Host crashed & enters into xmon after updating to 4.15.0-15.16 kernel kernel. Steps to re-create: == 1. boslcp3 is up with BMC:118 & PNOR: 20180330 levels 2. Installed boslcp3 with latest kernel 4.15.0-13-generic 3. Enabled "-proposed" kernel in /etc/apt/sources.list file 4. Ran sudo apt-get update & apt-get upgrade 5. root@boslcp3:~# ls /boot abi-4.15.0-13-generic retpoline-4.15.0-13-generic abi-4.15.0-15-generic retpoline-4.15.0-15-generic config-4.15.0-13-generic System.map-4.15.0-13-generic config-4.15.0-15-generic System.map-4.15.0-15-generic grub vmlinux initrd.imgvmlinux-4.15.0-13-generic initrd.img-4.15.0-13-generic vmlinux-4.15.0-15-generic initrd.img-4.15.0-15-generic vmlinux.old initrd.img.old 6. Rebooted & booted with 4.15.0-15 kernel 7. Enabled xmon by editing file "vi /etc/default/grub" and ran update-grub 8. Rebooted host. 9. Booted with 4.15.0-15 & provided root/password credentials in login prompt 10. Host crashed & enters into XMON state with 'Unable to handle kernel paging request' root@boslcp3:~# [ 66.295233] Unable to handle kernel paging request for data at address 0x8882f6ed90e9151a [ 66.295297] Faulting instruction address: 0xc038a110 cpu 0x50: Vector: 380 (Data Access Out of Range) at [c692f650] pc: c038a110: kmem_cache_alloc_node+0x2f0/0x350 lr: c038a0fc: kmem_cache_alloc_node+0x2dc/0x350 sp: c692f8d0 msr: 90009033 dar: 8882f6ed90e9151a current = 0xc698fd00 paca= 0xcfab7000 softe: 0irq_happened: 0x01 pid = 1762, comm = systemd-journal Linux version 4.15.0-15-generic (buildd@bos02-ppc64el-002) (gcc version 7.3.0 (Ubuntu 7.3.0-14ubuntu1)) #16-Ubuntu SMP Wed Apr 4 13:57:51 UTC 2018 (Ubuntu 4.15.0-15.16-generic 4.15.15) enter ? for help [c692f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c692f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c692f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c692fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c692fae0] c0c5705c unix_dgram_sendmsg+0x15c/0x8f0 [c692fbc0] c0b1ec64 sock_sendmsg+0x64/0x90 [c692fbf0] c0b20abc ___sys_sendmsg+0x31c/0x390 [c692fd90] c0b221ec __sys_sendmsg+0x5c/0xc0 [c692fe30] c000b184 system_call+0x58/0x6c --- Exception: c00 (System Call) at 74826f6fa9c4 SP (75dc5510) is in userspace 50:mon> 50:mon> 10. Attached Host console logs I rebooted the host just to see if it would hit the issue again and this time I didn't even get to the login prompt but it crashed in the same location: 50:mon> r R00 = c0389fd4 R16 = c000200e0b20fdc0 R01 = c000200e0b20f8d0 R17 = 0048 R02 = c16eb400 R18 = 0001fe80 R03 = 0001 R19 = R04 = 0048ca1cff37803d R20 = R05 = 0688 R21 = R06 = 0001 R22 = 0048 R07 = 0687 R23 = 4882d6e3c8b7ab55 R08 = 48ca1cff37802b68 R24 = c000200e5851df01 R09 = R25 = 8882f6ed90e67454 R10 = R26 = c0b2ec6c R11 = c0d10f78 R27 = c00ff901ee00 R12 = 2000 R28 = R13 = cfab7000 R29 = 015004c0 R14 = c000200e4c973fc8 R30 = c000200e5851df01 R15 = c000200e4c974238 R31 = c00ff901ee00 pc = c038a110 kmem_cache_alloc_node+0x2f0/0x350 cfar= c0016e1c arch_local_irq_restore+0x1c/0x90 lr = c038a0fc kmem_cache_alloc_node+0x2dc/0x350 msr = 90009033 cr = 28002844 ctr = c061e1b0 xer = trap = 380 dar = 8882f6ed90e67454 dsisr = c000200e40bd8400 50:mon> t [c000200e0b20f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c000200e0b20f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c000200e0b20f9a0] c0b30b6c
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
** Changed in: linux (Ubuntu Bionic) Assignee: Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) => Canonical Kernel Team (canonical-kernel-team) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel Status in The Ubuntu-power-systems project: Triaged Status in linux package in Ubuntu: Triaged Status in linux source package in Bionic: Triaged Bug description: Problem Description: === Host crashed & enters into xmon after updating to 4.15.0-15.16 kernel kernel. Steps to re-create: == 1. boslcp3 is up with BMC:118 & PNOR: 20180330 levels 2. Installed boslcp3 with latest kernel 4.15.0-13-generic 3. Enabled "-proposed" kernel in /etc/apt/sources.list file 4. Ran sudo apt-get update & apt-get upgrade 5. root@boslcp3:~# ls /boot abi-4.15.0-13-generic retpoline-4.15.0-13-generic abi-4.15.0-15-generic retpoline-4.15.0-15-generic config-4.15.0-13-generic System.map-4.15.0-13-generic config-4.15.0-15-generic System.map-4.15.0-15-generic grub vmlinux initrd.imgvmlinux-4.15.0-13-generic initrd.img-4.15.0-13-generic vmlinux-4.15.0-15-generic initrd.img-4.15.0-15-generic vmlinux.old initrd.img.old 6. Rebooted & booted with 4.15.0-15 kernel 7. Enabled xmon by editing file "vi /etc/default/grub" and ran update-grub 8. Rebooted host. 9. Booted with 4.15.0-15 & provided root/password credentials in login prompt 10. Host crashed & enters into XMON state with 'Unable to handle kernel paging request' root@boslcp3:~# [ 66.295233] Unable to handle kernel paging request for data at address 0x8882f6ed90e9151a [ 66.295297] Faulting instruction address: 0xc038a110 cpu 0x50: Vector: 380 (Data Access Out of Range) at [c692f650] pc: c038a110: kmem_cache_alloc_node+0x2f0/0x350 lr: c038a0fc: kmem_cache_alloc_node+0x2dc/0x350 sp: c692f8d0 msr: 90009033 dar: 8882f6ed90e9151a current = 0xc698fd00 paca= 0xcfab7000 softe: 0irq_happened: 0x01 pid = 1762, comm = systemd-journal Linux version 4.15.0-15-generic (buildd@bos02-ppc64el-002) (gcc version 7.3.0 (Ubuntu 7.3.0-14ubuntu1)) #16-Ubuntu SMP Wed Apr 4 13:57:51 UTC 2018 (Ubuntu 4.15.0-15.16-generic 4.15.15) enter ? for help [c692f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c692f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c692f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c692fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c692fae0] c0c5705c unix_dgram_sendmsg+0x15c/0x8f0 [c692fbc0] c0b1ec64 sock_sendmsg+0x64/0x90 [c692fbf0] c0b20abc ___sys_sendmsg+0x31c/0x390 [c692fd90] c0b221ec __sys_sendmsg+0x5c/0xc0 [c692fe30] c000b184 system_call+0x58/0x6c --- Exception: c00 (System Call) at 74826f6fa9c4 SP (75dc5510) is in userspace 50:mon> 50:mon> 10. Attached Host console logs I rebooted the host just to see if it would hit the issue again and this time I didn't even get to the login prompt but it crashed in the same location: 50:mon> r R00 = c0389fd4 R16 = c000200e0b20fdc0 R01 = c000200e0b20f8d0 R17 = 0048 R02 = c16eb400 R18 = 0001fe80 R03 = 0001 R19 = R04 = 0048ca1cff37803d R20 = R05 = 0688 R21 = R06 = 0001 R22 = 0048 R07 = 0687 R23 = 4882d6e3c8b7ab55 R08 = 48ca1cff37802b68 R24 = c000200e5851df01 R09 = R25 = 8882f6ed90e67454 R10 = R26 = c0b2ec6c R11 = c0d10f78 R27 = c00ff901ee00 R12 = 2000 R28 = R13 = cfab7000 R29 = 015004c0 R14 = c000200e4c973fc8 R30 = c000200e5851df01 R15 = c000200e4c974238 R31 = c00ff901ee00 pc = c038a110 kmem_cache_alloc_node+0x2f0/0x350 cfar= c0016e1c arch_local_irq_restore+0x1c/0x90 lr = c038a0fc kmem_cache_alloc_node+0x2dc/0x350 msr = 90009033 cr = 28002844 ctr = c061e1b0 xer = trap = 380 dar = 8882f6ed90e67454 dsisr = c000200e40bd8400 50:mon> t [c000200e0b20f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c000200e0b20f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c000200e0b20f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c000200e0b20fa30] c0b247cc
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
Can you see if the bug happens with and of these mainline kernels? We can perform a kernel bisect if we can narrow down to the last good kernel version and first bad one: v4.14 Final: http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.14/ v4.15-rc1: http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.15-rc1/ v4.15-rc4: http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.15-rc4/ v4.15 Final: http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.15/ You don't have to test every kernel, just up until the kernel that first has this bug. Thanks in advance! ** Changed in: linux (Ubuntu) Importance: Undecided => Critical ** Changed in: linux (Ubuntu) Status: New => Triaged ** Also affects: linux (Ubuntu Bionic) Importance: Critical Assignee: Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) Status: Triaged ** Tags added: kernel-key -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel Status in The Ubuntu-power-systems project: Triaged Status in linux package in Ubuntu: Triaged Status in linux source package in Bionic: Triaged Bug description: Problem Description: === Host crashed & enters into xmon after updating to 4.15.0-15.16 kernel kernel. Steps to re-create: == 1. boslcp3 is up with BMC:118 & PNOR: 20180330 levels 2. Installed boslcp3 with latest kernel 4.15.0-13-generic 3. Enabled "-proposed" kernel in /etc/apt/sources.list file 4. Ran sudo apt-get update & apt-get upgrade 5. root@boslcp3:~# ls /boot abi-4.15.0-13-generic retpoline-4.15.0-13-generic abi-4.15.0-15-generic retpoline-4.15.0-15-generic config-4.15.0-13-generic System.map-4.15.0-13-generic config-4.15.0-15-generic System.map-4.15.0-15-generic grub vmlinux initrd.imgvmlinux-4.15.0-13-generic initrd.img-4.15.0-13-generic vmlinux-4.15.0-15-generic initrd.img-4.15.0-15-generic vmlinux.old initrd.img.old 6. Rebooted & booted with 4.15.0-15 kernel 7. Enabled xmon by editing file "vi /etc/default/grub" and ran update-grub 8. Rebooted host. 9. Booted with 4.15.0-15 & provided root/password credentials in login prompt 10. Host crashed & enters into XMON state with 'Unable to handle kernel paging request' root@boslcp3:~# [ 66.295233] Unable to handle kernel paging request for data at address 0x8882f6ed90e9151a [ 66.295297] Faulting instruction address: 0xc038a110 cpu 0x50: Vector: 380 (Data Access Out of Range) at [c692f650] pc: c038a110: kmem_cache_alloc_node+0x2f0/0x350 lr: c038a0fc: kmem_cache_alloc_node+0x2dc/0x350 sp: c692f8d0 msr: 90009033 dar: 8882f6ed90e9151a current = 0xc698fd00 paca= 0xcfab7000 softe: 0irq_happened: 0x01 pid = 1762, comm = systemd-journal Linux version 4.15.0-15-generic (buildd@bos02-ppc64el-002) (gcc version 7.3.0 (Ubuntu 7.3.0-14ubuntu1)) #16-Ubuntu SMP Wed Apr 4 13:57:51 UTC 2018 (Ubuntu 4.15.0-15.16-generic 4.15.15) enter ? for help [c692f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c692f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c692f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c692fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c692fae0] c0c5705c unix_dgram_sendmsg+0x15c/0x8f0 [c692fbc0] c0b1ec64 sock_sendmsg+0x64/0x90 [c692fbf0] c0b20abc ___sys_sendmsg+0x31c/0x390 [c692fd90] c0b221ec __sys_sendmsg+0x5c/0xc0 [c692fe30] c000b184 system_call+0x58/0x6c --- Exception: c00 (System Call) at 74826f6fa9c4 SP (75dc5510) is in userspace 50:mon> 50:mon> 10. Attached Host console logs I rebooted the host just to see if it would hit the issue again and this time I didn't even get to the login prompt but it crashed in the same location: 50:mon> r R00 = c0389fd4 R16 = c000200e0b20fdc0 R01 = c000200e0b20f8d0 R17 = 0048 R02 = c16eb400 R18 = 0001fe80 R03 = 0001 R19 = R04 = 0048ca1cff37803d R20 = R05 = 0688 R21 = R06 = 0001 R22 = 0048 R07 = 0687 R23 = 4882d6e3c8b7ab55 R08 = 48ca1cff37802b68 R24 = c000200e5851df01 R09 = R25 = 8882f6ed90e67454 R10 = R26 = c0b2ec6c R11 = c0d10f78 R27 = c00ff901ee00 R12 = 2000 R28 = R13 = cfab7000 R29 = 015004c0 R14 = c000200e4c973fc8
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
** Changed in: ubuntu-power-systems Status: Incomplete => Triaged -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel Status in The Ubuntu-power-systems project: Triaged Status in linux package in Ubuntu: New Bug description: Problem Description: === Host crashed & enters into xmon after updating to 4.15.0-15.16 kernel kernel. Steps to re-create: == 1. boslcp3 is up with BMC:118 & PNOR: 20180330 levels 2. Installed boslcp3 with latest kernel 4.15.0-13-generic 3. Enabled "-proposed" kernel in /etc/apt/sources.list file 4. Ran sudo apt-get update & apt-get upgrade 5. root@boslcp3:~# ls /boot abi-4.15.0-13-generic retpoline-4.15.0-13-generic abi-4.15.0-15-generic retpoline-4.15.0-15-generic config-4.15.0-13-generic System.map-4.15.0-13-generic config-4.15.0-15-generic System.map-4.15.0-15-generic grub vmlinux initrd.imgvmlinux-4.15.0-13-generic initrd.img-4.15.0-13-generic vmlinux-4.15.0-15-generic initrd.img-4.15.0-15-generic vmlinux.old initrd.img.old 6. Rebooted & booted with 4.15.0-15 kernel 7. Enabled xmon by editing file "vi /etc/default/grub" and ran update-grub 8. Rebooted host. 9. Booted with 4.15.0-15 & provided root/password credentials in login prompt 10. Host crashed & enters into XMON state with 'Unable to handle kernel paging request' root@boslcp3:~# [ 66.295233] Unable to handle kernel paging request for data at address 0x8882f6ed90e9151a [ 66.295297] Faulting instruction address: 0xc038a110 cpu 0x50: Vector: 380 (Data Access Out of Range) at [c692f650] pc: c038a110: kmem_cache_alloc_node+0x2f0/0x350 lr: c038a0fc: kmem_cache_alloc_node+0x2dc/0x350 sp: c692f8d0 msr: 90009033 dar: 8882f6ed90e9151a current = 0xc698fd00 paca= 0xcfab7000 softe: 0irq_happened: 0x01 pid = 1762, comm = systemd-journal Linux version 4.15.0-15-generic (buildd@bos02-ppc64el-002) (gcc version 7.3.0 (Ubuntu 7.3.0-14ubuntu1)) #16-Ubuntu SMP Wed Apr 4 13:57:51 UTC 2018 (Ubuntu 4.15.0-15.16-generic 4.15.15) enter ? for help [c692f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c692f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c692f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c692fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c692fae0] c0c5705c unix_dgram_sendmsg+0x15c/0x8f0 [c692fbc0] c0b1ec64 sock_sendmsg+0x64/0x90 [c692fbf0] c0b20abc ___sys_sendmsg+0x31c/0x390 [c692fd90] c0b221ec __sys_sendmsg+0x5c/0xc0 [c692fe30] c000b184 system_call+0x58/0x6c --- Exception: c00 (System Call) at 74826f6fa9c4 SP (75dc5510) is in userspace 50:mon> 50:mon> 10. Attached Host console logs I rebooted the host just to see if it would hit the issue again and this time I didn't even get to the login prompt but it crashed in the same location: 50:mon> r R00 = c0389fd4 R16 = c000200e0b20fdc0 R01 = c000200e0b20f8d0 R17 = 0048 R02 = c16eb400 R18 = 0001fe80 R03 = 0001 R19 = R04 = 0048ca1cff37803d R20 = R05 = 0688 R21 = R06 = 0001 R22 = 0048 R07 = 0687 R23 = 4882d6e3c8b7ab55 R08 = 48ca1cff37802b68 R24 = c000200e5851df01 R09 = R25 = 8882f6ed90e67454 R10 = R26 = c0b2ec6c R11 = c0d10f78 R27 = c00ff901ee00 R12 = 2000 R28 = R13 = cfab7000 R29 = 015004c0 R14 = c000200e4c973fc8 R30 = c000200e5851df01 R15 = c000200e4c974238 R31 = c00ff901ee00 pc = c038a110 kmem_cache_alloc_node+0x2f0/0x350 cfar= c0016e1c arch_local_irq_restore+0x1c/0x90 lr = c038a0fc kmem_cache_alloc_node+0x2dc/0x350 msr = 90009033 cr = 28002844 ctr = c061e1b0 xer = trap = 380 dar = 8882f6ed90e67454 dsisr = c000200e40bd8400 50:mon> t [c000200e0b20f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c000200e0b20f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c000200e0b20f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c000200e0b20fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c000200e0b20fae0] c0c56ae4 unix_stream_sendmsg+0x264/0x5c0 [c000200e0b20fbc0] c0b1ec64 sock_sendmsg+0x64/0x90
[Kernel-packages] [Bug 1762844] Re: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel
Can you test again on a third system? Can this be a hw problem on the first system? ** Also affects: ubuntu-power-systems Importance: Undecided Status: New ** Changed in: ubuntu-power-systems Status: New => Triaged ** Changed in: ubuntu-power-systems Importance: Undecided => Critical ** Changed in: ubuntu-power-systems Assignee: (unassigned) => Canonical Kernel Team (canonical-kernel-team) ** Tags added: triage-g ** Changed in: ubuntu-power-systems Status: Triaged => Incomplete -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1762844 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: Host crashed & enters into xmon after moving to 4.15.0-15.16 kernel Status in The Ubuntu-power-systems project: Incomplete Status in linux package in Ubuntu: New Bug description: Problem Description: === Host crashed & enters into xmon after updating to 4.15.0-15.16 kernel kernel. Steps to re-create: == 1. boslcp3 is up with BMC:118 & PNOR: 20180330 levels 2. Installed boslcp3 with latest kernel 4.15.0-13-generic 3. Enabled "-proposed" kernel in /etc/apt/sources.list file 4. Ran sudo apt-get update & apt-get upgrade 5. root@boslcp3:~# ls /boot abi-4.15.0-13-generic retpoline-4.15.0-13-generic abi-4.15.0-15-generic retpoline-4.15.0-15-generic config-4.15.0-13-generic System.map-4.15.0-13-generic config-4.15.0-15-generic System.map-4.15.0-15-generic grub vmlinux initrd.imgvmlinux-4.15.0-13-generic initrd.img-4.15.0-13-generic vmlinux-4.15.0-15-generic initrd.img-4.15.0-15-generic vmlinux.old initrd.img.old 6. Rebooted & booted with 4.15.0-15 kernel 7. Enabled xmon by editing file "vi /etc/default/grub" and ran update-grub 8. Rebooted host. 9. Booted with 4.15.0-15 & provided root/password credentials in login prompt 10. Host crashed & enters into XMON state with 'Unable to handle kernel paging request' root@boslcp3:~# [ 66.295233] Unable to handle kernel paging request for data at address 0x8882f6ed90e9151a [ 66.295297] Faulting instruction address: 0xc038a110 cpu 0x50: Vector: 380 (Data Access Out of Range) at [c692f650] pc: c038a110: kmem_cache_alloc_node+0x2f0/0x350 lr: c038a0fc: kmem_cache_alloc_node+0x2dc/0x350 sp: c692f8d0 msr: 90009033 dar: 8882f6ed90e9151a current = 0xc698fd00 paca= 0xcfab7000 softe: 0irq_happened: 0x01 pid = 1762, comm = systemd-journal Linux version 4.15.0-15-generic (buildd@bos02-ppc64el-002) (gcc version 7.3.0 (Ubuntu 7.3.0-14ubuntu1)) #16-Ubuntu SMP Wed Apr 4 13:57:51 UTC 2018 (Ubuntu 4.15.0-15.16-generic 4.15.15) enter ? for help [c692f8d0] c0389fd4 kmem_cache_alloc_node+0x1b4/0x350 (unreliable) [c692f940] c0b2ec6c __alloc_skb+0x6c/0x220 [c692f9a0] c0b30b6c alloc_skb_with_frags+0x7c/0x2e0 [c692fa30] c0b247cc sock_alloc_send_pskb+0x29c/0x2c0 [c692fae0] c0c5705c unix_dgram_sendmsg+0x15c/0x8f0 [c692fbc0] c0b1ec64 sock_sendmsg+0x64/0x90 [c692fbf0] c0b20abc ___sys_sendmsg+0x31c/0x390 [c692fd90] c0b221ec __sys_sendmsg+0x5c/0xc0 [c692fe30] c000b184 system_call+0x58/0x6c --- Exception: c00 (System Call) at 74826f6fa9c4 SP (75dc5510) is in userspace 50:mon> 50:mon> 10. Attached Host console logs I rebooted the host just to see if it would hit the issue again and this time I didn't even get to the login prompt but it crashed in the same location: 50:mon> r R00 = c0389fd4 R16 = c000200e0b20fdc0 R01 = c000200e0b20f8d0 R17 = 0048 R02 = c16eb400 R18 = 0001fe80 R03 = 0001 R19 = R04 = 0048ca1cff37803d R20 = R05 = 0688 R21 = R06 = 0001 R22 = 0048 R07 = 0687 R23 = 4882d6e3c8b7ab55 R08 = 48ca1cff37802b68 R24 = c000200e5851df01 R09 = R25 = 8882f6ed90e67454 R10 = R26 = c0b2ec6c R11 = c0d10f78 R27 = c00ff901ee00 R12 = 2000 R28 = R13 = cfab7000 R29 = 015004c0 R14 = c000200e4c973fc8 R30 = c000200e5851df01 R15 = c000200e4c974238 R31 = c00ff901ee00 pc = c038a110 kmem_cache_alloc_node+0x2f0/0x350 cfar= c0016e1c arch_local_irq_restore+0x1c/0x90 lr = c038a0fc kmem_cache_alloc_node+0x2dc/0x350 msr = 90009033 cr = 28002844 ctr = c061e1b0 xer = trap = 380 dar = 8882f6ed90e67454 dsisr =
