[Kernel-packages] [Bug 1743529] Comment bridged from LTC Bugzilla
--- Comment From indira.pr...@in.ibm.com 2018-03-26 06:13 EDT--- Hi, Verified issue with latest ubunu1804 daily build kernel and now not seeing problem with triggering crash with below levels root@whip:~# dpkg -l | grep kexec-tools ii kexec-tools 1:2.0.16-1ubuntu1 ppc64el tools to support fast kexec reboots root@whip:~# dpkg -l | grep makedumpfile ii makedumpfile1:1.6.3-1 ppc64el VMcore extraction tool root@whip:~# uname -a Linux whip 4.15.0-12-generic #13 SMP Thu Mar 22 07:28:54 CDT 2018 ppc64le ppc64le ppc64le GNU/Linux Triggered crash: * root@whip:/etc/default/grub.d# echo c > /proc/sysrq-trigger [ 183.215596] sysrq: SysRq : This sysrq operation is disabled. root@whip:/etc/default/grub.d# echo 1 > /proc/sys/kernel/sysrq root@whip:/etc/default/grub.d# echo c > /proc/sysrq-trigger [ 210.082354] sysrq: SysRq : Trigger a crash [ 210.082396] Unable to handle kernel paging request for data at address 0x [ 210.082518] Faulting instruction address: 0xc07ec4e8 [ 210.082581] Oops: Kernel access of bad area, sig: 11 [#1] [ 210.082646] LE SMP NR_CPUS=2048 NUMA PowerNV [ 210.082713] Modules linked in: rpcsec_gss_krb5 nfsv4 nfs fscache rdma_ucm(OE) ib_ucm(OE) rdma_cm(OE) iw_cm(OE) ib_ipoib(OE) ib_cm(OE) ib_uverbs(OE) ib_umad(OE) esp6_offload esp6 esp4_offload esp4 xfrm_algo mlx5_fpga_tools(OE) mlx4_en(OE) mlx4_ib(OE) mlx4_core(OE) ofpart cmdlinepart vmx_crypto powernv_flash mtd idt_89hpesx crct10dif_vpmsum ipmi_powernv ipmi_devintf ipmi_msghandler at24 uio_pdrv_genirq uio opal_prd ibmpowernv binfmt_misc nfsd auth_rpcgss nfs_acl lockd grace sunrpc sch_fq_codel knem(OE) ip_tables x_tables autofs4 btrfs xor zstd_compress raid6_pq mlx5_ib(OE) ib_core(OE) mlx5_core(OE) nouveau mlxfw(OE) devlink mlx_compat(OE) lpfc ast i2c_algo_bit ttm drm_kms_helper nvmet_fc syscopyarea nvmet cxl sysfillrect sysimgblt nvme_fc fb_sys_fops ahci nvme_fabrics crc32c_vpmsum drm tg3 pnv_php [ 210.083672] libahci scsi_transport_fc [ 210.083722] CPU: 10 PID: 5235 Comm: bash Tainted: G OE 4.15.0-12-generic #13 [ 210.083792] NIP: c07ec4e8 LR: c07ed428 CTR: c07ec4c0 [ 210.083895] REGS: c07fb73279f0 TRAP: 0300 Tainted: G OE (4.15.0-12-generic) [ 210.084027] MSR: 90009033CR: 2822 XER: 2004 [ 210.084154] CFAR: c07ed424 DAR: DSISR: 4200 SOFTE: 1 [ 210.084154] GPR00: c07ed428 c07fb7327c70 c16eaf00 0063 [ 210.084154] GPR04: c07fdeb7ce18 c07fdeb94368 90009033 000a [ 210.084154] GPR08: 0007 0001 90001003 [ 210.084154] GPR12: c07ec4c0 c3266e00 0f1697af6b08 [ 210.084154] GPR16: 0f167ebce9f0 0f167ec61998 0f167ec619d0 0f167ec98204 [ 210.084154] GPR20: 0001 7fffc5069ac4 [ 210.084154] GPR24: 7fffc5069ac0 0f167ec9afc4 c15e9968 0002 [ 210.084154] GPR28: 0063 0007 c1572a9c c15e9d08 [ 210.085152] NIP [c07ec4e8] sysrq_handle_crash+0x28/0x30 [ 210.085269] LR [c07ed428] __handle_sysrq+0xf8/0x2c0 [ 210.085328] Call Trace: [ 210.085378] [c07fb7327c70] [c07ed408] __handle_sysrq+0xd8/0x2c0 (unreliable) [ 210.085482] [c07fb7327d10] [c07edc34] write_sysrq_trigger+0x64/0x90 [ 210.085584] [c07fb7327d40] [c047de88] proc_reg_write+0x88/0xd0 [ 210.085673] [c07fb7327d70] [c03d11bc] __vfs_write+0x3c/0x70 [ 210.085751] [c07fb7327d90] [c03d1418] vfs_write+0xd8/0x220 [ 210.085824] [c07fb7327de0] [c03d1738] SyS_write+0x68/0x110 [ 210.085941] [c07fb7327e30] [c000b184] system_call+0x58/0x6c [ 210.086030] Instruction dump: [ 210.086067] 4bfff9f1 4bfffe50 3c4c00f0 3842ea40 7c0802a6 6000 3921 3d42001c [ 210.086185] 394a6db0 912a 7c0004ac 3940 <992a> 4e800020 3c4c00f0 3842ea10 [ 210.086293] ---[ end trace 2141bc6e05b3cc02 ]--- [ 211.090273] 211.090393] Sending IPI to other CP[ 373.057331960,5] OPAL: Switch to big-endian OS Us [ 211.12[ 377.207676398,5] OPAL: Switch to little-endian OS 0361] IPI complete [ 213.393057] kexec: Starting switchover sequence. [1.295245] i ntegrity: Unable to open file: / etc/keys/x509_im a.der (-2) [1.295249] integrity: Unable to open file: /etc/keys/x509_evm.der (-2) [1.353447] vio vio: uevent: failed to send synthetic uevent [2.089461] nouveau 0004:04:00.0: unknown chipset (14a1) [2.131257] nouveau 0004:05:00.0: unknown chipset (14a1) [2.131538] nouveau 0035:03:00.0: unknown chipset (14a1) [2.131664] nouveau 0035:04:00.0: unknown
[Kernel-packages] [Bug 1743529] Comment bridged from LTC Bugzilla
--- Comment From indira.pr...@in.ibm.com 2018-02-13 03:49 EDT--- Any update on fix availability in official ubunu1804 build Regards, Indira -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1743529 Title: ISST-LTE:PowerNV:Ubuntu1804:Witherspoon:whip: System hung with Kernel panic -not syncing: Out of memory message when crash is triggered. Status in The Ubuntu-power-systems project: Triaged Status in linux package in Ubuntu: Triaged Bug description: == Comment: #0 - INDIRA P. JOGA Problem Description: === System hung with kernel panic Kernel panic - not syncing: Out of memory message when crash is triggered Steps to re-create: == > Installed ubuntu1804 daily build on Witherspoon test system root@whip:~# uname -a Linux whip 4.13.0-17-generic #20-Ubuntu SMP Mon Nov 6 10:03:08 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux root@whip:~# uname -r 4.13.0-17-generic > root@whip:~# free -h totalusedfree shared buff/cache available Mem: 507G2.0G504G 19M728M 503G Swap: 2.0G 0B2.0G > Edited the grub /etc/default/grub.d/kexec-tools.cfg file and set the crash kernel parameter=4096M > Updated grub using update-grub command and reboot system. cat root@whip:~# cat /proc/cmdline root=UUID=46c6aa02-8215-44cc-b3fc-0bc79c3c8815 ro splash quiet crashkernel=4096M > kdump status before triggering crash root@whip:~# kdump-config show DUMP_MODE:kdump USE_KDUMP:1 KDUMP_SYSCTL: kernel.panic_on_oops=1 KDUMP_COREDIR:/var/crash crashkernel addr: /var/lib/kdump/vmlinuz: symbolic link to /boot/vmlinux-4.13.0-17-generic kdump initrd: /var/lib/kdump/initrd.img: symbolic link to /var/lib/kdump/initrd.img-4.13.0-17-generic current state:ready to kdump kexec command: /sbin/kexec -p --command-line="root=UUID=46c6aa02-8215-44cc-b3fc-0bc79c3c8815 ro splash quiet irqpoll noirqdistrib nr_cpus=1 nousb systemd.unit=kdump-tools.service ata_piix.prefer_ms_hyperv=0" --initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz root@whip:~# kdump-config status current state : ready to kdump > Enabled sysrq root@whip:~# sysctl -w kernel.sysrq=1 kernel.sysrq = 1 > Triggered crash and it hangs with kernel panic- OOM message as below root@whip:~# echo c > /proc/sysrq-trigger [ 85.731415] sysrq: SysRq : Trigger a crash [ 85.731472] Unable to handle kernel paging request for data at address 0x [ 85.731584] Faulting instruction address: 0xc078f588 [ 85.731670] Oops: Kernel access of bad area, sig: 11 [#1] [ 85.731744] SMP NR_CPUS=2048 [ 85.731745] NUMA [ 85.731790] PowerNV [ 85.731853] Modules linked in: rpcsec_gss_krb5 nfsv4 nfs fscache sctp_diag sctp dccp_diag dccp tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag netlink_diag binfmt_misc vmx_crypto crct10dif_vpmsum ofpart cmdlinepart idt_89hpesx powernv_flash ipmi_powernv opal_prd ibmpowernv mtd ipmi_devintf ipmi_msghandler at24 uio_pdrv_genirq uio dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua nfsd auth_rpcgss sch_fq_codel nfs_acl lockd grace sunrpc ip_tables x_tables autofs4 btrfs xor raid6_pq nouveau bnx2x ast i2c_algo_bit ttm drm_kms_helper mdio libcrc32c crc32c_vpmsum mlx5_core syscopyarea sysfillrect sysimgblt fb_sys_fops tg3 drm ahci mlxfw libahci nvme devlink nvme_core [ 85.732704] CPU: 10 PID: 4316 Comm: bash Not tainted 4.13.0-17-generic #20-Ubuntu [ 85.732764] task: c03fcb141700 task.stack: c03fc2374000 [ 85.732858] NIP: c078f588 LR: c07904b8 CTR: c078f560 [ 85.732977] REGS: c03fc23779f0 TRAP: 0300 Not tainted (4.13.0-17-generic) [ 85.733066] MSR: 90009033[ 85.733075] CR: 2842 XER: 2004 [ 85.733201] CFAR: c07904b4 DAR: DSISR: 4200 SOFTE: 1 [ 85.733201] GPR00: c07904b8 c03fc2377c70 c15f6000 0063 [ 85.733201] GPR04: c03feedfade8 c03feee12068 90009033 000a [ 85.733201] GPR08: 0007 0001 90001003 [ 85.733201] GPR12: c078f560 c7a66900 10180df8 10189e30 [ 85.733201] GPR16: 10189ea8 10151210 1018bd58 1018de48 [ 85.733201] GPR20: 321168d8 0001 10164590 10163bb0 [ 85.733201] GPR24: 7fffcfa6e7d4 7fffcfa6e7d0 c14fa570 0002 [ 85.733201] GPR28: 0063 0004 c14822f4 c14fa910 [ 85.734116] NIP [c078f588] sysrq_handle_crash+0x28/0x30 [ 85.734211] LR
[Kernel-packages] [Bug 1743529] Comment bridged from LTC Bugzilla
--- Comment From indira.pr...@in.ibm.com 2018-01-31 03:30 EDT--- Hi Breno, Is fix available in official ubunu1804 build Regards, Indira --- Comment From indira.pr...@in.ibm.com 2018-01-31 03:31 EDT--- When can we expect the fix in official ubunu1804 build. Regards, Indira -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1743529 Title: ISST-LTE:PowerNV:Ubuntu1804:Witherspoon:whip: System hung with Kernel panic -not syncing: Out of memory message when crash is triggered. Status in The Ubuntu-power-systems project: Triaged Status in linux package in Ubuntu: Triaged Bug description: == Comment: #0 - INDIRA P. JOGA Problem Description: === System hung with kernel panic Kernel panic - not syncing: Out of memory message when crash is triggered Steps to re-create: == > Installed ubuntu1804 daily build on Witherspoon test system root@whip:~# uname -a Linux whip 4.13.0-17-generic #20-Ubuntu SMP Mon Nov 6 10:03:08 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux root@whip:~# uname -r 4.13.0-17-generic > root@whip:~# free -h totalusedfree shared buff/cache available Mem: 507G2.0G504G 19M728M 503G Swap: 2.0G 0B2.0G > Edited the grub /etc/default/grub.d/kexec-tools.cfg file and set the crash kernel parameter=4096M > Updated grub using update-grub command and reboot system. cat root@whip:~# cat /proc/cmdline root=UUID=46c6aa02-8215-44cc-b3fc-0bc79c3c8815 ro splash quiet crashkernel=4096M > kdump status before triggering crash root@whip:~# kdump-config show DUMP_MODE:kdump USE_KDUMP:1 KDUMP_SYSCTL: kernel.panic_on_oops=1 KDUMP_COREDIR:/var/crash crashkernel addr: /var/lib/kdump/vmlinuz: symbolic link to /boot/vmlinux-4.13.0-17-generic kdump initrd: /var/lib/kdump/initrd.img: symbolic link to /var/lib/kdump/initrd.img-4.13.0-17-generic current state:ready to kdump kexec command: /sbin/kexec -p --command-line="root=UUID=46c6aa02-8215-44cc-b3fc-0bc79c3c8815 ro splash quiet irqpoll noirqdistrib nr_cpus=1 nousb systemd.unit=kdump-tools.service ata_piix.prefer_ms_hyperv=0" --initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz root@whip:~# kdump-config status current state : ready to kdump > Enabled sysrq root@whip:~# sysctl -w kernel.sysrq=1 kernel.sysrq = 1 > Triggered crash and it hangs with kernel panic- OOM message as below root@whip:~# echo c > /proc/sysrq-trigger [ 85.731415] sysrq: SysRq : Trigger a crash [ 85.731472] Unable to handle kernel paging request for data at address 0x [ 85.731584] Faulting instruction address: 0xc078f588 [ 85.731670] Oops: Kernel access of bad area, sig: 11 [#1] [ 85.731744] SMP NR_CPUS=2048 [ 85.731745] NUMA [ 85.731790] PowerNV [ 85.731853] Modules linked in: rpcsec_gss_krb5 nfsv4 nfs fscache sctp_diag sctp dccp_diag dccp tcp_diag udp_diag raw_diag inet_diag unix_diag af_packet_diag netlink_diag binfmt_misc vmx_crypto crct10dif_vpmsum ofpart cmdlinepart idt_89hpesx powernv_flash ipmi_powernv opal_prd ibmpowernv mtd ipmi_devintf ipmi_msghandler at24 uio_pdrv_genirq uio dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua nfsd auth_rpcgss sch_fq_codel nfs_acl lockd grace sunrpc ip_tables x_tables autofs4 btrfs xor raid6_pq nouveau bnx2x ast i2c_algo_bit ttm drm_kms_helper mdio libcrc32c crc32c_vpmsum mlx5_core syscopyarea sysfillrect sysimgblt fb_sys_fops tg3 drm ahci mlxfw libahci nvme devlink nvme_core [ 85.732704] CPU: 10 PID: 4316 Comm: bash Not tainted 4.13.0-17-generic #20-Ubuntu [ 85.732764] task: c03fcb141700 task.stack: c03fc2374000 [ 85.732858] NIP: c078f588 LR: c07904b8 CTR: c078f560 [ 85.732977] REGS: c03fc23779f0 TRAP: 0300 Not tainted (4.13.0-17-generic) [ 85.733066] MSR: 90009033[ 85.733075] CR: 2842 XER: 2004 [ 85.733201] CFAR: c07904b4 DAR: DSISR: 4200 SOFTE: 1 [ 85.733201] GPR00: c07904b8 c03fc2377c70 c15f6000 0063 [ 85.733201] GPR04: c03feedfade8 c03feee12068 90009033 000a [ 85.733201] GPR08: 0007 0001 90001003 [ 85.733201] GPR12: c078f560 c7a66900 10180df8 10189e30 [ 85.733201] GPR16: 10189ea8 10151210 1018bd58 1018de48 [ 85.733201] GPR20: 321168d8 0001 10164590 10163bb0 [ 85.733201] GPR24: 7fffcfa6e7d4 7fffcfa6e7d0 c14fa570 0002 [ 85.733201] GPR28: 0063