[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
** Tags added: cscc -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in Linux: Fix Released Status in linux package in Ubuntu: Confirmed Status in linux source package in Xenial: Confirmed Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7 [31016.239899] RIP [] __xen_evtchn_do_upcall+0x43/0x80 [31016.239899] RSP [31016.239899] CR2: 82c4801e6bda [31016.239899] ---[ end trace 5b3e8ea32013e327 ]--- [31016.239899] Kernel panic - not syncing: Fatal exception in interrupt [31016.239899] Kernel Offset: disabled We believe this appeared in the last 1-2mo of releases, since this started happening after we did a `apt-get upgrade` on our machines after a bit of a pause. These are EC2 m4.xlarge servers running Ubuntu 16.04.1. They are Kubernetes minions, so I assume docker is likely the trigger. Unfortunately there aren't really any interesting logs in journalctl from the previous boot prior to the panic. Let me know what I can do to debug further. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-53-generic 4.4.0-53.74 ProcVersionSignature: Ubuntu 4.4.0-53.74-generic 4.4.30 Uname: Linux 4.4.0-53-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Dec 19 15:56 seq crw-rw 1 root audio 116, 33 Dec 19 15:56 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.1-0ubuntu2.4 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Mon Dec 19 18:12:12 2016 Ec2AMI:
[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
** Changed in: linux Status: New => Fix Released -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in Linux: Fix Released Status in linux package in Ubuntu: Confirmed Status in linux source package in Xenial: Confirmed Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7 [31016.239899] RIP [] __xen_evtchn_do_upcall+0x43/0x80 [31016.239899] RSP [31016.239899] CR2: 82c4801e6bda [31016.239899] ---[ end trace 5b3e8ea32013e327 ]--- [31016.239899] Kernel panic - not syncing: Fatal exception in interrupt [31016.239899] Kernel Offset: disabled We believe this appeared in the last 1-2mo of releases, since this started happening after we did a `apt-get upgrade` on our machines after a bit of a pause. These are EC2 m4.xlarge servers running Ubuntu 16.04.1. They are Kubernetes minions, so I assume docker is likely the trigger. Unfortunately there aren't really any interesting logs in journalctl from the previous boot prior to the panic. Let me know what I can do to debug further. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-53-generic 4.4.0-53.74 ProcVersionSignature: Ubuntu 4.4.0-53.74-generic 4.4.30 Uname: Linux 4.4.0-53-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Dec 19 15:56 seq crw-rw 1 root audio 116, 33 Dec 19 15:56 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.1-0ubuntu2.4 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date:
[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
No problems since installation of "4.4.42-040442-generic #201701120631". Everything is fine -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in Linux: New Status in linux package in Ubuntu: Confirmed Status in linux source package in Xenial: Confirmed Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7 [31016.239899] RIP [] __xen_evtchn_do_upcall+0x43/0x80 [31016.239899] RSP [31016.239899] CR2: 82c4801e6bda [31016.239899] ---[ end trace 5b3e8ea32013e327 ]--- [31016.239899] Kernel panic - not syncing: Fatal exception in interrupt [31016.239899] Kernel Offset: disabled We believe this appeared in the last 1-2mo of releases, since this started happening after we did a `apt-get upgrade` on our machines after a bit of a pause. These are EC2 m4.xlarge servers running Ubuntu 16.04.1. They are Kubernetes minions, so I assume docker is likely the trigger. Unfortunately there aren't really any interesting logs in journalctl from the previous boot prior to the panic. Let me know what I can do to debug further. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-53-generic 4.4.0-53.74 ProcVersionSignature: Ubuntu 4.4.0-53.74-generic 4.4.30 Uname: Linux 4.4.0-53-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Dec 19 15:56 seq crw-rw 1 root audio 116, 33 Dec 19 15:56 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.1-0ubuntu2.4 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit
[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
Same problem here. Server died frequently with kernel version >4.4.0-38, =4.13.0-45 and =4.15.0-23. After installing linux-image-4.4.42-040442-generic_4.4.42-040442.201701120631_amd64.deb the server seems to be stable. > docker01:~ $ uname -a > Linux docker01 4.4.42-040442-generic #201701120631 SMP Thu Jan 12 11:32:59 > UTC 2017 x86_64 x86_64 x86_64 GNU/Linux I'll let it run with this kernel and report if the problem occurs again. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in Linux: New Status in linux package in Ubuntu: Confirmed Status in linux source package in Xenial: Confirmed Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7 [31016.239899] RIP [] __xen_evtchn_do_upcall+0x43/0x80 [31016.239899] RSP [31016.239899] CR2: 82c4801e6bda [31016.239899] ---[ end trace 5b3e8ea32013e327 ]--- [31016.239899] Kernel panic - not syncing: Fatal exception in interrupt [31016.239899] Kernel Offset: disabled We believe this appeared in the last 1-2mo of releases, since this started happening after we did a `apt-get upgrade` on our machines after a bit of a pause. These are EC2 m4.xlarge servers running Ubuntu 16.04.1. They are Kubernetes minions, so I assume docker is likely the trigger. Unfortunately there aren't really any interesting logs in journalctl from the previous boot prior to the panic. Let me know what I can do to debug further. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-53-generic 4.4.0-53.74 ProcVersionSignature: Ubuntu 4.4.0-53.74-generic 4.4.30 Uname: Linux 4.4.0-53-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Dec 19 15:56 seq
[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
Has this been fixed in any of the newer linux-image packages? I, unfortunately, can't offer to test since I was only seeing this on my production database server... For now I've just frozen my kernel version to 4.4.0-38. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in Linux: New Status in linux package in Ubuntu: Confirmed Status in linux source package in Xenial: Confirmed Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7 [31016.239899] RIP [] __xen_evtchn_do_upcall+0x43/0x80 [31016.239899] RSP [31016.239899] CR2: 82c4801e6bda [31016.239899] ---[ end trace 5b3e8ea32013e327 ]--- [31016.239899] Kernel panic - not syncing: Fatal exception in interrupt [31016.239899] Kernel Offset: disabled We believe this appeared in the last 1-2mo of releases, since this started happening after we did a `apt-get upgrade` on our machines after a bit of a pause. These are EC2 m4.xlarge servers running Ubuntu 16.04.1. They are Kubernetes minions, so I assume docker is likely the trigger. Unfortunately there aren't really any interesting logs in journalctl from the previous boot prior to the panic. Let me know what I can do to debug further. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-53-generic 4.4.0-53.74 ProcVersionSignature: Ubuntu 4.4.0-53.74-generic 4.4.30 Uname: Linux 4.4.0-53-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Dec 19 15:56 seq crw-rw 1 root audio 116, 33 Dec 19 15:56 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.1-0ubuntu2.4 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such
[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
Can you see if the latest 4.4 upstream kernel also fixes this bug? That will tell us if the fix in mainline was already cc'd to stable. It can be downloaded from: http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.4.42/ -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in Linux: New Status in linux package in Ubuntu: Confirmed Status in linux source package in Xenial: Confirmed Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7 [31016.239899] RIP [] __xen_evtchn_do_upcall+0x43/0x80 [31016.239899] RSP [31016.239899] CR2: 82c4801e6bda [31016.239899] ---[ end trace 5b3e8ea32013e327 ]--- [31016.239899] Kernel panic - not syncing: Fatal exception in interrupt [31016.239899] Kernel Offset: disabled We believe this appeared in the last 1-2mo of releases, since this started happening after we did a `apt-get upgrade` on our machines after a bit of a pause. These are EC2 m4.xlarge servers running Ubuntu 16.04.1. They are Kubernetes minions, so I assume docker is likely the trigger. Unfortunately there aren't really any interesting logs in journalctl from the previous boot prior to the panic. Let me know what I can do to debug further. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-53-generic 4.4.0-53.74 ProcVersionSignature: Ubuntu 4.4.0-53.74-generic 4.4.30 Uname: Linux 4.4.0-53-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Dec 19 15:56 seq crw-rw 1 root audio 116, 33 Dec 19 15:56 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.1-0ubuntu2.4 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file
[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
After upgrading to mainline builds everywhere, I haven't seen a restart in a least a day. Seems this is fixed in the mainline kernel. ** Changed in: linux (Ubuntu) Status: Incomplete => Confirmed ** Changed in: linux (Ubuntu Xenial) Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in Linux: New Status in linux package in Ubuntu: Confirmed Status in linux source package in Xenial: Confirmed Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7 [31016.239899] RIP [] __xen_evtchn_do_upcall+0x43/0x80 [31016.239899] RSP [31016.239899] CR2: 82c4801e6bda [31016.239899] ---[ end trace 5b3e8ea32013e327 ]--- [31016.239899] Kernel panic - not syncing: Fatal exception in interrupt [31016.239899] Kernel Offset: disabled We believe this appeared in the last 1-2mo of releases, since this started happening after we did a `apt-get upgrade` on our machines after a bit of a pause. These are EC2 m4.xlarge servers running Ubuntu 16.04.1. They are Kubernetes minions, so I assume docker is likely the trigger. Unfortunately there aren't really any interesting logs in journalctl from the previous boot prior to the panic. Let me know what I can do to debug further. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-53-generic 4.4.0-53.74 ProcVersionSignature: Ubuntu 4.4.0-53.74-generic 4.4.30 Uname: Linux 4.4.0-53-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Dec 19 15:56 seq crw-rw 1 root audio 116, 33 Dec 19 15:56 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.1-0ubuntu2.4
[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
** Tags added: kernel-fixed-upstream -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in Linux: New Status in linux package in Ubuntu: Confirmed Status in linux source package in Xenial: Confirmed Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7 [31016.239899] RIP [] __xen_evtchn_do_upcall+0x43/0x80 [31016.239899] RSP [31016.239899] CR2: 82c4801e6bda [31016.239899] ---[ end trace 5b3e8ea32013e327 ]--- [31016.239899] Kernel panic - not syncing: Fatal exception in interrupt [31016.239899] Kernel Offset: disabled We believe this appeared in the last 1-2mo of releases, since this started happening after we did a `apt-get upgrade` on our machines after a bit of a pause. These are EC2 m4.xlarge servers running Ubuntu 16.04.1. They are Kubernetes minions, so I assume docker is likely the trigger. Unfortunately there aren't really any interesting logs in journalctl from the previous boot prior to the panic. Let me know what I can do to debug further. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-53-generic 4.4.0-53.74 ProcVersionSignature: Ubuntu 4.4.0-53.74-generic 4.4.30 Uname: Linux 4.4.0-53-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Dec 19 15:56 seq crw-rw 1 root audio 116, 33 Dec 19 15:56 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.1-0ubuntu2.4 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Mon Dec 19 18:12:12 2016
[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
I also noticed that one of the boxes wasn't upgraded (for whatever reason) and has not had any panics in the last few days. The kernel version on that box is: Linux ip-10-0-26-30.us-west-2.compute.internal 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in Linux: New Status in linux package in Ubuntu: Incomplete Status in linux source package in Xenial: Incomplete Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7 [31016.239899] RIP [] __xen_evtchn_do_upcall+0x43/0x80 [31016.239899] RSP [31016.239899] CR2: 82c4801e6bda [31016.239899] ---[ end trace 5b3e8ea32013e327 ]--- [31016.239899] Kernel panic - not syncing: Fatal exception in interrupt [31016.239899] Kernel Offset: disabled We believe this appeared in the last 1-2mo of releases, since this started happening after we did a `apt-get upgrade` on our machines after a bit of a pause. These are EC2 m4.xlarge servers running Ubuntu 16.04.1. They are Kubernetes minions, so I assume docker is likely the trigger. Unfortunately there aren't really any interesting logs in journalctl from the previous boot prior to the panic. Let me know what I can do to debug further. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-53-generic 4.4.0-53.74 ProcVersionSignature: Ubuntu 4.4.0-53.74-generic 4.4.30 Uname: Linux 4.4.0-53-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Dec 19 15:56 seq crw-rw 1 root audio 116, 33 Dec 19 15:56 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion:
[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
My canary box is running 4.9.0-040900.201612111631 now and hasn't rebooted in ~1.5 days. I'll roll this out further, but this may be fixed in mainline. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in Linux: New Status in linux package in Ubuntu: Incomplete Status in linux source package in Xenial: Incomplete Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7 [31016.239899] RIP [] __xen_evtchn_do_upcall+0x43/0x80 [31016.239899] RSP [31016.239899] CR2: 82c4801e6bda [31016.239899] ---[ end trace 5b3e8ea32013e327 ]--- [31016.239899] Kernel panic - not syncing: Fatal exception in interrupt [31016.239899] Kernel Offset: disabled We believe this appeared in the last 1-2mo of releases, since this started happening after we did a `apt-get upgrade` on our machines after a bit of a pause. These are EC2 m4.xlarge servers running Ubuntu 16.04.1. They are Kubernetes minions, so I assume docker is likely the trigger. Unfortunately there aren't really any interesting logs in journalctl from the previous boot prior to the panic. Let me know what I can do to debug further. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-53-generic 4.4.0-53.74 ProcVersionSignature: Ubuntu 4.4.0-53.74-generic 4.4.30 Uname: Linux 4.4.0-53-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Dec 19 15:56 seq crw-rw 1 root audio 116, 33 Dec 19 15:56 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.1-0ubuntu2.4 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command
[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
BTW, I'm seeing this same bug (same symptom at least; the exact same call stack in the kernel panic) and I'm *not* using docker. The main workload on the machine where I'm seeing this is Postgres. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in Linux: New Status in linux package in Ubuntu: Incomplete Status in linux source package in Xenial: Incomplete Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7 [31016.239899] RIP [] __xen_evtchn_do_upcall+0x43/0x80 [31016.239899] RSP [31016.239899] CR2: 82c4801e6bda [31016.239899] ---[ end trace 5b3e8ea32013e327 ]--- [31016.239899] Kernel panic - not syncing: Fatal exception in interrupt [31016.239899] Kernel Offset: disabled We believe this appeared in the last 1-2mo of releases, since this started happening after we did a `apt-get upgrade` on our machines after a bit of a pause. These are EC2 m4.xlarge servers running Ubuntu 16.04.1. They are Kubernetes minions, so I assume docker is likely the trigger. Unfortunately there aren't really any interesting logs in journalctl from the previous boot prior to the panic. Let me know what I can do to debug further. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-53-generic 4.4.0-53.74 ProcVersionSignature: Ubuntu 4.4.0-53.74-generic 4.4.30 Uname: Linux 4.4.0-53-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Dec 19 15:56 seq crw-rw 1 root audio 116, 33 Dec 19 15:56 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.1-0ubuntu2.4 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory:
[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
I'm seeing the same thing on an m4.16xlarge EC2 instance. It has crashed twice in the past 24 hours. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in Linux: New Status in linux package in Ubuntu: Incomplete Status in linux source package in Xenial: Incomplete Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7 [31016.239899] RIP [] __xen_evtchn_do_upcall+0x43/0x80 [31016.239899] RSP [31016.239899] CR2: 82c4801e6bda [31016.239899] ---[ end trace 5b3e8ea32013e327 ]--- [31016.239899] Kernel panic - not syncing: Fatal exception in interrupt [31016.239899] Kernel Offset: disabled We believe this appeared in the last 1-2mo of releases, since this started happening after we did a `apt-get upgrade` on our machines after a bit of a pause. These are EC2 m4.xlarge servers running Ubuntu 16.04.1. They are Kubernetes minions, so I assume docker is likely the trigger. Unfortunately there aren't really any interesting logs in journalctl from the previous boot prior to the panic. Let me know what I can do to debug further. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-53-generic 4.4.0-53.74 ProcVersionSignature: Ubuntu 4.4.0-53.74-generic 4.4.30 Uname: Linux 4.4.0-53-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Dec 19 15:56 seq crw-rw 1 root audio 116, 33 Dec 19 15:56 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.1-0ubuntu2.4 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed
[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
Joseph - I have not identified a version of the kernel without the bug, though we have not seen this behavior from the released kernel ~2mo ago. I can try to pinpoint which kernel release it was, though I'd like to wait and see if the mainline kernel panics as well before bisecting. Unfortunately these happen several times a day so the turnaround cycle is a bit long. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in Linux: New Status in linux package in Ubuntu: Incomplete Status in linux source package in Xenial: Incomplete Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7 [31016.239899] RIP [] __xen_evtchn_do_upcall+0x43/0x80 [31016.239899] RSP [31016.239899] CR2: 82c4801e6bda [31016.239899] ---[ end trace 5b3e8ea32013e327 ]--- [31016.239899] Kernel panic - not syncing: Fatal exception in interrupt [31016.239899] Kernel Offset: disabled We believe this appeared in the last 1-2mo of releases, since this started happening after we did a `apt-get upgrade` on our machines after a bit of a pause. These are EC2 m4.xlarge servers running Ubuntu 16.04.1. They are Kubernetes minions, so I assume docker is likely the trigger. Unfortunately there aren't really any interesting logs in journalctl from the previous boot prior to the panic. Let me know what I can do to debug further. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-53-generic 4.4.0-53.74 ProcVersionSignature: Ubuntu 4.4.0-53.74-generic 4.4.30 Uname: Linux 4.4.0-53-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Dec 19 15:56 seq crw-rw 1 root audio 116, 33 Dec 19 15:56 timer AplayDevices: Error:
[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
Joseph - I switched one of the 6 machines that previously showed this behavior to the mainline kernel: Linux ip-10-0-32-196.us- west-2.compute.internal 4.9.0-040900-generic #201612111631 SMP Sun Dec 11 21:33:00 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux . I'll let you know the results of this experiment tomorrow. Fumihito - I saw that docker bug when researching this issue, and I downgraded docker-engine on those machines to 1.12.2. However, that didn't help. Hopefully the mainline kernel test will shed some light on this. For reference this is my docker config: $ sudo docker info Containers: 102 Running: 13 Paused: 0 Stopped: 89 Images: 13 Server Version: 1.12.2 Storage Driver: overlay2 Backing Filesystem: extfs Logging Driver: journald Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host overlay null Swarm: inactive Runtimes: runc Default Runtime: runc Security Options: apparmor seccomp Kernel Version: 4.9.0-040900-generic Operating System: Ubuntu 16.04.1 LTS OSType: linux Architecture: x86_64 CPUs: 4 Total Memory: 15.67 GiB Name: ip-10-0-32-196.us-west-2.compute.internal ID: TSIL:W3HI:APTU:DSVF:QDEF:HOJ6:PRB7:L7EL:QRQ2:K63C:CH22:RVQR Docker Root Dir: /data/docker Debug Mode (client): false Debug Mode (server): true File Descriptors: 59 Goroutines: 116 System Time: 2016-12-20T07:49:15.79205206Z EventsListeners: 0 Registry: https://index.docker.io/v1/ WARNING: No swap limit support Insecure Registries: 127.0.0.0/8 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in Linux: New Status in linux package in Ubuntu: Incomplete Status in linux source package in Xenial: Incomplete Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7
[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
** Changed in: linux Status: Unknown => New -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in Linux: New Status in linux package in Ubuntu: Incomplete Status in linux source package in Xenial: Incomplete Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7 [31016.239899] RIP [] __xen_evtchn_do_upcall+0x43/0x80 [31016.239899] RSP [31016.239899] CR2: 82c4801e6bda [31016.239899] ---[ end trace 5b3e8ea32013e327 ]--- [31016.239899] Kernel panic - not syncing: Fatal exception in interrupt [31016.239899] Kernel Offset: disabled We believe this appeared in the last 1-2mo of releases, since this started happening after we did a `apt-get upgrade` on our machines after a bit of a pause. These are EC2 m4.xlarge servers running Ubuntu 16.04.1. They are Kubernetes minions, so I assume docker is likely the trigger. Unfortunately there aren't really any interesting logs in journalctl from the previous boot prior to the panic. Let me know what I can do to debug further. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-53-generic 4.4.0-53.74 ProcVersionSignature: Ubuntu 4.4.0-53.74-generic 4.4.30 Uname: Linux 4.4.0-53-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Dec 19 15:56 seq crw-rw 1 root audio 116, 33 Dec 19 15:56 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.1-0ubuntu2.4 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A Date: Mon Dec 19
[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
This bugs seems reported at upstream https://github.com/docker/docker/issues/27875 , linked. ** Bug watch added: github.com/docker/docker/issues #27875 https://github.com/docker/docker/issues/27875 ** Also affects: linux via https://github.com/docker/docker/issues/27875 Importance: Unknown Status: Unknown -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in Linux: Unknown Status in linux package in Ubuntu: Incomplete Status in linux source package in Xenial: Incomplete Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7 [31016.239899] RIP [] __xen_evtchn_do_upcall+0x43/0x80 [31016.239899] RSP [31016.239899] CR2: 82c4801e6bda [31016.239899] ---[ end trace 5b3e8ea32013e327 ]--- [31016.239899] Kernel panic - not syncing: Fatal exception in interrupt [31016.239899] Kernel Offset: disabled We believe this appeared in the last 1-2mo of releases, since this started happening after we did a `apt-get upgrade` on our machines after a bit of a pause. These are EC2 m4.xlarge servers running Ubuntu 16.04.1. They are Kubernetes minions, so I assume docker is likely the trigger. Unfortunately there aren't really any interesting logs in journalctl from the previous boot prior to the panic. Let me know what I can do to debug further. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-53-generic 4.4.0-53.74 ProcVersionSignature: Ubuntu 4.4.0-53.74-generic 4.4.30 Uname: Linux 4.4.0-53-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Dec 19 15:56 seq crw-rw 1 root audio 116, 33 Dec 19 15:56 timer AplayDevices: Error: [Errno 2] No such file or directory:
[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
Also, is there a prior kernel version you've identified that does not exhibit this bug? ** Tags added: kernel-da-key ** Also affects: linux (Ubuntu Xenial) Importance: Undecided Status: New ** Changed in: linux (Ubuntu Xenial) Importance: Undecided => High ** Changed in: linux (Ubuntu Xenial) Status: New => Confirmed ** Changed in: linux (Ubuntu Xenial) Status: Confirmed => Incomplete ** Changed in: linux (Ubuntu) Status: Confirmed => Incomplete -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in linux package in Ubuntu: Incomplete Status in linux source package in Xenial: Incomplete Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7 [31016.239899] RIP [] __xen_evtchn_do_upcall+0x43/0x80 [31016.239899] RSP [31016.239899] CR2: 82c4801e6bda [31016.239899] ---[ end trace 5b3e8ea32013e327 ]--- [31016.239899] Kernel panic - not syncing: Fatal exception in interrupt [31016.239899] Kernel Offset: disabled We believe this appeared in the last 1-2mo of releases, since this started happening after we did a `apt-get upgrade` on our machines after a bit of a pause. These are EC2 m4.xlarge servers running Ubuntu 16.04.1. They are Kubernetes minions, so I assume docker is likely the trigger. Unfortunately there aren't really any interesting logs in journalctl from the previous boot prior to the panic. Let me know what I can do to debug further. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-53-generic 4.4.0-53.74 ProcVersionSignature: Ubuntu 4.4.0-53.74-generic 4.4.30 Uname: Linux 4.4.0-53-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116,
[Kernel-packages] [Bug 1651199] Re: 16.04 - Kernel panic on docker server
Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v4.9 kernel[0]. If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'. If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'. Once testing of the upstream kernel is complete, please mark this bug as "Confirmed". Thanks in advance. [0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.9 ** Changed in: linux (Ubuntu) Importance: Undecided => High -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1651199 Title: 16.04 - Kernel panic on docker server Status in linux package in Ubuntu: Incomplete Status in linux source package in Xenial: Incomplete Bug description: [31016.057405] BUG: unable to handle kernel paging request at 82c4801e6bda [31016.061249] IP: [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] PGD 0 [31016.061380] Oops: 0010 [#1] SMP [31016.061380] Modules linked in: binfmt_misc xt_REDIRECT nf_nat_redirect veth xt_comment xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo xt_addrtype iptable_filter xt_conntrack br_netfilter bridge stp llc overlay xt_nat xt_tcpudp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ip_tables x_tables isofs ppdev serio_raw parport_pc parport ib_iser rdma_cm iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd ixgbevf psmouse floppy [31016.061380] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-53-generic #74-Ubuntu [31016.061380] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016 [31016.061380] task: 880406496040 ti: 8804064e task.ti: 8804064e [31016.061380] RIP: 0010:[] [] __xen_evtchn_do_upcall+0x43/0x80 [31016.061380] RSP: 0018:88040fc43f78 EFLAGS: 00010082 [31016.061380] RAX: RBX: 88040fc4b840 RCX: 0001 [31016.061380] RDX: fffe RSI: 000123a0 RDI: 88040fc43f30 [31016.061380] RBP: 88040fc43f90 R08: f24a R09: 0001 [31016.061380] R10: 0001 R11: R12: 0001 [31016.061380] R13: 0003 R14: R15: 8804064e [31016.178419] FS: () GS:88040fc4() knlGS: [31016.178419] CS: 0010 DS: ES: CR0: 80050033 [31016.178419] CR2: 82c4801e6bda CR3: 000204f2f000 CR4: 001406e0 [31016.178419] DR0: DR1: DR2: [31016.178419] DR3: DR6: fffe0ff0 DR7: 0400 [31016.178419] Stack: [31016.178419] 0003 88040fc43fa8 [31016.178419] 814d6bc0 81f36a00 8804064e3e90 81837f32 [31016.178419] 8804064e3de8 8804064e [31016.178419] Call Trace: [31016.178419] [31016.178419] [] xen_evtchn_do_upcall+0x30/0x40 [31016.178419] [] xen_hvm_callback_vector+0x82/0x90 [31016.178419] [31016.178419] [] ? native_safe_halt+0x6/0x10 [31016.178419] [] default_idle+0x1e/0xe0 [31016.239315] [] arch_cpu_idle+0xf/0x20 [31016.239899] [] default_idle_call+0x2a/0x40 [31016.239899] [] cpu_startup_entry+0x2f1/0x350 [31016.239899] [] start_secondary+0x154/0x190 [31016.239899] Code: 01 00 00 00 65 44 8b 2d dc 56 b3 7e c6 03 00 44 89 e0 65 0f c1 05 2e d8 b3 7e 85 c0 75 35 48 8b 05 63 ce d0 00 44 89 ef ff 50 50 <9c> 58 0f 1f 44 00 00 f6 c4 02 75 23 65 8b 05 0a d8 b3 7e 65 c7 [31016.239899] RIP [] __xen_evtchn_do_upcall+0x43/0x80 [31016.239899] RSP [31016.239899] CR2: 82c4801e6bda [31016.239899] ---[ end trace 5b3e8ea32013e327 ]--- [31016.239899] Kernel panic - not syncing: Fatal exception in interrupt [31016.239899] Kernel Offset: disabled We believe this appeared in the last 1-2mo of releases, since this started happening after we did a `apt-get upgrade` on our machines after a bit of a pause. These are EC2 m4.xlarge servers running Ubuntu 16.04.1. They are Kubernetes minions, so I assume docker is likely the trigger. Unfortunately there aren't really any interesting logs in journalctl from the previous boot prior to the panic. Let me know what I can do to debug further. ProblemType: Bug DistroRelease: Ubuntu 16.04 Package: linux-image-4.4.0-53-generic 4.4.0-53.74 ProcVersionSignature: Ubuntu 4.4.0-53.74-generic 4.4.30
