Public bug reported: Hi,
On one of the main US Ubuntu Archive servers (banjo), we decided to reboot into a HWE kernel. The latest being 5.4.0-122 but on doing so, ran into this kernel panic: | [ 350.776585] BUG: kernel NULL pointer dereference, address: 0000000000000008 | [ 350.783674] #PF: supervisor read access in kernel mode | [ 350.788846] #PF: error_code(0x0000) - not-present page | [ 350.794019] PGD 0 P4D 0 | [ 350.796631] Oops: 0000 [#1] SMP NOPTI | [ 350.800425] CPU: 8 PID: 0 Comm: swapper/8 Not tainted 5.4.0-122-generic #138~18.04.1-Ubuntu | [ 350.808918] Hardware name: HPE ProLiant DL385 Gen10/ProLiant DL385 Gen10, BIOS A40 02/10/2022 | [ 350.817666] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 350.823187] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 350.842068] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 350.847324] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 350.854502] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 350.861682] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 350.868859] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 350.876039] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 350.883219] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 350.891358] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 350.897138] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 350.904319] Call Trace: | [ 350.906787] <IRQ> | [ 350.908824] tcp_v6_syn_recv_sock+0x8d/0x710 | [ 350.913259] ? ip6_route_output_flags_noref+0xd0/0x110 | [ 350.918435] tcp_get_cookie_sock+0x48/0x140 | [ 350.922688] cookie_v6_check+0x5a2/0x700 | [ 350.926714] tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.930589] ? tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.934589] tcp_v6_rcv+0xa16/0xa60 | [ 350.938102] ip6_protocol_deliver_rcu+0xd8/0x4d0 | [ 350.942750] ip6_input+0x41/0xb0 | [ 350.946000] ip6_sublist_rcv_finish+0x42/0x60 | [ 350.950385] ip6_sublist_rcv+0x235/0x260 | [ 350.954333] ? __netif_receive_skb_core+0x19d/0xc60 | [ 350.959245] ipv6_list_rcv+0x110/0x140 | [ 350.963018] __netif_receive_skb_list_core+0x157/0x260 | [ 350.968192] ? build_skb+0x17/0x80 | [ 350.971615] netif_receive_skb_list_internal+0x187/0x2a0 | [ 350.976961] gro_normal_list.part.131+0x1e/0x40 | [ 350.981519] napi_complete_done+0x94/0x120 | [ 350.985700] mlx5e_napi_poll+0x178/0x630 [mlx5_core] | [ 350.990697] net_rx_action+0x140/0x3e0 | [ 350.994475] __do_softirq+0xe4/0x2da | [ 350.998079] irq_exit+0xae/0xb0 | [ 351.001239] do_IRQ+0x59/0xe0 | [ 351.004228] common_interrupt+0xf/0xf | [ 351.007913] </IRQ> | [ 351.010029] RIP: 0010:cpuidle_enter_state+0xbc/0x440 | [ 351.015023] Code: ff e8 b8 ca 80 ff 80 7d d3 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 54 03 00 00 31 ff e8 4b 4f 87 ff fb 66 0f 1f 44 00 00 <45> 85 ed 0f 88 1a 03 00 00 4c 2b 7d c8 48 ba cf f7 53 e3 a5 9b c4 | [ 351.033952] RSP: 0018:ffff9a958026fe48 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffd6 | [ 351.041633] RAX: ffff89763ec2fe00 RBX: ffffffff84b66b40 RCX: 000000000000001f | [ 351.048816] RDX: 00000051abe96150 RSI: 000000002abf3234 RDI: 0000000000000000 | [ 351.055997] RBP: ffff9a958026fe88 R08: 0000000000000002 R09: 000000000002f680 | [ 351.063176] R10: ffff9a958026fe18 R11: 0000000000000115 R12: ffff8976274c3800 | [ 351.070355] R13: 0000000000000001 R14: ffffffff84b66bb8 R15: 00000051abe96150 | [ 351.077540] ? cpuidle_enter_state+0x98/0x440 | [ 351.081930] ? menu_select+0x377/0x600 | [ 351.085706] cpuidle_enter+0x2e/0x40 | [ 351.089310] call_cpuidle+0x23/0x40 | [ 351.092821] do_idle+0x1f6/0x270 | [ 351.096069] cpu_startup_entry+0x1d/0x20 | [ 351.100024] start_secondary+0x166/0x1c0 | [ 351.103977] secondary_startup_64+0xa4/0xb0 | [ 351.108186] Modules linked in: binfmt_misc bonding nls_iso8859_1 ipmi_ssif edac_mce_amd kvm_amd kvm hpilo ccp ipmi_si ipmi_devintf ipmi_msghandler acpi_tad k10temp mac_hid acpi_power_meter sch_fq tcp_bbr ib_iser rdma_cm iw_cm ib_cm iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid0 multipath linear mlx5_ib raid1 ses enclosure ib_uverbs ib_core mgag200 drm_vram_helper ttm drm_kms_helper syscopyarea crct10dif_pclmul sysfillrect mlx5_core crc32_pclmul sysimgblt smartpqi fb_sys_fops uas ghash_clmulni_intel aesni_intel crypto_simd igb pci_hyperv_intf cryptd glue_helper usb_storage dca tls drm i2c_algo_bit scsi_transport_sas mlxfw nvme i2c_piix4 nvme_core wmi | [ 351.180156] CR2: 0000000000000008 | [ 351.183629] ---[ end trace 23210cdf0c6d5851 ]--- | [ 351.322276] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 351.327974] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 351.346878] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 351.352166] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 351.359348] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 351.366526] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 351.373705] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 351.380886] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 351.388065] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 351.396203] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 351.401982] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 351.409162] Kernel panic - not syncing: Fatal exception in interrupt | [ 351.415613] Kernel Offset: 0x2000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) | [ 351.437793] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]--- Per ~IS-Outage on Mattermost, tried various other older kernels and it seems -121 is working fine so looks to be introduced in -122. | [hloeung@banjo ~]$ lsb_release -a | No LSB modules are available. | Distributor ID: Ubuntu | Description: Ubuntu 18.04.6 LTS | Release: 18.04 | Codename: bionic ** Affects: linux (Ubuntu) Importance: Undecided Status: Incomplete ** Affects: linux-hwe-5.4 (Ubuntu) Importance: Undecided Status: New ** Affects: linux (Ubuntu Bionic) Importance: Undecided Status: Incomplete ** Affects: linux-hwe-5.4 (Ubuntu Bionic) Importance: Undecided Status: New ** Also affects: linux (Ubuntu) Importance: Undecided Status: New ** Also affects: linux (Ubuntu Bionic) Importance: Undecided Status: New ** Also affects: linux-hwe-5.4 (Ubuntu Bionic) Importance: Undecided Status: New ** Description changed: Hi, - On one of the main US Ubuntu Archive servers, we decided to reboot into - a HWE kernel. The latest being 5.4.0-122 but on doing so, ran into this - kernel panic: + On one of the main US Ubuntu Archive servers (banjo), we decided to + reboot into a HWE kernel. The latest being 5.4.0-122 but on doing so, + ran into this kernel panic: | [ 350.776585] BUG: kernel NULL pointer dereference, address: 0000000000000008 | [ 350.783674] #PF: supervisor read access in kernel mode | [ 350.788846] #PF: error_code(0x0000) - not-present page - | [ 350.794019] PGD 0 P4D 0 + | [ 350.794019] PGD 0 P4D 0 | [ 350.796631] Oops: 0000 [#1] SMP NOPTI | [ 350.800425] CPU: 8 PID: 0 Comm: swapper/8 Not tainted 5.4.0-122-generic #138~18.04.1-Ubuntu | [ 350.808918] Hardware name: HPE ProLiant DL385 Gen10/ProLiant DL385 Gen10, BIOS A40 02/10/2022 | [ 350.817666] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 350.823187] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 350.842068] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 350.847324] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 350.854502] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 350.861682] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 350.868859] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 350.876039] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 350.883219] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 350.891358] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 350.897138] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 350.904319] Call Trace: | [ 350.906787] <IRQ> | [ 350.908824] tcp_v6_syn_recv_sock+0x8d/0x710 | [ 350.913259] ? ip6_route_output_flags_noref+0xd0/0x110 | [ 350.918435] tcp_get_cookie_sock+0x48/0x140 | [ 350.922688] cookie_v6_check+0x5a2/0x700 | [ 350.926714] tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.930589] ? tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.934589] tcp_v6_rcv+0xa16/0xa60 | [ 350.938102] ip6_protocol_deliver_rcu+0xd8/0x4d0 | [ 350.942750] ip6_input+0x41/0xb0 | [ 350.946000] ip6_sublist_rcv_finish+0x42/0x60 | [ 350.950385] ip6_sublist_rcv+0x235/0x260 | [ 350.954333] ? __netif_receive_skb_core+0x19d/0xc60 | [ 350.959245] ipv6_list_rcv+0x110/0x140 | [ 350.963018] __netif_receive_skb_list_core+0x157/0x260 | [ 350.968192] ? build_skb+0x17/0x80 | [ 350.971615] netif_receive_skb_list_internal+0x187/0x2a0 | [ 350.976961] gro_normal_list.part.131+0x1e/0x40 | [ 350.981519] napi_complete_done+0x94/0x120 | [ 350.985700] mlx5e_napi_poll+0x178/0x630 [mlx5_core] | [ 350.990697] net_rx_action+0x140/0x3e0 | [ 350.994475] __do_softirq+0xe4/0x2da | [ 350.998079] irq_exit+0xae/0xb0 | [ 351.001239] do_IRQ+0x59/0xe0 | [ 351.004228] common_interrupt+0xf/0xf | [ 351.007913] </IRQ> | [ 351.010029] RIP: 0010:cpuidle_enter_state+0xbc/0x440 | [ 351.015023] Code: ff e8 b8 ca 80 ff 80 7d d3 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 54 03 00 00 31 ff e8 4b 4f 87 ff fb 66 0f 1f 44 00 00 <45> 85 ed 0f 88 1a 03 00 00 4c 2b 7d c8 48 ba cf f7 53 e3 a5 9b c4 | [ 351.033952] RSP: 0018:ffff9a958026fe48 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffd6 | [ 351.041633] RAX: ffff89763ec2fe00 RBX: ffffffff84b66b40 RCX: 000000000000001f | [ 351.048816] RDX: 00000051abe96150 RSI: 000000002abf3234 RDI: 0000000000000000 | [ 351.055997] RBP: ffff9a958026fe88 R08: 0000000000000002 R09: 000000000002f680 | [ 351.063176] R10: ffff9a958026fe18 R11: 0000000000000115 R12: ffff8976274c3800 | [ 351.070355] R13: 0000000000000001 R14: ffffffff84b66bb8 R15: 00000051abe96150 | [ 351.077540] ? cpuidle_enter_state+0x98/0x440 | [ 351.081930] ? menu_select+0x377/0x600 | [ 351.085706] cpuidle_enter+0x2e/0x40 | [ 351.089310] call_cpuidle+0x23/0x40 | [ 351.092821] do_idle+0x1f6/0x270 | [ 351.096069] cpu_startup_entry+0x1d/0x20 | [ 351.100024] start_secondary+0x166/0x1c0 | [ 351.103977] secondary_startup_64+0xa4/0xb0 | [ 351.108186] Modules linked in: binfmt_misc bonding nls_iso8859_1 ipmi_ssif edac_mce_amd kvm_amd kvm hpilo ccp ipmi_si ipmi_devintf ipmi_msghandler acpi_tad k10temp mac_hid acpi_power_meter sch_fq tcp_bbr ib_iser rdma_cm iw_cm ib_cm iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid0 multipath linear mlx5_ib raid1 ses enclosure ib_uverbs ib_core mgag200 drm_vram_helper ttm drm_kms_helper syscopyarea crct10dif_pclmul sysfillrect mlx5_core crc32_pclmul sysimgblt smartpqi fb_sys_fops uas ghash_clmulni_intel aesni_intel crypto_simd igb pci_hyperv_intf cryptd glue_helper usb_storage dca tls drm i2c_algo_bit scsi_transport_sas mlxfw nvme i2c_piix4 nvme_core wmi | [ 351.180156] CR2: 0000000000000008 | [ 351.183629] ---[ end trace 23210cdf0c6d5851 ]--- | [ 351.322276] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 351.327974] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 351.346878] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 351.352166] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 351.359348] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 351.366526] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 351.373705] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 351.380886] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 351.388065] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 351.396203] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 351.401982] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 351.409162] Kernel panic - not syncing: Fatal exception in interrupt | [ 351.415613] Kernel Offset: 0x2000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) | [ 351.437793] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]--- Per ~IS-Outage on Mattermost, tried various other older kernels and it seems -121 is working fine so looks to be introduced in -122 (maybe LP:1978719?). ** Description changed: Hi, On one of the main US Ubuntu Archive servers (banjo), we decided to reboot into a HWE kernel. The latest being 5.4.0-122 but on doing so, ran into this kernel panic: | [ 350.776585] BUG: kernel NULL pointer dereference, address: 0000000000000008 | [ 350.783674] #PF: supervisor read access in kernel mode | [ 350.788846] #PF: error_code(0x0000) - not-present page | [ 350.794019] PGD 0 P4D 0 | [ 350.796631] Oops: 0000 [#1] SMP NOPTI | [ 350.800425] CPU: 8 PID: 0 Comm: swapper/8 Not tainted 5.4.0-122-generic #138~18.04.1-Ubuntu | [ 350.808918] Hardware name: HPE ProLiant DL385 Gen10/ProLiant DL385 Gen10, BIOS A40 02/10/2022 | [ 350.817666] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 350.823187] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 350.842068] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 350.847324] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 350.854502] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 350.861682] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 350.868859] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 350.876039] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 350.883219] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 350.891358] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 350.897138] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 350.904319] Call Trace: | [ 350.906787] <IRQ> | [ 350.908824] tcp_v6_syn_recv_sock+0x8d/0x710 | [ 350.913259] ? ip6_route_output_flags_noref+0xd0/0x110 | [ 350.918435] tcp_get_cookie_sock+0x48/0x140 | [ 350.922688] cookie_v6_check+0x5a2/0x700 | [ 350.926714] tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.930589] ? tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.934589] tcp_v6_rcv+0xa16/0xa60 | [ 350.938102] ip6_protocol_deliver_rcu+0xd8/0x4d0 | [ 350.942750] ip6_input+0x41/0xb0 | [ 350.946000] ip6_sublist_rcv_finish+0x42/0x60 | [ 350.950385] ip6_sublist_rcv+0x235/0x260 | [ 350.954333] ? __netif_receive_skb_core+0x19d/0xc60 | [ 350.959245] ipv6_list_rcv+0x110/0x140 | [ 350.963018] __netif_receive_skb_list_core+0x157/0x260 | [ 350.968192] ? build_skb+0x17/0x80 | [ 350.971615] netif_receive_skb_list_internal+0x187/0x2a0 | [ 350.976961] gro_normal_list.part.131+0x1e/0x40 | [ 350.981519] napi_complete_done+0x94/0x120 | [ 350.985700] mlx5e_napi_poll+0x178/0x630 [mlx5_core] | [ 350.990697] net_rx_action+0x140/0x3e0 | [ 350.994475] __do_softirq+0xe4/0x2da | [ 350.998079] irq_exit+0xae/0xb0 | [ 351.001239] do_IRQ+0x59/0xe0 | [ 351.004228] common_interrupt+0xf/0xf | [ 351.007913] </IRQ> | [ 351.010029] RIP: 0010:cpuidle_enter_state+0xbc/0x440 | [ 351.015023] Code: ff e8 b8 ca 80 ff 80 7d d3 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 54 03 00 00 31 ff e8 4b 4f 87 ff fb 66 0f 1f 44 00 00 <45> 85 ed 0f 88 1a 03 00 00 4c 2b 7d c8 48 ba cf f7 53 e3 a5 9b c4 | [ 351.033952] RSP: 0018:ffff9a958026fe48 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffd6 | [ 351.041633] RAX: ffff89763ec2fe00 RBX: ffffffff84b66b40 RCX: 000000000000001f | [ 351.048816] RDX: 00000051abe96150 RSI: 000000002abf3234 RDI: 0000000000000000 | [ 351.055997] RBP: ffff9a958026fe88 R08: 0000000000000002 R09: 000000000002f680 | [ 351.063176] R10: ffff9a958026fe18 R11: 0000000000000115 R12: ffff8976274c3800 | [ 351.070355] R13: 0000000000000001 R14: ffffffff84b66bb8 R15: 00000051abe96150 | [ 351.077540] ? cpuidle_enter_state+0x98/0x440 | [ 351.081930] ? menu_select+0x377/0x600 | [ 351.085706] cpuidle_enter+0x2e/0x40 | [ 351.089310] call_cpuidle+0x23/0x40 | [ 351.092821] do_idle+0x1f6/0x270 | [ 351.096069] cpu_startup_entry+0x1d/0x20 | [ 351.100024] start_secondary+0x166/0x1c0 | [ 351.103977] secondary_startup_64+0xa4/0xb0 | [ 351.108186] Modules linked in: binfmt_misc bonding nls_iso8859_1 ipmi_ssif edac_mce_amd kvm_amd kvm hpilo ccp ipmi_si ipmi_devintf ipmi_msghandler acpi_tad k10temp mac_hid acpi_power_meter sch_fq tcp_bbr ib_iser rdma_cm iw_cm ib_cm iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid0 multipath linear mlx5_ib raid1 ses enclosure ib_uverbs ib_core mgag200 drm_vram_helper ttm drm_kms_helper syscopyarea crct10dif_pclmul sysfillrect mlx5_core crc32_pclmul sysimgblt smartpqi fb_sys_fops uas ghash_clmulni_intel aesni_intel crypto_simd igb pci_hyperv_intf cryptd glue_helper usb_storage dca tls drm i2c_algo_bit scsi_transport_sas mlxfw nvme i2c_piix4 nvme_core wmi | [ 351.180156] CR2: 0000000000000008 | [ 351.183629] ---[ end trace 23210cdf0c6d5851 ]--- | [ 351.322276] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 351.327974] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 351.346878] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 351.352166] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 351.359348] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 351.366526] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 351.373705] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 351.380886] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 351.388065] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 351.396203] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 351.401982] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 351.409162] Kernel panic - not syncing: Fatal exception in interrupt | [ 351.415613] Kernel Offset: 0x2000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) | [ 351.437793] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]--- Per ~IS-Outage on Mattermost, tried various other older kernels and it - seems -121 is working fine so looks to be introduced in -122 (maybe - LP:1978719?). + seems -121 is working fine so looks to be introduced in -122. + + | [hloeung@banjo ~]$ lsb_release -a + | No LSB modules are available. + | Distributor ID: Ubuntu + | Description: Ubuntu 18.04.6 LTS + | Release: 18.04 + | Codename: bionic -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-hwe-5.4 in Ubuntu. https://bugs.launchpad.net/bugs/1981658 Title: BUG: kernel NULL pointer dereference, address: 0000000000000008 Status in linux package in Ubuntu: Incomplete Status in linux-hwe-5.4 package in Ubuntu: New Status in linux source package in Bionic: Incomplete Status in linux-hwe-5.4 source package in Bionic: New Bug description: Hi, On one of the main US Ubuntu Archive servers (banjo), we decided to reboot into a HWE kernel. The latest being 5.4.0-122 but on doing so, ran into this kernel panic: | [ 350.776585] BUG: kernel NULL pointer dereference, address: 0000000000000008 | [ 350.783674] #PF: supervisor read access in kernel mode | [ 350.788846] #PF: error_code(0x0000) - not-present page | [ 350.794019] PGD 0 P4D 0 | [ 350.796631] Oops: 0000 [#1] SMP NOPTI | [ 350.800425] CPU: 8 PID: 0 Comm: swapper/8 Not tainted 5.4.0-122-generic #138~18.04.1-Ubuntu | [ 350.808918] Hardware name: HPE ProLiant DL385 Gen10/ProLiant DL385 Gen10, BIOS A40 02/10/2022 | [ 350.817666] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 350.823187] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 350.842068] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 350.847324] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 350.854502] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 350.861682] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 350.868859] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 350.876039] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 350.883219] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 350.891358] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 350.897138] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 350.904319] Call Trace: | [ 350.906787] <IRQ> | [ 350.908824] tcp_v6_syn_recv_sock+0x8d/0x710 | [ 350.913259] ? ip6_route_output_flags_noref+0xd0/0x110 | [ 350.918435] tcp_get_cookie_sock+0x48/0x140 | [ 350.922688] cookie_v6_check+0x5a2/0x700 | [ 350.926714] tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.930589] ? tcp_v6_do_rcv+0x36c/0x3e0 | [ 350.934589] tcp_v6_rcv+0xa16/0xa60 | [ 350.938102] ip6_protocol_deliver_rcu+0xd8/0x4d0 | [ 350.942750] ip6_input+0x41/0xb0 | [ 350.946000] ip6_sublist_rcv_finish+0x42/0x60 | [ 350.950385] ip6_sublist_rcv+0x235/0x260 | [ 350.954333] ? __netif_receive_skb_core+0x19d/0xc60 | [ 350.959245] ipv6_list_rcv+0x110/0x140 | [ 350.963018] __netif_receive_skb_list_core+0x157/0x260 | [ 350.968192] ? build_skb+0x17/0x80 | [ 350.971615] netif_receive_skb_list_internal+0x187/0x2a0 | [ 350.976961] gro_normal_list.part.131+0x1e/0x40 | [ 350.981519] napi_complete_done+0x94/0x120 | [ 350.985700] mlx5e_napi_poll+0x178/0x630 [mlx5_core] | [ 350.990697] net_rx_action+0x140/0x3e0 | [ 350.994475] __do_softirq+0xe4/0x2da | [ 350.998079] irq_exit+0xae/0xb0 | [ 351.001239] do_IRQ+0x59/0xe0 | [ 351.004228] common_interrupt+0xf/0xf | [ 351.007913] </IRQ> | [ 351.010029] RIP: 0010:cpuidle_enter_state+0xbc/0x440 | [ 351.015023] Code: ff e8 b8 ca 80 ff 80 7d d3 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 54 03 00 00 31 ff e8 4b 4f 87 ff fb 66 0f 1f 44 00 00 <45> 85 ed 0f 88 1a 03 00 00 4c 2b 7d c8 48 ba cf f7 53 e3 a5 9b c4 | [ 351.033952] RSP: 0018:ffff9a958026fe48 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffd6 | [ 351.041633] RAX: ffff89763ec2fe00 RBX: ffffffff84b66b40 RCX: 000000000000001f | [ 351.048816] RDX: 00000051abe96150 RSI: 000000002abf3234 RDI: 0000000000000000 | [ 351.055997] RBP: ffff9a958026fe88 R08: 0000000000000002 R09: 000000000002f680 | [ 351.063176] R10: ffff9a958026fe18 R11: 0000000000000115 R12: ffff8976274c3800 | [ 351.070355] R13: 0000000000000001 R14: ffffffff84b66bb8 R15: 00000051abe96150 | [ 351.077540] ? cpuidle_enter_state+0x98/0x440 | [ 351.081930] ? menu_select+0x377/0x600 | [ 351.085706] cpuidle_enter+0x2e/0x40 | [ 351.089310] call_cpuidle+0x23/0x40 | [ 351.092821] do_idle+0x1f6/0x270 | [ 351.096069] cpu_startup_entry+0x1d/0x20 | [ 351.100024] start_secondary+0x166/0x1c0 | [ 351.103977] secondary_startup_64+0xa4/0xb0 | [ 351.108186] Modules linked in: binfmt_misc bonding nls_iso8859_1 ipmi_ssif edac_mce_amd kvm_amd kvm hpilo ccp ipmi_si ipmi_devintf ipmi_msghandler acpi_tad k10temp mac_hid acpi_power_meter sch_fq tcp_bbr ib_iser rdma_cm iw_cm ib_cm iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid0 multipath linear mlx5_ib raid1 ses enclosure ib_uverbs ib_core mgag200 drm_vram_helper ttm drm_kms_helper syscopyarea crct10dif_pclmul sysfillrect mlx5_core crc32_pclmul sysimgblt smartpqi fb_sys_fops uas ghash_clmulni_intel aesni_intel crypto_simd igb pci_hyperv_intf cryptd glue_helper usb_storage dca tls drm i2c_algo_bit scsi_transport_sas mlxfw nvme i2c_piix4 nvme_core wmi | [ 351.180156] CR2: 0000000000000008 | [ 351.183629] ---[ end trace 23210cdf0c6d5851 ]--- | [ 351.322276] RIP: 0010:tcp_create_openreq_child+0x2e1/0x3e0 | [ 351.327974] Code: 08 00 00 41 8b 84 24 18 01 00 00 48 c7 83 80 08 00 00 00 00 00 00 4c 89 e6 4c 89 ef 89 83 c4 05 00 00 49 8b 84 24 f8 00 00 00 <48> 8b 40 08 e8 96 28 42 00 48 85 c0 0f b7 83 68 05 00 00 74 0a 83 | [ 351.346878] RSP: 0018:ffff9a958cce8858 EFLAGS: 00010246 | [ 351.352166] RAX: 0000000000000000 RBX: ffff897618739c80 RCX: 0000000000000007 | [ 351.359348] RDX: 0000000000000020 RSI: ffff897607afb0b0 RDI: ffff897605c85580 | [ 351.366526] RBP: ffff9a958cce8878 R08: 0000000000000178 R09: ffff89763e407800 | [ 351.373705] R10: 00000000000004c4 R11: ffff9a958cce89c7 R12: ffff897607afb0b0 | [ 351.380886] R13: ffff897605c85580 R14: ffff8976205fbe00 R15: ffff89762688b400 | [ 351.388065] FS: 0000000000000000(0000) GS:ffff89763ec00000(0000) knlGS:0000000000000000 | [ 351.396203] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 | [ 351.401982] CR2: 0000000000000008 CR3: 0000001fd7914000 CR4: 0000000000340ee0 | [ 351.409162] Kernel panic - not syncing: Fatal exception in interrupt | [ 351.415613] Kernel Offset: 0x2000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) | [ 351.437793] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]--- Per ~IS-Outage on Mattermost, tried various other older kernels and it seems -121 is working fine so looks to be introduced in -122. | [hloeung@banjo ~]$ lsb_release -a | No LSB modules are available. | Distributor ID: Ubuntu | Description: Ubuntu 18.04.6 LTS | Release: 18.04 | Codename: bionic To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1981658/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp