Hi list,
After updating from 1.8.13 to 2.0.5 (also with 2.0.10) we are seeing
kernel panics on our production servers. I haven't been able to trigger
them on a test server, and we rollbacked haproxy to 1.8 for now.
I am attaching a panic log, hope something useful is in there.
Anybody an idea what might be going on here?
Kind regards,
Sander Hoentjen
Dec 2 21:17:08 hostname [2409002.997008] NMI watchdog: Watchdog detected hard
LOCKUP on cpu 12
Dec 2 21:17:08 hostname
Dec 2 21:17:08 hostname [2409002.999146] Kernel panic - not syncing: Hard
LOCKUP
Dec 2 21:17:08 hostname [2409003.000808] CPU: 12 PID: 3428667 Comm: haproxy
ve: 0 Tainted: P O ------------
3.10.0-962.3.2.lve1.5.26.2.el6h.x86_64 #1 61.16
Dec 2 21:17:08 hostname [2409003.004086] Hardware name: Supermicro
SSG-6039P-E1CR16L/X11DPH-T, BIOS 3.1 05/22/2019
Dec 2 21:17:08 hostname [2409003.005737] Call Trace:
Dec 2 21:17:08 hostname [2409003.007316] <NMI> [<ffffffffa2f45ff9>]
dump_stack+0x19/0x1b
Dec 2 21:17:08 hostname [2409003.008894] [<ffffffffa2f4443a>] panic+0xe6/0x21c
Dec 2 21:17:08 hostname [2409003.010512] [<ffffffffa28954db>]
nmi_panic+0x3b/0x40
Dec 2 21:17:08 hostname [2409003.012081] [<ffffffffa2958daf>]
watchdog_overflow_callback+0x11f/0x140
Dec 2 21:17:08 hostname [2409003.013649] [<ffffffffa299aa77>]
__perf_event_overflow+0x57/0x110
Dec 2 21:17:08 hostname [2409003.015204] [<ffffffffa29a3dd4>]
perf_event_overflow+0x14/0x20
Dec 2 21:17:08 hostname [2409003.016680] [<ffffffffa280aac5>]
intel_pmu_handle_irq+0x245/0x4f0
Dec 2 21:17:08 hostname [2409003.018200] [<ffffffffa29fc69e>] ?
vunmap_page_range+0x24e/0x450
Dec 2 21:17:08 hostname [2409003.019636] [<ffffffffa2c34ec6>] ?
ghes_copy_tofrom_phys+0x116/0x2c0
Dec 2 21:17:08 hostname [2409003.021053] [<ffffffffa2c35110>] ?
ghes_read_estatus+0xa0/0x190
Dec 2 21:17:08 hostname [2409003.022447] [<ffffffffa2f4e030>]
perf_event_nmi_handler+0x30/0x50
Dec 2 21:17:08 hostname [2409003.023867] [<ffffffffa2f4f928>]
nmi_handle.constprop.0+0x78/0x130
Dec 2 21:17:08 hostname [2409003.025272] [<ffffffffa2f4fb61>]
do_nmi+0x181/0x480
Dec 2 21:17:08 hostname [2409003.026649] [<ffffffffa2f4ed89>]
end_repeat_nmi+0x1e/0x81
Dec 2 21:17:08 hostname [2409003.028002] [<ffffffffa291e9f2>] ?
native_queued_spin_lock_slowpath+0x1d2/0x220
Dec 2 21:17:08 hostname [2409003.029297] [<ffffffffa291e9f2>] ?
native_queued_spin_lock_slowpath+0x1d2/0x220
Dec 2 21:17:08 hostname [2409003.030558] [<ffffffffa291e9f2>] ?
native_queued_spin_lock_slowpath+0x1d2/0x220
Dec 2 21:17:08 hostname [2409003.031781] <EOE> [<ffffffffa291ebdb>]
queued_write_lock_slowpath+0x8b/0x90
Dec 2 21:17:08 hostname [2409003.033003] [<ffffffffa2f4ce69>]
_raw_qwrite_lock+0x29/0x30
Dec 2 21:17:08 hostname [2409003.034228] [<ffffffffa2891e9b>]
tasklist_write_lock_irq+0x3b/0x50
Dec 2 21:17:08 hostname [2409003.035447] [<ffffffffa289c872>]
do_exit+0x452/0xac0
Dec 2 21:17:08 hostname [2409003.036647] [<ffffffffa28c630b>] ?
hrtimer_try_to_cancel.part.27+0x5b/0x100
Dec 2 21:17:08 hostname [2409003.037828] [<ffffffffa289cf63>]
do_group_exit+0x43/0xb0
Dec 2 21:17:08 hostname [2409003.038982] [<ffffffffa28ae3dc>]
get_signal_to_deliver+0x28c/0x630
Dec 2 21:17:08 hostname [2409003.040084] [<ffffffffa282b4d7>]
do_signal+0x57/0x710
Dec 2 21:17:08 hostname [2409003.041161] [<ffffffffa2a9d627>] ?
ep_poll+0x347/0x3d0
Dec 2 21:17:08 hostname [2409003.042210] [<ffffffffa28c3485>] ?
cpu_clock_sample+0x85/0xb0
Dec 2 21:17:08 hostname [2409003.043280] [<ffffffffa28d8000>] ?
wake_up_state+0x20/0x20
Dec 2 21:17:08 hostname [2409003.044329] [<ffffffffa282bc02>]
do_notify_resume+0x72/0xd0
Dec 2 21:17:08 hostname [2409003.045336] [<ffffffffa2f585b4>]
int_signal+0x12/0x17
Dec 2 21:17:08 hostname [2409003.059190] NMI watchdog: Watchdog detected hard
LOCKUP on cpu 17
Dec 2 21:17:08 hostname
Dec 2 21:17:08 hostname [2409003.059225] Modules linked in: cls_fw sch_sfq
sch_htb kcare(O) arc4 md4 nls_utf8 cifs dns_resolver xt_comment ip6t_REJECT
nf_reject_ipv6 ip_set_hash_ipport ip_set_hash_ip xt_set ip_set_hash_net
nf_conntrack_ipv6 nf_defrag_ipv6 nf_log_ipv6 nf_log_ipv4 nf_log_common ip_set
nfnetlink ip6table_filter xt_owner nf_nat_ftp xt_REDIRECT nf_nat_redirect
xt_conntrack nf_conntrack_ftp ipt_REJECT nf_reject_ipv4 xt_LOG xt_limit
iptable_filter xt_multiport iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4
nf_nat_ipv4 nf_nat nf_conntrack libcrc32c netconsole 8021q mrp garp stp llc
bonding kmodlve(O) vzdev ip6table_mangle ip6_tables iptable_mangle xt_mark
ip_tables ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm
ib_core binfmt_misc iTCO_wdt iTCO_vendor_support pcspkr zfs(PO) zcommon(PO)
znvpair(PO) icp(PO) spl(O) zavl(PO) zunicode(PO) ipmi_devintf ipmi_si
ipmi_msghandler ses enclosure i2c_i801 shpchp lpc_ich joydev mei_me mei sg nfit
libnvdimm acpi_power_meter acpi_pad acpi_cpufreq ext4 jbd2 mbcache raid1 sd_mod
crc_t10dif crct10dif_common mpt3sas scsi_transport_sas raid_class i40e ptp
pps_core ahci libahci libata wmi ast ttm drm_kms_helper drm i2c_algo_bit
i2c_core dm_mirror dm_region_hash dm_log dm_mod
Dec 2 21:17:08 hostname [2409003.070286] CPU: 17 PID: 3428639 Comm: haproxy
ve: 0 Tainted: P O ------------
3.10.0-962.3.2.lve1.5.26.2.el6h.x86_64 #1 61.16
Dec 2 21:17:08 hostname [2409003.072324] Hardware name: Supermicro
SSG-6039P-E1CR16L/X11DPH-T, BIOS 3.1 05/22/2019
Dec 2 21:17:08 hostname [2409003.073366] Call Trace:
Dec 2 21:17:08 hostname [2409003.074367] <NMI>
Dec 2 21:17:08 hostname [<ffffffffa2f45ff9>] dump_stack+0x19/0x1b
Dec 2 21:17:08 hostname [2409003.075348] [<ffffffffa2958dc1>]
watchdog_overflow_callback+0x131/0x140
Dec 2 21:17:08 hostname [2409003.076364] [<ffffffffa299aa77>]
__perf_event_overflow+0x57/0x110
Dec 2 21:17:08 hostname [2409003.077337] [<ffffffffa29a3dd4>]
perf_event_overflow+0x14/0x20
Dec 2 21:17:08 hostname [2409003.078341] [<ffffffffa280aac5>]
intel_pmu_handle_irq+0x245/0x4f0
Dec 2 21:17:08 hostname [2409003.079361] [<ffffffffa29fc69e>] ?
vunmap_page_range+0x24e/0x450
Dec 2 21:17:08 hostname [2409003.080354] [<ffffffffa2c34ec6>] ?
ghes_copy_tofrom_phys+0x116/0x2c0
Dec 2 21:17:08 hostname [2409003.081364] [<ffffffffa2c35110>] ?
ghes_read_estatus+0xa0/0x190
Dec 2 21:17:08 hostname [2409003.082373] [<ffffffffa2f4e030>]
perf_event_nmi_handler+0x30/0x50
Dec 2 21:17:08 hostname [2409003.083365] [<ffffffffa2f4f928>]
nmi_handle.constprop.0+0x78/0x130
Dec 2 21:17:08 hostname [2409003.084333] [<ffffffffa2f4fb61>]
do_nmi+0x181/0x480
Dec 2 21:17:08 hostname [2409003.085312] [<ffffffffa2f4ed89>]
end_repeat_nmi+0x1e/0x81
Dec 2 21:17:08 hostname [2409003.086278] [<ffffffffa291e927>] ?
native_queued_spin_lock_slowpath+0x107/0x220
Dec 2 21:17:08 hostname [2409003.087289] [<ffffffffa291e927>] ?
native_queued_spin_lock_slowpath+0x107/0x220
Dec 2 21:17:08 hostname [2409003.088278] [<ffffffffa291e927>] ?
native_queued_spin_lock_slowpath+0x107/0x220
Dec 2 21:17:08 hostname [2409003.089254] <EOE> [<ffffffffa291ebdb>]
queued_write_lock_slowpath+0x8b/0x90
Dec 2 21:17:08 hostname [2409003.090220] [<ffffffffa2f4ce69>]
_raw_qwrite_lock+0x29/0x30
Dec 2 21:17:08 hostname [2409003.091217] [<ffffffffa2891e9b>]
tasklist_write_lock_irq+0x3b/0x50
Dec 2 21:17:08 hostname [2409003.092203] [<ffffffffa289c872>]
do_exit+0x452/0xac0
Dec 2 21:17:08 hostname [2409003.093183] [<ffffffffa28c630b>] ?
hrtimer_try_to_cancel.part.27+0x5b/0x100
Dec 2 21:17:08 hostname [2409003.094205] [<ffffffffa289cf63>]
do_group_exit+0x43/0xb0
Dec 2 21:17:08 hostname [2409003.095176] [<ffffffffa28ae3dc>]
get_signal_to_deliver+0x28c/0x630
Dec 2 21:17:08 hostname [2409003.096152] [<ffffffffa282b4d7>]
do_signal+0x57/0x710
Dec 2 21:17:08 hostname [2409003.097120] [<ffffffffa2a9d627>] ?
ep_poll+0x347/0x3d0
Dec 2 21:17:08 hostname [2409003.098123] [<ffffffffa28c3485>] ?
cpu_clock_sample+0x85/0xb0
Dec 2 21:17:08 hostname [2409003.099083] [<ffffffffa28c3400>] ?
cpu_clock_sample_group+0xb0/0xb0
Dec 2 21:17:08 hostname [2409003.100076] [<ffffffffa28d8000>] ?
wake_up_state+0x20/0x20
Dec 2 21:17:08 hostname [2409003.101044] [<ffffffffa282bc02>]
do_notify_resume+0x72/0xd0
Dec 2 21:17:08 hostname [2409003.102024] [<ffffffffa2f585b4>]
int_signal+0x12/0x17
Dec 2 21:17:08 hostname [2409003.148597] NMI watchdog: Watchdog detected hard
LOCKUP on cpu 5
Dec 2 21:17:08 hostname
Dec 2 21:17:08 hostname [2409003.148645] Modules linked in: cls_fw sch_sfq
sch_htb kcare(O) arc4 md4 nls_utf8 cifs dns_resolver xt_comment ip6t_REJECT
nf_reject_ipv6 ip_set_hash_ipport ip_set_hash_ip xt_set ip_set_hash_net
nf_conntrack_ipv6 nf_defrag_ipv6 nf_log_ipv6 nf_log_ipv4 nf_log_common ip_set
nfnetlink ip6table_filter xt_owner nf_nat_ftp xt_REDIRECT nf_nat_redirect
xt_conntrack nf_conntrack_ftp ipt_REJECT nf_reject_ipv4 xt_LOG xt_limit
iptable_filter xt_multiport iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4
nf_nat_ipv4 nf_nat nf_conntrack libcrc32c netconsole 8021q mrp garp stp llc
bonding kmodlve(O) vzdev ip6table_mangle ip6_tables iptable_mangle xt_mark
ip_tables ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm
ib_core binfmt_misc iTCO_wdt iTCO_vendor_support pcspkr zfs(PO) zcommon(PO)
znvpair(PO) icp(PO) spl(O) zavl(PO) zunicode(PO) ipmi_devintf ipmi_si
ipmi_msghandler ses enclosure i2c_i801 shpchp lpc_ich joydev mei_me mei sg nfit
libnvdimm acpi_power_meter acpi_pad acpi_cpufreq ext4 jbd2 mbcache raid1 sd_mod
crc_t10dif crct10dif_common mpt3sas scsi_transport_sas raid_class i40e ptp
pps_core ahci libahci libata wmi ast ttm drm_kms_helper drm i2c_algo_bit
i2c_core dm_mirror dm_region_hash dm_log dm_mod
Dec 2 21:17:08 hostname
Dec 2 21:17:08 hostname [2409003.160602] CPU: 5 PID: 3428663 Comm: haproxy ve:
0 Tainted: P O ------------
3.10.0-962.3.2.lve1.5.26.2.el6h.x86_64 #1 61.16
Dec 2 21:17:08 hostname [2409003.162699] Hardware name: Supermicro
SSG-6039P-E1CR16L/X11DPH-T, BIOS 3.1 05/22/2019
Dec 2 21:17:08 hostname [2409003.163735] Call Trace:
Dec 2 21:17:08 hostname [2409003.164725] <NMI> [<ffffffffa2f45ff9>]
dump_stack+0x19/0x1b
Dec 2 21:17:08 hostname [2409003.165727] [<ffffffffa2958dc1>]
watchdog_overflow_callback+0x131/0x140
Dec 2 21:17:08 hostname [2409003.166712] [<ffffffffa299aa77>]
__perf_event_overflow+0x57/0x110
Dec 2 21:17:08 hostname [2409003.167714] [<ffffffffa29a3dd4>]
perf_event_overflow+0x14/0x20
Dec 2 21:17:08 hostname [2409003.168698] [<ffffffffa280aac5>]
intel_pmu_handle_irq+0x245/0x4f0
Dec 2 21:17:08 hostname [2409003.169690] [<ffffffffa29fc69e>] ?
vunmap_page_range+0x24e/0x450
Dec 2 21:17:08 hostname [2409003.170659] [<ffffffffa2c34ec6>] ?
ghes_copy_tofrom_phys+0x116/0x2c0
Dec 2 21:17:08 hostname [2409003.171625] [<ffffffffa2c35110>] ?
ghes_read_estatus+0xa0/0x190
Dec 2 21:17:08 hostname [2409003.172627] [<ffffffffa2f4e030>]
perf_event_nmi_handler+0x30/0x50
Dec 2 21:17:08 hostname [2409003.173605] [<ffffffffa2f4f928>]
nmi_handle.constprop.0+0x78/0x130
Dec 2 21:17:08 hostname [2409003.174566] [<ffffffffa2f4fb61>]
do_nmi+0x181/0x480
Dec 2 21:17:08 hostname [2409003.175526] [<ffffffffa2f4ed89>]
end_repeat_nmi+0x1e/0x81
Dec 2 21:17:08 hostname [2409003.176524] [<ffffffffa291e9f2>] ?
native_queued_spin_lock_slowpath+0x1d2/0x220
Dec 2 21:17:08 hostname [2409003.177531] [<ffffffffa291e9f2>] ?
native_queued_spin_lock_slowpath+0x1d2/0x220
Dec 2 21:17:08 hostname [2409003.178487] [<ffffffffa291e9f2>] ?
native_queued_spin_lock_slowpath+0x1d2/0x220
Dec 2 21:17:08 hostname [2409003.179433] <EOE> [<ffffffffa291ebdb>]
queued_write_lock_slowpath+0x8b/0x90
Dec 2 21:17:08 hostname [2409003.180408] [<ffffffffa2f4ce69>]
_raw_qwrite_lock+0x29/0x30
Dec 2 21:17:08 hostname [2409003.181373] [<ffffffffa2891e9b>]
tasklist_write_lock_irq+0x3b/0x50
Dec 2 21:17:08 hostname [2409003.182373] [<ffffffffa289c872>]
do_exit+0x452/0xac0
Dec 2 21:17:08 hostname [2409003.183378] [<ffffffffa28c630b>] ?
hrtimer_try_to_cancel.part.27+0x5b/0x100
Dec 2 21:17:08 hostname [2409003.184384] [<ffffffffa289cf63>]
do_group_exit+0x43/0xb0
Dec 2 21:17:08 hostname [2409003.185358] [<ffffffffa28ae3dc>]
get_signal_to_deliver+0x28c/0x630
Dec 2 21:17:08 hostname [2409003.186346] [<ffffffffa282b4d7>]
do_signal+0x57/0x710
Dec 2 21:17:08 hostname [2409003.187336] [<ffffffffa2a9d627>] ?
ep_poll+0x347/0x3d0
Dec 2 21:17:08 hostname [2409003.188314] [<ffffffffa28c3485>] ?
cpu_clock_sample+0x85/0xb0
Dec 2 21:17:08 hostname [2409003.189270] [<ffffffffa28c3640>] ?
posix_cpu_clock_get+0x190/0x190
Dec 2 21:17:08 hostname [2409003.190225] [<ffffffffa28d8000>] ?
wake_up_state+0x20/0x20
Dec 2 21:17:08 hostname [2409003.191220] [<ffffffffa282bc02>]
do_notify_resume+0x72/0xd0
Dec 2 21:17:08 hostname [2409003.192211] [<ffffffffa2f585b4>]
int_signal+0x12/0x17
Dec 2 21:17:08 hostname [2409003.882222] NMI watchdog: Watchdog detected hard
LOCKUP on cpu 35
Dec 2 21:17:08 hostname
Dec 2 21:17:08 hostname [2409003.882256] Modules linked in: cls_fw sch_sfq
sch_htb kcare(O) arc4 md4 nls_utf8 cifs dns_resolver xt_comment ip6t_REJECT
nf_reject_ipv6 ip_set_hash_ipport ip_set_hash_ip xt_set ip_set_hash_net
nf_conntrack_ipv6 nf_defrag_ipv6 nf_log_ipv6 nf_log_ipv4 nf_log_common ip_set
nfnetlink ip6table_filter xt_owner nf_nat_ftp xt_REDIRECT nf_nat_redirect
xt_conntrack nf_conntrack_ftp ipt_REJECT nf_reject_ipv4 xt_LOG xt_limit
iptable_filter xt_multiport iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4
nf_nat_ipv4 nf_nat nf_conntrack libcrc32c netconsole 8021q mrp garp stp llc
bonding kmodlve(O) vzdev ip6table_mangle ip6_tables iptable_mangle xt_mark
ip_tables ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm
ib_core binfmt_misc iTCO_wdt iTCO_vendor_support pcspkr zfs(PO) zcommon(PO)
znvpair(PO) icp(PO) spl(O) zavl(PO) zunicode(PO) ipmi_devintf ipmi_si
ipmi_msghandler ses enclosure i2c_i801 shpchp lpc_ich joydev mei_me mei sg nfit
libnvdimm acpi_power_meter acpi_pad acpi_cpufreq ext4 jbd2 mbcache raid1 sd_mod
crc_t10dif crct10dif_common mpt3sas scsi_transport_sas raid_class i40e ptp
pps_core ahci libahci libata wmi ast ttm drm_kms_helper drm i2c_algo_bit
i2c_core dm_mirror dm_region_hash dm_log dm_mod
Dec 2 21:17:08 hostname
Dec 2 21:17:08 hostname [2409003.894343] CPU: 35 PID: 3428675 Comm: haproxy
ve: 0 Tainted: P O ------------
3.10.0-962.3.2.lve1.5.26.2.el6h.x86_64 #1 61.16
Dec 2 21:17:08 hostname [2409003.896479] Hardware name: Supermicro
SSG-6039P-E1CR16L/X11DPH-T, BIOS 3.1 05/22/2019
Dec 2 21:17:08 hostname [2409003.897541] Call Trace:
Dec 2 21:17:08 hostname [2409003.898540] <NMI> [<ffffffffa2f45ff9>]
dump_stack+0x19/0x1b
Dec 2 21:17:08 hostname [2409003.899569] [<ffffffffa2958dc1>]
watchdog_overflow_callback+0x131/0x140
Dec 2 21:17:08 hostname [2409003.900597] [<ffffffffa299aa77>]
__perf_event_overflow+0x57/0x110
Dec 2 21:17:08 hostname [2409003.901602] [<ffffffffa29a3dd4>]
perf_event_overflow+0x14/0x20
Dec 2 21:17:08 hostname [2409003.902599] [<ffffffffa280aac5>]
intel_pmu_handle_irq+0x245/0x4f0
Dec 2 21:17:08 hostname [2409003.903581] [<ffffffffa29fc69e>] ?
vunmap_page_range+0x24e/0x450
Dec 2 21:17:08 hostname [2409003.904595] [<ffffffffa2c34ec6>] ?
ghes_copy_tofrom_phys+0x116/0x2c0
Dec 2 21:17:08 hostname [2409003.905583] [<ffffffffa2c35110>] ?
ghes_read_estatus+0xa0/0x190
Dec 2 21:17:08 hostname [2409003.906591] [<ffffffffa2f4e030>]
perf_event_nmi_handler+0x30/0x50
Dec 2 21:17:08 hostname [2409003.907565] [<ffffffffa2f4f928>]
nmi_handle.constprop.0+0x78/0x130
Dec 2 21:17:08 hostname [2409003.908570] [<ffffffffa2f4fb61>]
do_nmi+0x181/0x480
Dec 2 21:17:08 hostname [2409003.909574] [<ffffffffa2f4ed89>]
end_repeat_nmi+0x1e/0x81
Dec 2 21:17:08 hostname [2409003.910581] [<ffffffffa291e9f2>] ?
native_queued_spin_lock_slowpath+0x1d2/0x220
Dec 2 21:17:08 hostname [2409003.911561] [<ffffffffa291e9f2>] ?
native_queued_spin_lock_slowpath+0x1d2/0x220
Dec 2 21:17:08 hostname [2409003.912562] [<ffffffffa291e9f2>] ?
native_queued_spin_lock_slowpath+0x1d2/0x220
Dec 2 21:17:08 hostname [2409003.913515] <EOE> [<ffffffffa291ebdb>]
queued_write_lock_slowpath+0x8b/0x90
Dec 2 21:17:08 hostname [2409003.914480] [<ffffffffa2f4ce69>]
_raw_qwrite_lock+0x29/0x30
Dec 2 21:17:08 hostname [2409003.915488] [<ffffffffa2891e9b>]
tasklist_write_lock_irq+0x3b/0x50
Dec 2 21:17:08 hostname [2409003.916515] [<ffffffffa289c872>]
do_exit+0x452/0xac0
Dec 2 21:17:08 hostname [2409003.917492] [<ffffffffa28c630b>] ?
hrtimer_try_to_cancel.part.27+0x5b/0x100
Dec 2 21:17:08 hostname [2409003.918493] [<ffffffffa289cf63>]
do_group_exit+0x43/0xb0
Dec 2 21:17:08 hostname [2409003.919491] [<ffffffffa28ae3dc>]
get_signal_to_deliver+0x28c/0x630
Dec 2 21:17:08 hostname [2409003.920470] [<ffffffffa282b4d7>]
do_signal+0x57/0x710
Dec 2 21:17:08 hostname [2409003.921453] [<ffffffffa2a9d627>] ?
ep_poll+0x347/0x3d0
Dec 2 21:17:08 hostname [2409003.922418] [<ffffffffa28c3485>] ?
cpu_clock_sample+0x85/0xb0
Dec 2 21:17:08 hostname [2409003.923387] [<ffffffffa28d8000>] ?
wake_up_state+0x20/0x20
Dec 2 21:17:08 hostname [2409003.924359] [<ffffffffa282bc02>]
do_notify_resume+0x72/0xd0
Dec 2 21:17:08 hostname [2409003.925351] [<ffffffffa2f585b4>]
int_signal+0x12/0x17
Dec 2 21:17:08 hostname [2409003.933957] NMI watchdog: Watchdog detected hard
LOCKUP on cpu 30
Dec 2 21:17:08 hostname
Dec 2 21:17:08 hostname [2409003.933989] Modules linked in: cls_fw sch_sfq
sch_htb kcare(O) arc4 md4 nls_utf8 cifs dns_resolver xt_comment ip6t_REJECT
nf_reject_ipv6 ip_set_hash_ipport ip_set_hash_ip xt_set ip_set_hash_net
nf_conntrack_ipv6 nf_defrag_ipv6 nf_log_ipv6 nf_log_ipv4 nf_log_common ip_set
nfnetlink ip6table_filter xt_owner nf_nat_ftp xt_REDIRECT nf_nat_redirect
xt_conntrack nf_conntrack_ftp ipt_REJECT nf_reject_ipv4 xt_LOG xt_limit
iptable_filter xt_multiport iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4
nf_nat_ipv4 nf_nat nf_conntrack libcrc32c netconsole 8021q mrp garp stp llc
bonding kmodlve(O) vzdev ip6table_mangle ip6_tables iptable_mangle xt_mark
ip_tables ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm
ib_core binfmt_misc iTCO_wdt iTCO_vendor_support pcspkr zfs(PO) zcommon(PO)
znvpair(PO) icp(PO) spl(O) zavl(PO) zunicode(PO) ipmi_devintf ipmi_si
ipmi_msghandler ses enclosure i2c_i801 shpchp lpc_ich joydev mei_me mei sg nfit
libnvdimm acpi_power_meter acpi_pad acpi_cpufreq ext4 jbd2 mbcache raid1 sd_mod
crc_t10dif crct10dif_common mpt3sas scsi_transport_sas raid_class i40e ptp
pps_core ahci libahci libata wmi ast ttm drm_kms_helper drm i2c_algo_bit
i2c_core dm_mirror dm_region_hash dm_log dm_mod
Dec 2 21:17:09 hostname
Dec 2 21:17:09 hostname [2409003.946127] CPU: 30 PID: 3428642 Comm: haproxy
ve: 0 Tainted: P O ------------
3.10.0-962.3.2.lve1.5.26.2.el6h.x86_64 #1 61.16
Dec 2 21:17:09 hostname [2409003.948290] Hardware name: Supermicro
SSG-6039P-E1CR16L/X11DPH-T, BIOS 3.1 05/22/2019
Dec 2 21:17:09 hostname [2409003.949376] Call Trace:
Dec 2 21:17:09 hostname [2409003.950396] <NMI> [<ffffffffa2f45ff9>]
dump_stack+0x19/0x1b
Dec 2 21:17:09 hostname [2409003.951452] [<ffffffffa2958dc1>]
watchdog_overflow_callback+0x131/0x140
Dec 2 21:17:09 hostname [2409003.952490] [<ffffffffa299aa77>]
__perf_event_overflow+0x57/0x110
Dec 2 21:17:09 hostname [2409003.953483] [<ffffffffa29a3dd4>]
perf_event_overflow+0x14/0x20
Dec 2 21:17:09 hostname [2409003.954498] [<ffffffffa280aac5>]
intel_pmu_handle_irq+0x245/0x4f0
Dec 2 21:17:09 hostname [2409003.955515] [<ffffffffa29fc69e>] ?
vunmap_page_range+0x24e/0x450
Dec 2 21:17:09 hostname [2409003.956492] [<ffffffffa2c34ec6>] ?
ghes_copy_tofrom_phys+0x116/0x2c0
Dec 2 21:17:09 hostname [2409003.957464] [<ffffffffa2c35110>] ?
ghes_read_estatus+0xa0/0x190
Dec 2 21:17:09 hostname [2409003.958458] [<ffffffffa2f4e030>]
perf_event_nmi_handler+0x30/0x50
Dec 2 21:17:09 hostname [2409003.959459] [<ffffffffa2f4f928>]
nmi_handle.constprop.0+0x78/0x130
Dec 2 21:17:09 hostname [2409003.960463] [<ffffffffa2f4fb61>]
do_nmi+0x181/0x480
Dec 2 21:17:09 hostname [2409003.961464] [<ffffffffa2f4ed89>]
end_repeat_nmi+0x1e/0x81
Dec 2 21:17:09 hostname [2409003.962449] [<ffffffffa291e9f5>] ?
native_queued_spin_lock_slowpath+0x1d5/0x220
Dec 2 21:17:09 hostname [2409003.963446] [<ffffffffa291e9f5>] ?
native_queued_spin_lock_slowpath+0x1d5/0x220
Dec 2 21:17:09 hostname [2409003.964412] [<ffffffffa291e9f5>] ?
native_queued_spin_lock_slowpath+0x1d5/0x220
Dec 2 21:17:09 hostname [2409003.965378] <EOE> [<ffffffffa291ebdb>]
queued_write_lock_slowpath+0x8b/0x90
Dec 2 21:17:09 hostname [2409003.966363] [<ffffffffa2f4ce69>]
_raw_qwrite_lock+0x29/0x30
Dec 2 21:17:09 hostname [2409003.967321] [<ffffffffa2891e9b>]
tasklist_write_lock_irq+0x3b/0x50
Dec 2 21:17:09 hostname [2409003.968315] [<ffffffffa289c872>]
do_exit+0x452/0xac0
Dec 2 21:17:09 hostname [2409003.969336] [<ffffffffa28c630b>] ?
hrtimer_try_to_cancel.part.27+0x5b/0x100
Dec 2 21:17:09 hostname [2409003.970322] [<ffffffffa289cf63>]
do_group_exit+0x43/0xb0
Dec 2 21:17:09 hostname [2409003.971296] [<ffffffffa28ae3dc>]
get_signal_to_deliver+0x28c/0x630
Dec 2 21:17:09 hostname [2409003.972277] [<ffffffffa282b4d7>]
do_signal+0x57/0x710
Dec 2 21:17:09 hostname [2409003.973281] [<ffffffffa2a9d627>] ?
ep_poll+0x347/0x3d0
Dec 2 21:17:09 hostname [2409003.974256] [<ffffffffa28c3485>] ?
cpu_clock_sample+0x85/0xb0
Dec 2 21:17:09 hostname [2409003.975227] [<ffffffffa28d8000>] ?
wake_up_state+0x20/0x20
Dec 2 21:17:09 hostname [2409003.976198] [<ffffffffa282bc02>]
do_notify_resume+0x72/0xd0
Dec 2 21:17:09 hostname [2409003.977174] [<ffffffffa2f585b4>]
int_signal+0x12/0x17
Dec 2 21:17:09 hostname [2409004.089821] Shutting down cpus with NMI
Dec 2 21:17:09 hostname [2409004.101232] Kernel Offset: 0x21800000 from
0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
Dec 2 21:17:09 hostname [2409004.194444] Rebooting in 30 seconds..
Dec 2 21:17:39 hostname
Dec 2 21:17:39 hostname [2409034.186961] ACPI MEMORY or I/O RESET_REG.
