Hello again,
Here is another issue found in the same setup, looks a bit different.
Please see below.
BR,
--
Juhamatti
[ 6276.191737] Unable to remove publication from failed node
[ 6276.191737] (type=18888, lower=43818, node=0x100136c,
ref=936542876, key=936542877)
[ 6276.205086] BUG: unable to handle kernel NULL pointer dereference
at 0000000000000088
[ 6276.213016] IP: __call_rcu.constprop.69+0x25/0x210
[ 6276.217837] PGD 0 P4D 0
[ 6276.220405] Oops: 0002 [#1] SMP NOPTI
[ 6276.224103] Modules linked in: tipc ip6_udp_tunnel udp_tunnel
xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4
iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat
nf_conntrack xt_tcpudp bridge stp llc veth iptable_filter ipmi_ssif
nls_iso8859_1 intel_rapl pnd2_edac x86_pkg_temp_thermal
intel_powerclamp coretemp kvm_intel asix kvm usbnet input_leds
irqbypass joydev intel_cstate mii intel_rapl_perf mac_hid ipmi_si
ipmi_devintf shpchp ipmi_msghandler sch_fq_codel ib_iser rdma_cm iw_cm
ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi
ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456
async_raid6_recov async_memcpy async_pq async_xor async_tx xor
raid6_pq libcrc32c raid1 raid0 multipath linear hid_generic usbhid hid
crct10dif_pclmul crc32_pclmul
[ 6276.295019] ghash_clmulni_intel pcbc ast ttm aesni_intel
drm_kms_helper aes_x86_64 syscopyarea crypto_simd glue_helper
sysfillrect sysimgblt cryptd ixgbe fb_sys_fops igb i40e drm ahci dca
i2c_algo_bit mdio libahci ptp i2c_ismt pps_core
[ 6276.316114] CPU: 15 PID: 100 Comm: ksoftirqd/15 Not tainted
4.15.0-38-generic #41-Ubuntu
[ 6276.324259] Hardware name: Supermicro Super Server/A2SDV-16C-TLN5F,
BIOS 1.0b 12/12/2017
[ 6276.332413] RIP: 0010:__call_rcu.constprop.69+0x25/0x210
[ 6276.337773] RSP: 0018:ffffb215c34cbbd0 EFLAGS: 00010246
[ 6276.343033] RAX: 0000000000000076 RBX: ffff9a5046cdd110 RCX: 0000000000000001
[ 6276.350213] RDX: ffffffffbe4a9200 RSI: 0000000000000080 RDI: 0000000000000000
[ 6276.357402] RBP: ffffb215c34cbc08 R08: 0000000000000000 R09: 0000000000001550
[ 6276.364595] R10: 00000000000012df R11: 00000000ffffffff R12: ffff9a4fe3bbfc80
[ 6276.371778] R13: ffff9a5221a54740 R14: ffffffffbe4a9200 R15: 0000000000000080
[ 6276.378963] FS: 0000000000000000(0000) GS:ffff9a561f5c0000(0000)
knlGS:0000000000000000
[ 6276.387096] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 6276.392878] CR2: 0000000000000088 CR3: 00000006f4818000 CR4: 00000000003406e0
[ 6276.400057] Call Trace:
[ 6276.402544] kfree_call_rcu+0x1a/0x20
[ 6276.406256] tipc_publ_notify+0xae/0x190 [tipc]
[ 6276.410840] tipc_node_write_unlock+0xf3/0x100 [tipc]
[ 6276.415979] tipc_node_link_down+0x99/0x140 [tipc]
[ 6276.420858] tipc_node_timeout+0x126/0x130 [tipc]
[ 6276.425644] ? tipc_node_calculate_timer.isra.13+0x60/0x60 [tipc]
[ 6276.431795] call_timer_fn+0x30/0x130
[ 6276.435487] run_timer_softirq+0x3fb/0x450
[ 6276.439614] ? __switch_to_asm+0x34/0x70
[ 6276.443573] ? __switch_to+0xad/0x500
[ 6276.447262] ? __switch_to_asm+0x40/0x70
[ 6276.451219] ? __switch_to_asm+0x34/0x70
[ 6276.455178] ? __switch_to_asm+0x40/0x70
[ 6276.459141] __do_softirq+0xe4/0x2bb
[ 6276.462750] run_ksoftirqd+0x22/0x60
[ 6276.466364] smpboot_thread_fn+0xfc/0x170
[ 6276.470397] kthread+0x121/0x140
[ 6276.473645] ? sort_range+0x30/0x30
[ 6276.477153] ? kthread_create_worker_on_cpu+0x70/0x70
[ 6276.482237] ret_from_fork+0x35/0x40
[ 6276.485835] Code: b7 f8 ff 0f 1f 00 0f 1f 44 00 00 55 48 89 e5 41
57 41 56 41 55 41 54 49 89 ff 53 49 89 d6 48 83 ec 10 83 e7 07 0f 85
dd 00 00 00 <49> 89 77 08 49 c7 07 00 00 00 00 9c 58 0f 1f 44 00 00 49
89 c5
[ 6276.504818] RIP: __call_rcu.constprop.69+0x25/0x210 RSP: ffffb215c34cbbd0
[ 6276.511642] CR2: 0000000000000088
[ 6276.515000] ---[ end trace f418a9afa145bb5d ]---
[ 6276.528130] Kernel panic - not syncing: Fatal exception in interrupt
[ 6276.534610] Kernel Offset: 0x3c000000 from 0xffffffff81000000
(relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 6276.553092] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
[ 6276.560285] WARNING: CPU: 15 PID: 100 at
/build/linux-CuEUJv/linux-4.15.0/kernel/sched/core.c:1192
set_task_cpu+0x158/0x160
[ 6276.571459] Modules linked in: tipc ip6_udp_tunnel udp_tunnel
xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4
iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat
nf_conntrack xt_tcpudp bridge stp llc veth iptable_filter ipmi_ssif
nls_iso8859_1 intel_rapl pnd2_edac x86_pkg_temp_thermal
intel_powerclamp coretemp kvm_intel asix kvm usbnet input_leds
irqbypass joydev intel_cstate mii intel_rapl_perf mac_hid ipmi_si
ipmi_devintf shpchp ipmi_msghandler sch_fq_codel ib_iser rdma_cm iw_cm
ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi
ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456
async_raid6_recov async_memcpy async_pq async_xor async_tx xor
raid6_pq libcrc32c raid1 raid0 multipath linear hid_generic usbhid hid
crct10dif_pclmul crc32_pclmul
[ 6276.642267] ghash_clmulni_intel pcbc ast ttm aesni_intel
drm_kms_helper aes_x86_64 syscopyarea crypto_simd glue_helper
sysfillrect sysimgblt cryptd ixgbe fb_sys_fops igb i40e drm ahci dca
i2c_algo_bit mdio libahci ptp i2c_ismt pps_core
[ 6276.663306] CPU: 15 PID: 100 Comm: ksoftirqd/15 Tainted: G D
4.15.0-38-generic #41-Ubuntu
[ 6276.672734] Hardware name: Supermicro Super Server/A2SDV-16C-TLN5F,
BIOS 1.0b 12/12/2017
[ 6276.683335] RIP: 0010:set_task_cpu+0x158/0x160
[ 6276.690288] RSP: 0018:ffff9a561f5c3dc8 EFLAGS: 00010006
[ 6276.698025] RAX: 0000000000000200 RBX: ffff9a50a7565900 RCX: 0000000000000000
[ 6276.707673] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff9a50a7565900
[ 6276.717305] RBP: ffff9a561f5c3de8 R08: 0000000000000000 R09: 000000000000ffff
[ 6276.726926] R10: ffff9a561f5c3c90 R11: 0000000000000000 R12: ffff9a50a756642c
[ 6276.736527] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000022880
[ 6276.746122] FS: 0000000000000000(0000) GS:ffff9a561f5c0000(0000)
knlGS:0000000000000000
[ 6276.756700] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 6276.764891] CR2: 0000000000000088 CR3: 00000006f4818000 CR4: 00000000003406e0
[ 6276.774418] Call Trace:
[ 6276.779188] <IRQ>
[ 6276.783446] try_to_wake_up+0x159/0x480
[ 6276.789458] default_wake_function+0x12/0x20
[ 6276.795834] autoremove_wake_function+0x12/0x40
[ 6276.802444] __wake_up_common+0x73/0x130
[ 6276.808426] __wake_up_common_lock+0x80/0xc0
[ 6276.814730] __wake_up+0x13/0x20
[ 6276.819987] wake_up_klogd_work_func+0x40/0x60
[ 6276.826444] irq_work_run_list+0x52/0x80
[ 6276.832376] irq_work_run+0x2c/0x40
[ 6276.837876] flush_smp_call_function_queue+0x66/0xf0
[ 6276.844845] generic_smp_call_function_single_interrupt+0x13/0x30
[ 6276.852952] smp_call_function_interrupt+0x3e/0xd0
[ 6276.859743] call_function_interrupt+0x84/0x90
[ 6276.866171] </IRQ>
[ 6276.870234] RIP: 0010:panic+0x1fe/0x244
[ 6276.876021] RSP: 0018:ffffb215c34cb918 EFLAGS: 00000286 ORIG_RAX:
ffffffffffffff03
[ 6276.885572] RAX: 0000000000000041 RBX: 0000000000000000 RCX: 0000000000000006
[ 6276.894699] RDX: 0000000000000000 RSI: 0000000000000092 RDI: ffff9a561f5d6490
[ 6276.903823] RBP: ffffb215c34cb990 R08: 0000000000000000 R09: 000000000000157e
[ 6276.912957] R10: 0000000000000008 R11: 00000000ffffffff R12: 0000000000000000
[ 6276.922071] R13: 0000000000000000 R14: 0000000000000009 R15: 0000000000000002
[ 6276.931172] ? panic+0x1f7/0x244
[ 6276.936358] oops_end+0xb6/0xd0
[ 6276.941463] no_context+0x17c/0x440
[ 6276.946934] ? __const_udelay+0x41/0x50
[ 6276.952753] __bad_area_nosemaphore+0xf3/0x1b0
[ 6276.959186] bad_area_nosemaphore+0x14/0x20
[ 6276.965378] __do_page_fault+0xd4/0x4d0
[ 6276.971226] ? up+0x32/0x50
[ 6276.976035] do_page_fault+0x2e/0xe0
[ 6276.981632] page_fault+0x25/0x50
[ 6276.986954] RIP: 0010:__call_rcu.constprop.69+0x25/0x210
[ 6276.994290] RSP: 0018:ffffb215c34cbbd0 EFLAGS: 00010246
[ 6277.001543] RAX: 0000000000000076 RBX: ffff9a5046cdd110 RCX: 0000000000000001
[ 6277.010723] RDX: ffffffffbe4a9200 RSI: 0000000000000080 RDI: 0000000000000000
[ 6277.019901] RBP: ffffb215c34cbc08 R08: 0000000000000000 R09: 0000000000001550
[ 6277.029015] R10: 00000000000012df R11: 00000000ffffffff R12: ffff9a4fe3bbfc80
[ 6277.038070] R13: ffff9a5221a54740 R14: ffffffffbe4a9200 R15: 0000000000000080
[ 6277.047129] kfree_call_rcu+0x1a/0x20
[ 6277.052708] tipc_publ_notify+0xae/0x190 [tipc]
[ 6277.059161] tipc_node_write_unlock+0xf3/0x100 [tipc]
[ 6277.066123] tipc_node_link_down+0x99/0x140 [tipc]
[ 6277.072792] tipc_node_timeout+0x126/0x130 [tipc]
[ 6277.079348] ? tipc_node_calculate_timer.isra.13+0x60/0x60 [tipc]
[ 6277.087259] call_timer_fn+0x30/0x130
[ 6277.092670] run_timer_softirq+0x3fb/0x450
[ 6277.098438] ? __switch_to_asm+0x34/0x70
[ 6277.103961] ? __switch_to+0xad/0x500
[ 6277.109137] ? __switch_to_asm+0x40/0x70
[ 6277.114493] ? __switch_to_asm+0x34/0x70
[ 6277.119765] ? __switch_to_asm+0x40/0x70
[ 6277.125006] __do_softirq+0xe4/0x2bb
[ 6277.129861] run_ksoftirqd+0x22/0x60
[ 6277.134699] smpboot_thread_fn+0xfc/0x170
[ 6277.139965] kthread+0x121/0x140
[ 6277.144439] ? sort_range+0x30/0x30
[ 6277.149163] ? kthread_create_worker_on_cpu+0x70/0x70
[ 6277.155452] ret_from_fork+0x35/0x40
[ 6277.160258] Code: 32 ff ff ff 80 8b 64 08 00 00 04 e9 4e ff ff ff
0f 0b e9 f2 fe ff ff f7 43 5c fd ff ff ff 0f 84 fc fe ff ff 0f 0b e9
f5 fe ff ff <0f> 0b e9 ff fe ff ff 90 0f 1f 44 00 00 55 48 89 e5 41 56
41 55
[ 6277.181837] ---[ end trace f418a9afa145bb5e ]---
[ 6277.187806] ------------[ cut here ]------------
[ 6277.193765] sched: Unexpected reschedule of offline CPU#0!
[ 6277.200584] WARNING: CPU: 15 PID: 100 at
/build/linux-CuEUJv/linux-4.15.0/arch/x86/kernel/smp.c:128
native_smp_send_reschedule+0x3a/0x40
[ 6277.214247] Modules linked in: tipc ip6_udp_tunnel udp_tunnel
xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4
iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat
nf_conntrack xt_tcpudp bridge stp llc veth iptable_filter ipmi_ssif
nls_iso8859_1 intel_rapl pnd2_edac x86_pkg_temp_thermal
intel_powerclamp coretemp kvm_intel asix kvm usbnet input_leds
irqbypass joydev intel_cstate mii intel_rapl_perf mac_hid ipmi_si
ipmi_devintf shpchp ipmi_msghandler sch_fq_codel ib_iser rdma_cm iw_cm
ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi
ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456
async_raid6_recov async_memcpy async_pq async_xor async_tx xor
raid6_pq libcrc32c raid1 raid0 multipath linear hid_generic usbhid hid
crct10dif_pclmul crc32_pclmul
[ 6277.291150] ghash_clmulni_intel pcbc ast ttm aesni_intel
drm_kms_helper aes_x86_64 syscopyarea crypto_simd glue_helper
sysfillrect sysimgblt cryptd ixgbe fb_sys_fops igb i40e drm ahci dca
i2c_algo_bit mdio libahci ptp i2c_ismt pps_core
[ 6277.315672] CPU: 15 PID: 100 Comm: ksoftirqd/15 Tainted: G D W
4.15.0-38-generic #41-Ubuntu
[ 6277.326902] Hardware name: Supermicro Super Server/A2SDV-16C-TLN5F,
BIOS 1.0b 12/12/2017
[ 6277.336859] RIP: 0010:native_smp_send_reschedule+0x3a/0x40
[ 6277.344224] RSP: 0018:ffff9a561f5c3de8 EFLAGS: 00010086
[ 6277.351330] RAX: 0000000000000000 RBX: ffff9a50a7565900 RCX: 0000000000000006
[ 6277.360358] RDX: 0000000000000007 RSI: 0000000000000092 RDI: ffff9a561f5d6490
[ 6277.369378] RBP: ffff9a561f5c3de8 R08: 0000000000000000 R09: 00000000000015cd
[ 6277.378405] R10: ffff9a561f5c3dc8 R11: 00000000ffffffff R12: ffff9a50a756642c
[ 6277.387428] R13: 0000000000000004 R14: 0000000000000046 R15: 0000000000022880
[ 6277.396489] FS: 0000000000000000(0000) GS:ffff9a561f5c0000(0000)
knlGS:0000000000000000
[ 6277.406505] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 6277.414171] CR2: 0000000000000088 CR3: 00000006f4818000 CR4: 00000000003406e0
[ 6277.423241] Call Trace:
[ 6277.427619] <IRQ>
[ 6277.431550] try_to_wake_up+0x3c6/0x480
[ 6277.437319] default_wake_function+0x12/0x20
[ 6277.443503] autoremove_wake_function+0x12/0x40
[ 6277.449960] __wake_up_common+0x73/0x130
[ 6277.455811] __wake_up_common_lock+0x80/0xc0
[ 6277.462004] __wake_up+0x13/0x20
[ 6277.467146] wake_up_klogd_work_func+0x40/0x60
[ 6277.473510] irq_work_run_list+0x52/0x80
[ 6277.479353] irq_work_run+0x2c/0x40
[ 6277.484753] flush_smp_call_function_queue+0x66/0xf0
[ 6277.491642] generic_smp_call_function_single_interrupt+0x13/0x30
[ 6277.499674] smp_call_function_interrupt+0x3e/0xd0
[ 6277.506405] call_function_interrupt+0x84/0x90
[ 6277.512787] </IRQ>
[ 6277.516805] RIP: 0010:panic+0x1fe/0x244
[ 6277.522558] RSP: 0018:ffffb215c34cb918 EFLAGS: 00000286 ORIG_RAX:
ffffffffffffff03
[ 6277.532076] RAX: 0000000000000041 RBX: 0000000000000000 RCX: 0000000000000006
[ 6277.541160] RDX: 0000000000000000 RSI: 0000000000000092 RDI: ffff9a561f5d6490
[ 6277.550232] RBP: ffffb215c34cb990 R08: 0000000000000000 R09: 000000000000157e
[ 6277.559296] R10: 0000000000000008 R11: 00000000ffffffff R12: 0000000000000000
[ 6277.568356] R13: 0000000000000000 R14: 0000000000000009 R15: 0000000000000002
[ 6277.577422] ? panic+0x1f7/0x244
[ 6277.582578] oops_end+0xb6/0xd0
[ 6277.587644] no_context+0x17c/0x440
[ 6277.593041] ? __const_udelay+0x41/0x50
[ 6277.598779] __bad_area_nosemaphore+0xf3/0x1b0
[ 6277.605124] bad_area_nosemaphore+0x14/0x20
[ 6277.611212] __do_page_fault+0xd4/0x4d0
[ 6277.616958] ? up+0x32/0x50
[ 6277.621667] do_page_fault+0x2e/0xe0
[ 6277.627168] page_fault+0x25/0x50
[ 6277.632417] RIP: 0010:__call_rcu.constprop.69+0x25/0x210
[ 6277.639666] RSP: 0018:ffffb215c34cbbd0 EFLAGS: 00010246
[ 6277.646845] RAX: 0000000000000076 RBX: ffff9a5046cdd110 RCX: 0000000000000001
[ 6277.655951] RDX: ffffffffbe4a9200 RSI: 0000000000000080 RDI: 0000000000000000
[ 6277.665065] RBP: ffffb215c34cbc08 R08: 0000000000000000 R09: 0000000000001550
[ 6277.674176] R10: 00000000000012df R11: 00000000ffffffff R12: ffff9a4fe3bbfc80
[ 6277.683298] R13: ffff9a5221a54740 R14: ffffffffbe4a9200 R15: 0000000000000080
[ 6277.692426] kfree_call_rcu+0x1a/0x20
[ 6277.698025] tipc_publ_notify+0xae/0x190 [tipc]
[ 6277.704432] tipc_node_write_unlock+0xf3/0x100 [tipc]
[ 6277.711361] tipc_node_link_down+0x99/0x140 [tipc]
[ 6277.718027] tipc_node_timeout+0x126/0x130 [tipc]
[ 6277.724591] ? tipc_node_calculate_timer.isra.13+0x60/0x60 [tipc]
[ 6277.732516] call_timer_fn+0x30/0x130
[ 6277.737940] run_timer_softirq+0x3fb/0x450
[ 6277.743701] ? __switch_to_asm+0x34/0x70
[ 6277.749216] ? __switch_to+0xad/0x500
[ 6277.754386] ? __switch_to_asm+0x40/0x70
[ 6277.759732] ? __switch_to_asm+0x34/0x70
[ 6277.764993] ? __switch_to_asm+0x40/0x70
[ 6277.770220] __do_softirq+0xe4/0x2bb
[ 6277.775071] run_ksoftirqd+0x22/0x60
[ 6277.779899] smpboot_thread_fn+0xfc/0x170
[ 6277.785156] kthread+0x121/0x140
[ 6277.789619] ? sort_range+0x30/0x30
[ 6277.794333] ? kthread_create_worker_on_cpu+0x70/0x70
[ 6277.800617] ret_from_fork+0x35/0x40
[ 6277.805419] Code: 2d 61 60 01 73 17 48 8b 05 84 21 16 01 be fd 00
00 00 48 8b 40 30 e8 36 b6 ba 00 5d c3 89 fe 48 c7 c7 60 fe 0b be e8
06 48 03 00 <0f> 0b 5d c3 66 90 0f 1f 44 00 00 55 48 89 e5 53 48 83 ec
20 65
[ 6277.826959] ---[ end trace f418a9afa145bb5f ]---
ke 31. lokak. 2018 klo 14.39 [email protected] kirjoitti:
>
> Hello,
>
> Thanks Ying! While running a server with 250 LXCs and 2500 TIPC
> sockets running with modified connection demo, I am seeing the problem
> described below. The kernel is stock Ubuntu1804 4.15.0-38-generic with
> 16 cores. It is fairly easy to reproduce, after starting up containers
> it will happen within an hour or so.
>
> Please let me know if there is already a patch for it, I am happy to
> try. I can also try fix candidates quite easily.
>
> BR,
> --
> Juhamatti
>
> [ 3195.231191] BUG: unable to handle kernel paging request at 0000000200001dc7
> [ 3195.238220] IP: _raw_spin_lock_bh+0x1b/0x30
> [ 3195.242432] PGD 0 P4D 0
> [ 3195.244988] Oops: 0002 [#1] SMP NOPTI
> [ 3195.248668] Modules linked in: binfmt_misc tipc ip6_udp_tunnel
> udp_tunnel xt_CHECKSUM iptable_mangle ipt_MASQUERADE
> nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4
> nf_nat_ipv4 nf_nat nf_conntrack xt_tcpudp bridge stp llc veth
> iptable_filter ipmi_ssif nls_iso8859_1 intel_rapl pnd2_edac
> x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass
> asix intel_cstate usbnet intel_rapl_perf joydev input_leds mii ipmi_si
> ipmi_devintf ipmi_msghandler mac_hid shpchp sch_fq_codel ib_iser
> rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi
> scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress
> raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor
> async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear
> hid_generic usbhid hid crct10dif_pclmul
> [ 3195.319457] crc32_pclmul ghash_clmulni_intel pcbc ast aesni_intel
> ttm aes_x86_64 drm_kms_helper crypto_simd glue_helper cryptd
> syscopyarea sysfillrect sysimgblt ixgbe igb fb_sys_fops i40e drm ahci
> dca i2c_algo_bit libahci mdio i2c_ismt ptp pps_core
> [ 3195.341632] CPU: 1 PID: 348 Comm: kworker/u32:7 Not tainted
> 4.15.0-38-generic #41-Ubuntu
> [ 3195.349754] Hardware name: Supermicro Super Server/A2SDV-16C-TLN5F,
> BIOS 1.0b 12/12/2017
> [ 3195.357891] Workqueue: tipc_send tipc_send_work [tipc]
> [ 3195.363061] RIP: 0010:_raw_spin_lock_bh+0x1b/0x30
> [ 3195.367788] RSP: 0018:ffff90d3df243e90 EFLAGS: 00010246
> [ 3195.373032] RAX: 0000000000000000 RBX: ffff90d1317a6c18 RCX:
> ffff90d3df243f08
> [ 3195.380203] RDX: 0000000000000001 RSI: ffffffffc07d8290 RDI:
> 0000000200001dc7
> [ 3195.387393] RBP: ffff90d3df243e90 R08: ffff90d3df243f10 R09:
> ffff90d3df25a780
> [ 3195.394561] R10: ffff90d3df243f10 R11: ffff90d3df25a770 R12:
> 0000000200001dc7
> [ 3195.401729] R13: ffff90d1317a6c00 R14: ffffffffc07d8290 R15:
> ffff90d1317a6c18
> [ 3195.408897] FS: 0000000000000000(0000) GS:ffff90d3df240000(0000)
> knlGS:0000000000000000
> [ 3195.417025] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 3195.422798] CR2: 0000000200001dc7 CR3: 00000007711f8000 CR4:
> 00000000003406e0
> [ 3195.429965] Call Trace:
> [ 3195.432429] <IRQ>
> [ 3195.434467] tipc_subscrp_timeout+0x25/0x90 [tipc]
> [ 3195.439311] call_timer_fn+0x30/0x130
> [ 3195.442994] run_timer_softirq+0x3fb/0x450
> [ 3195.447113] ? lapic_next_deadline+0x26/0x30
> [ 3195.451404] ? clockevents_program_event+0xd8/0x110
> [ 3195.456304] __do_softirq+0xe4/0x2bb
> [ 3195.459903] do_softirq_own_stack+0x2a/0x40
> [ 3195.464107] </IRQ>
> [ 3195.466219] do_softirq.part.17+0x50/0x60
> [ 3195.470250] __local_bh_enable_ip+0x50/0x60
> [ 3195.474456] _raw_spin_unlock_bh+0x1e/0x20
> [ 3195.478576] tipc_sk_rcv+0x15a/0x440 [tipc]
> [ 3195.482782] tipc_node_xmit+0x147/0x190 [tipc]
> [ 3195.487254] __tipc_sendstream+0x1a9/0x2d0 [tipc]
> [ 3195.491987] ? aa_sk_perm+0x48/0x1a0
> [ 3195.495587] tipc_sendstream+0x35/0x50 [tipc]
> [ 3195.499976] tipc_send_packet+0x17/0x20 [tipc]
> [ 3195.504454] sock_sendmsg+0x3e/0x50
> [ 3195.507967] kernel_sendmsg+0x2b/0x30
> [ 3195.511663] tipc_send_to_sock+0x12f/0x290 [tipc]
> [ 3195.516395] ? __switch_to_asm+0x40/0x70
> [ 3195.520338] ? __switch_to_asm+0x34/0x70
> [ 3195.524280] ? __switch_to_asm+0x34/0x70
> [ 3195.528222] ? __switch_to_asm+0x40/0x70
> [ 3195.532174] ? __switch_to_asm+0x34/0x70
> [ 3195.536133] tipc_send_work+0x23/0x40 [tipc]
> [ 3195.540433] process_one_work+0x1de/0x410
> [ 3195.544468] worker_thread+0x32/0x410
> [ 3195.548147] kthread+0x121/0x140
> [ 3195.551403] ? process_one_work+0x410/0x410
> [ 3195.555611] ? kthread_create_worker_on_cpu+0x70/0x70
> [ 3195.560686] ? do_syscall_64+0x73/0x130
> [ 3195.564545] ? SyS_exit_group+0x14/0x20
> [ 3195.568404] ret_from_fork+0x35/0x40
> [ 3195.571998] Code: 00 5d c3 31 c0 5d c3 66 0f 1f 84 00 00 00 00 00
> 0f 1f 44 00 00 55 65 81 05 8f 8a c7 73 00 02 00 00 48 89 e5 31 c0 ba
> 01 00 00 00 <f0> 0f b1 17 85 c0 75 02 5d c3 89 c6 e8 24 12 74 ff 66 90
> 5d c3
> [ 3195.590981] RIP: _raw_spin_lock_bh+0x1b/0x30 RSP: ffff90d3df243e90
> [ 3195.599011] CR2: 0000000200001dc7
> [ 3195.604403] ---[ end trace 4ce4054b7f724f81 ]---
> [ 3195.619914] Kernel panic - not syncing: Fatal exception in interrupt
> [ 3195.628264] Kernel Offset: 0xaa00000 from 0xffffffff81000000
> (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
> [ 3195.649393] ---[ end Kernel panic - not syncing: Fatal exception in
> interrupt
> [ 3195.658447] WARNING: CPU: 1 PID: 348 at
> /build/linux-CuEUJv/linux-4.15.0/kernel/sched/core.c:1192
> set_task_cpu+0x158/0x160
> [ 3195.671428] Modules linked in: binfmt_misc tipc ip6_udp_tunnel
> udp_tunnel xt_CHECKSUM iptable_mangle ipt_MASQUERADE
> nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4
> nf_nat_ipv4 nf_nat nf_conntrack xt_tcpudp bridge stp llc veth
> iptable_filter ipmi_ssif nls_iso8859_1 intel_rapl pnd2_edac
> x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass
> asix intel_cstate usbnet intel_rapl_perf joydev input_leds mii ipmi_si
> ipmi_devintf ipmi_msghandler mac_hid shpchp sch_fq_codel ib_iser
> rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi
> scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress
> raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor
> async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear
> hid_generic usbhid hid crct10dif_pclmul
> [ 3195.749731] crc32_pclmul ghash_clmulni_intel pcbc ast aesni_intel
> ttm aes_x86_64 drm_kms_helper crypto_simd glue_helper cryptd
> syscopyarea sysfillrect sysimgblt ixgbe igb fb_sys_fops i40e drm ahci
> dca i2c_algo_bit libahci mdio i2c_ismt ptp pps_core
> [ 3195.775727] CPU: 1 PID: 348 Comm: kworker/u32:7 Tainted: G D
> 4.15.0-38-generic #41-Ubuntu
> [ 3195.787072] Hardware name: Supermicro Super Server/A2SDV-16C-TLN5F,
> BIOS 1.0b 12/12/2017
> [ 3195.797128] Workqueue: tipc_send tipc_send_work [tipc]
> [ 3195.804212] RIP: 0010:set_task_cpu+0x158/0x160
> [ 3195.810603] RSP: 0018:ffff90d3df243808 EFLAGS: 00010006
> [ 3195.817773] RAX: 0000000000000200 RBX: ffff90d2c5b11640 RCX:
> 0000000000000000
> [ 3195.826856] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
> ffff90d2c5b11640
> [ 3195.835936] RBP: ffff90d3df243828 R08: 0000000000000000 R09:
> 000000000000ffff
> [ 3195.845016] R10: 0000000000000010 R11: 0000000000000000 R12:
> ffff90d2c5b1216c
> [ 3195.854101] R13: 0000000000000000 R14: 0000000000000000 R15:
> 0000000000022880
> [ 3195.863174] FS: 0000000000000000(0000) GS:ffff90d3df240000(0000)
> knlGS:0000000000000000
> [ 3195.873202] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 3195.880878] CR2: 0000000200001dc7 CR3: 00000007711f8000 CR4:
> 00000000003406e0
> [ 3195.889943] Call Trace:
> [ 3195.894313] <IRQ>
> [ 3195.898250] try_to_wake_up+0x159/0x480
> [ 3195.904009] default_wake_function+0x12/0x20
> [ 3195.910199] autoremove_wake_function+0x12/0x40
> [ 3195.916660] __wake_up_common+0x73/0x130
> [ 3195.922508] __wake_up_common_lock+0x80/0xc0
> [ 3195.928695] __wake_up+0x13/0x20
> [ 3195.933831] wake_up_klogd_work_func+0x40/0x60
> [ 3195.940185] irq_work_run_list+0x52/0x80
> [ 3195.946026] ? tick_sched_do_timer+0x50/0x50
> [ 3195.952206] irq_work_tick+0x3b/0x50
> [ 3195.957696] update_process_times+0x42/0x60
> [ 3195.963797] tick_sched_handle+0x26/0x70
> [ 3195.969635] ? tick_sched_do_timer+0x50/0x50
> [ 3195.975827] tick_sched_timer+0x39/0x80
> [ 3195.981576] __hrtimer_run_queues+0xdf/0x230
> [ 3195.987756] hrtimer_interrupt+0xa0/0x1d0
> [ 3195.993676] smp_apic_timer_interrupt+0x6f/0x130
> [ 3196.000208] apic_timer_interrupt+0x84/0x90
> [ 3196.006312] RIP: 0010:panic+0x1fe/0x244
> [ 3196.012062] RSP: 0018:ffff90d3df243bd8 EFLAGS: 00000286 ORIG_RAX:
> ffffffffffffff11
> [ 3196.021578] RAX: 0000000000000041 RBX: 0000000000000000 RCX:
> 0000000000000006
> [ 3196.030668] RDX: 0000000000000000 RSI: 0000000000000092 RDI:
> ffff90d3df256490
> [ 3196.039773] RBP: ffff90d3df243c50 R08: 0000000000000000 R09:
> 000000000000113b
> [ 3196.048892] R10: 0000000000000008 R11: 00000000ffffffff R12:
> 0000000000000000
> [ 3196.058026] R13: 0000000000000000 R14: 0000000000000009 R15:
> 0000000000000002
> [ 3196.067174] ? panic+0x1f7/0x244
> [ 3196.072419] oops_end+0xb6/0xd0
> [ 3196.077569] no_context+0x17c/0x440
> [ 3196.083061] __bad_area_nosemaphore+0xf3/0x1b0
> [ 3196.089520] bad_area_nosemaphore+0x14/0x20
> [ 3196.095714] __do_page_fault+0xd4/0x4d0
> [ 3196.101560] ? update_load_avg+0x57f/0x6e0
> [ 3196.107657] do_page_fault+0x2e/0xe0
> [ 3196.113242] page_fault+0x25/0x50
> [ 3196.118560] RIP: 0010:_raw_spin_lock_bh+0x1b/0x30
> [ 3196.125207] RSP: 0018:ffff90d3df243e90 EFLAGS: 00010246
> [ 3196.132299] RAX: 0000000000000000 RBX: ffff90d1317a6c18 RCX:
> ffff90d3df243f08
> [ 3196.141292] RDX: 0000000000000001 RSI: ffffffffc07d8290 RDI:
> 0000000200001dc7
> [ 3196.150282] RBP: ffff90d3df243e90 R08: ffff90d3df243f10 R09:
> ffff90d3df25a780
> [ 3196.159277] R10: ffff90d3df243f10 R11: ffff90d3df25a770 R12:
> 0000000200001dc7
> [ 3196.168262] R13: ffff90d1317a6c00 R14: ffffffffc07d8290 R15:
> ffff90d1317a6c18
> [ 3196.177238] ? tipc_subscrb_release_cb+0x30/0x30 [tipc]
> [ 3196.184271] ? tipc_subscrb_release_cb+0x30/0x30 [tipc]
> [ 3196.191262] tipc_subscrp_timeout+0x25/0x90 [tipc]
> [ 3196.197751] call_timer_fn+0x30/0x130
> [ 3196.203029] run_timer_softirq+0x3fb/0x450
> [ 3196.208673] ? lapic_next_deadline+0x26/0x30
> [ 3196.214423] ? clockevents_program_event+0xd8/0x110
> [ 3196.220707] __do_softirq+0xe4/0x2bb
> [ 3196.225616] do_softirq_own_stack+0x2a/0x40
> [ 3196.231114] </IRQ>
> [ 3196.234491] do_softirq.part.17+0x50/0x60
> [ 3196.239759] __local_bh_enable_ip+0x50/0x60
> [ 3196.245196] _raw_spin_unlock_bh+0x1e/0x20
> [ 3196.250555] tipc_sk_rcv+0x15a/0x440 [tipc]
> [ 3196.255986] tipc_node_xmit+0x147/0x190 [tipc]
> [ 3196.261677] __tipc_sendstream+0x1a9/0x2d0 [tipc]
> [ 3196.267615] ? aa_sk_perm+0x48/0x1a0
> [ 3196.272420] tipc_sendstream+0x35/0x50 [tipc]
> [ 3196.278014] tipc_send_packet+0x17/0x20 [tipc]
> [ 3196.283673] sock_sendmsg+0x3e/0x50
> [ 3196.288359] kernel_sendmsg+0x2b/0x30
> [ 3196.293212] tipc_send_to_sock+0x12f/0x290 [tipc]
> [ 3196.299094] ? __switch_to_asm+0x40/0x70
> [ 3196.304205] ? __switch_to_asm+0x34/0x70
> [ 3196.309301] ? __switch_to_asm+0x34/0x70
> [ 3196.314379] ? __switch_to_asm+0x40/0x70
> [ 3196.319446] ? __switch_to_asm+0x34/0x70
> [ 3196.324510] tipc_send_work+0x23/0x40 [tipc]
> [ 3196.329931] process_one_work+0x1de/0x410
> [ 3196.335103] worker_thread+0x32/0x410
> [ 3196.339919] kthread+0x121/0x140
> [ 3196.344299] ? process_one_work+0x410/0x410
> [ 3196.349640] ? kthread_create_worker_on_cpu+0x70/0x70
> [ 3196.355853] ? do_syscall_64+0x73/0x130
> [ 3196.360851] ? SyS_exit_group+0x14/0x20
> [ 3196.365841] ret_from_fork+0x35/0x40
> [ 3196.370567] Code: 32 ff ff ff 80 8b 64 08 00 00 04 e9 4e ff ff ff
> 0f 0b e9 f2 fe ff ff f7 43 5c fd ff ff ff 0f 84 fc fe ff ff 0f 0b e9
> f5 fe ff ff <0f> 0b e9 ff fe ff ff 90 0f 1f 44 00 00 55 48 89 e5 41 56
> 41 55
> [ 3196.391999] ---[ end trace 4ce4054b7f724f82 ]---
> [ 3196.397922] ------------[ cut here ]------------
> [ 3196.403838] sched: Unexpected reschedule of offline CPU#0!
> [ 3196.410627] WARNING: CPU: 1 PID: 348 at
> /build/linux-CuEUJv/linux-4.15.0/arch/x86/kernel/smp.c:128
> native_smp_send_reschedule+0x3a/0x40
> [ 3196.424161] Modules linked in: binfmt_misc tipc ip6_udp_tunnel
> udp_tunnel xt_CHECKSUM iptable_mangle ipt_MASQUERADE
> nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4
> nf_nat_ipv4 nf_nat nf_conntrack xt_tcpudp bridge stp llc veth
> iptable_filter ipmi_ssif nls_iso8859_1 intel_rapl pnd2_edac
> x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass
> asix intel_cstate usbnet intel_rapl_perf joydev input_leds mii ipmi_si
> ipmi_devintf ipmi_msghandler mac_hid shpchp sch_fq_codel ib_iser
> rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi
> scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress
> raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor
> async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear
> hid_generic usbhid hid crct10dif_pclmul
> [ 3196.500686] crc32_pclmul ghash_clmulni_intel pcbc ast aesni_intel
> ttm aes_x86_64 drm_kms_helper crypto_simd glue_helper cryptd
> syscopyarea sysfillrect sysimgblt ixgbe igb fb_sys_fops i40e drm ahci
> dca i2c_algo_bit libahci mdio i2c_ismt ptp pps_core
> [ 3196.526181] CPU: 1 PID: 348 Comm: kworker/u32:7 Tainted: G D W
> 4.15.0-38-generic #41-Ubuntu
> [ 3196.537321] Hardware name: Supermicro Super Server/A2SDV-16C-TLN5F,
> BIOS 1.0b 12/12/2017
> [ 3196.547206] Workqueue: tipc_send tipc_send_work [tipc]
> [ 3196.554142] RIP: 0010:native_smp_send_reschedule+0x3a/0x40
> [ 3196.561423] RSP: 0018:ffff90d3df243790 EFLAGS: 00010082
> [ 3196.568445] RAX: 0000000000000000 RBX: 0000000000000000 RCX:
> 0000000000000006
> [ 3196.577395] RDX: 0000000000000007 RSI: 0000000000000086 RDI:
> ffff90d3df256490
> [ 3196.586344] RBP: ffff90d3df243790 R08: 0000000000000000 R09:
> 000000000000119e
> [ 3196.595297] R10: 000000000000025f R11: 00000000ffffffff R12:
> ffff90d3df222880
> [ 3196.604262] R13: ffff90d2c5b11640 R14: ffff90d3df243848 R15:
> ffff90d3df222880
> [ 3196.613232] FS: 0000000000000000(0000) GS:ffff90d3df240000(0000)
> knlGS:0000000000000000
> [ 3196.623178] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 3196.630768] CR2: 0000000200001dc7 CR3: 00000007711f8000 CR4:
> 00000000003406e0
> [ 3196.639761] Call Trace:
> [ 3196.644061] <IRQ>
> [ 3196.647923] resched_curr+0x5d/0xc0
> [ 3196.653276] check_preempt_curr+0x7a/0x90
> [ 3196.659130] ttwu_do_wakeup+0x1e/0x140
> [ 3196.664745] ttwu_do_activate+0x77/0x80
> [ 3196.670433] try_to_wake_up+0x1d6/0x480
> [ 3196.676117] default_wake_function+0x12/0x20
> [ 3196.682237] autoremove_wake_function+0x12/0x40
> [ 3196.688617] __wake_up_common+0x73/0x130
> [ 3196.694389] __wake_up_common_lock+0x80/0xc0
> [ 3196.700494] __wake_up+0x13/0x20
> [ 3196.705549] wake_up_klogd_work_func+0x40/0x60
> [ 3196.711831] irq_work_run_list+0x52/0x80
> [ 3196.717598] ? tick_sched_do_timer+0x50/0x50
> [ 3196.723707] irq_work_tick+0x3b/0x50
> [ 3196.729118] update_process_times+0x42/0x60
> [ 3196.735141] tick_sched_handle+0x26/0x70
> [ 3196.740900] ? tick_sched_do_timer+0x50/0x50
> [ 3196.747014] tick_sched_timer+0x39/0x80
> [ 3196.752684] __hrtimer_run_queues+0xdf/0x230
> [ 3196.758789] hrtimer_interrupt+0xa0/0x1d0
> [ 3196.764640] smp_apic_timer_interrupt+0x6f/0x130
> [ 3196.771101] apic_timer_interrupt+0x84/0x90
> [ 3196.777130] RIP: 0010:panic+0x1fe/0x244
> [ 3196.782802] RSP: 0018:ffff90d3df243bd8 EFLAGS: 00000286 ORIG_RAX:
> ffffffffffffff11
> [ 3196.792240] RAX: 0000000000000041 RBX: 0000000000000000 RCX:
> 0000000000000006
> [ 3196.801259] RDX: 0000000000000000 RSI: 0000000000000092 RDI:
> ffff90d3df256490
> [ 3196.810293] RBP: ffff90d3df243c50 R08: 0000000000000000 R09:
> 000000000000113b
> [ 3196.819342] R10: 0000000000000008 R11: 00000000ffffffff R12:
> 0000000000000000
> [ 3196.828408] R13: 0000000000000000 R14: 0000000000000009 R15:
> 0000000000000002
> [ 3196.837484] ? panic+0x1f7/0x244
> [ 3196.842654] oops_end+0xb6/0xd0
> [ 3196.847725] no_context+0x17c/0x440
> [ 3196.853142] __bad_area_nosemaphore+0xf3/0x1b0
> [ 3196.859519] bad_area_nosemaphore+0x14/0x20
> [ 3196.865635] __do_page_fault+0xd4/0x4d0
> [ 3196.871401] ? update_load_avg+0x57f/0x6e0
> [ 3196.877434] do_page_fault+0x2e/0xe0
> [ 3196.882942] page_fault+0x25/0x50
> [ 3196.888121] RIP: 0010:_raw_spin_lock_bh+0x1b/0x30
> [ 3196.894624] RSP: 0018:ffff90d3df243e90 EFLAGS: 00010246
> [ 3196.901641] RAX: 0000000000000000 RBX: ffff90d1317a6c18 RCX:
> ffff90d3df243f08
> [ 3196.910595] RDX: 0000000000000001 RSI: ffffffffc07d8290 RDI:
> 0000000200001dc7
> [ 3196.919541] RBP: ffff90d3df243e90 R08: ffff90d3df243f10 R09:
> ffff90d3df25a780
> [ 3196.928459] R10: ffff90d3df243f10 R11: ffff90d3df25a770 R12:
> 0000000200001dc7
> [ 3196.937309] R13: ffff90d1317a6c00 R14: ffffffffc07d8290 R15:
> ffff90d1317a6c18
> [ 3196.946097] ? tipc_subscrb_release_cb+0x30/0x30 [tipc]
> [ 3196.952912] ? tipc_subscrb_release_cb+0x30/0x30 [tipc]
> [ 3196.959636] tipc_subscrp_timeout+0x25/0x90 [tipc]
> [ 3196.965850] call_timer_fn+0x30/0x130
> [ 3196.970861] run_timer_softirq+0x3fb/0x450
> [ 3196.976282] ? lapic_next_deadline+0x26/0x30
> [ 3196.981856] ? clockevents_program_event+0xd8/0x110
> [ 3196.988004] __do_softirq+0xe4/0x2bb
> [ 3196.992839] do_softirq_own_stack+0x2a/0x40
> [ 3196.998277] </IRQ>
> [ 3197.001619] do_softirq.part.17+0x50/0x60
> [ 3197.006875] __local_bh_enable_ip+0x50/0x60
> [ 3197.012300] _raw_spin_unlock_bh+0x1e/0x20
> [ 3197.017639] tipc_sk_rcv+0x15a/0x440 [tipc]
> [ 3197.023057] tipc_node_xmit+0x147/0x190 [tipc]
> [ 3197.028715] __tipc_sendstream+0x1a9/0x2d0 [tipc]
> [ 3197.034618] ? aa_sk_perm+0x48/0x1a0
> [ 3197.039384] tipc_sendstream+0x35/0x50 [tipc]
> [ 3197.044919] tipc_send_packet+0x17/0x20 [tipc]
> [ 3197.050539] sock_sendmsg+0x3e/0x50
> [ 3197.055207] kernel_sendmsg+0x2b/0x30
> [ 3197.060044] tipc_send_to_sock+0x12f/0x290 [tipc]
> [ 3197.065927] ? __switch_to_asm+0x40/0x70
> [ 3197.071048] ? __switch_to_asm+0x34/0x70
> [ 3197.076162] ? __switch_to_asm+0x34/0x70
> [ 3197.081257] ? __switch_to_asm+0x40/0x70
> [ 3197.086345] ? __switch_to_asm+0x34/0x70
> [ 3197.091425] tipc_send_work+0x23/0x40 [tipc]
> [ 3197.096854] process_one_work+0x1de/0x410
> [ 3197.102030] worker_thread+0x32/0x410
> [ 3197.106854] kthread+0x121/0x140
> [ 3197.111239] ? process_one_work+0x410/0x410
> [ 3197.116576] ? kthread_create_worker_on_cpu+0x70/0x70
> [ 3197.122788] ? do_syscall_64+0x73/0x130
> [ 3197.127786] ? SyS_exit_group+0x14/0x20
> [ 3197.132784] ret_from_fork+0x35/0x40
> [ 3197.137508] Code: 2d 61 60 01 73 17 48 8b 05 84 21 16 01 be fd 00
> 00 00 48 8b 40 30 e8 36 b6 ba 00 5d c3 89 fe 48 c7 c7 60 fe ab 8c e8
> 06 48 03 00 <0f> 0b 5d c3 66 90 0f 1f 44 00 00 55 48 89 e5 53 48 83 ec
> 20 65
> [ 3197.158946] ---[ end trace 4ce4054b7f724f83 ]---
> [ 3197.164963] unchecked MSR access error: WRMSR to 0x83f (tried to
> write 0x00000000000000f6) at rIP: 0xffffffff8ba6c1a8
> (native_write_msr+0x8/0x30)
> [ 3197.179360] Call Trace:
> [ 3197.183165] <IRQ>
> [ 3197.186532] native_apic_msr_write+0x2b/0x40
> [ 3197.192143] x2apic_send_IPI_self+0x20/0x30
> [ 3197.197660] arch_irq_work_raise+0x2a/0x40
> [ 3197.203080] irq_work_queue+0x8d/0xa0
> [ 3197.208058] dbs_update_util_handler+0x80/0xa0
> [ 3197.213822] update_load_avg+0x5c5/0x6e0
> [ 3197.219068] enqueue_entity+0x5e/0x6b0
> [ 3197.224124] enqueue_task_fair+0x6c/0x7f0
> [ 3197.229449] ? __update_load_avg_blocked_se.isra.37+0xd1/0x150
> [ 3197.236608] activate_task+0x54/0xc0
> [ 3197.241510] ? sched_clock+0x9/0x10
> [ 3197.246338] ttwu_do_activate+0x49/0x80
> [ 3197.251514] try_to_wake_up+0x1d6/0x480
> [ 3197.256684] default_wake_function+0x12/0x20
> [ 3197.262288] autoremove_wake_function+0x12/0x40
> [ 3197.268162] __wake_up_common+0x73/0x130
> [ 3197.273428] __wake_up_common_lock+0x80/0xc0
> [ 3197.279026] __wake_up+0x13/0x20
> [ 3197.283562] wake_up_klogd_work_func+0x40/0x60
> [ 3197.289321] irq_work_run_list+0x52/0x80
> [ 3197.294545] ? tick_sched_do_timer+0x50/0x50
> [ 3197.300116] irq_work_tick+0x3b/0x50
> [ 3197.304987] update_process_times+0x42/0x60
> [ 3197.310455] tick_sched_handle+0x26/0x70
> [ 3197.315672] ? tick_sched_do_timer+0x50/0x50
> [ 3197.321233] tick_sched_timer+0x39/0x80
> [ 3197.326351] __hrtimer_run_queues+0xdf/0x230
> [ 3197.331909] hrtimer_interrupt+0xa0/0x1d0
> [ 3197.337205] smp_apic_timer_interrupt+0x6f/0x130
> [ 3197.343108] apic_timer_interrupt+0x84/0x90
> [ 3197.348574] RIP: 0010:panic+0x1fe/0x244
> [ 3197.353674] RSP: 0018:ffff90d3df243bd8 EFLAGS: 00000286 ORIG_RAX:
> ffffffffffffff11
> [ 3197.362537] RAX: 0000000000000041 RBX: 0000000000000000 RCX:
> 0000000000000006
> [ 3197.370972] RDX: 0000000000000000 RSI: 0000000000000092 RDI:
> ffff90d3df256490
> [ 3197.379405] RBP: ffff90d3df243c50 R08: 0000000000000000 R09:
> 000000000000113b
> [ 3197.387843] R10: 0000000000000008 R11: 00000000ffffffff R12:
> 0000000000000000
> [ 3197.396291] R13: 0000000000000000 R14: 0000000000000009 R15:
> 0000000000000002
> [ 3197.404743] ? panic+0x1f7/0x244
> [ 3197.409283] oops_end+0xb6/0xd0
> [ 3197.413730] no_context+0x17c/0x440
> [ 3197.418521] __bad_area_nosemaphore+0xf3/0x1b0
> [ 3197.424273] bad_area_nosemaphore+0x14/0x20
> [ 3197.429763] __do_page_fault+0xd4/0x4d0
> [ 3197.434912] ? update_load_avg+0x57f/0x6e0
> [ 3197.440318] do_page_fault+0x2e/0xe0
> [ 3197.445203] page_fault+0x25/0x50
> [ 3197.449827] RIP: 0010:_raw_spin_lock_bh+0x1b/0x30
> [ 3197.455862] RSP: 0018:ffff90d3df243e90 EFLAGS: 00010246
> [ 3197.462432] RAX: 0000000000000000 RBX: ffff90d1317a6c18 RCX:
> ffff90d3df243f08
> [ 3197.470940] RDX: 0000000000000001 RSI: ffffffffc07d8290 RDI:
> 0000000200001dc7
> [ 3197.479454] RBP: ffff90d3df243e90 R08: ffff90d3df243f10 R09:
> ffff90d3df25a780
> [ 3197.487973] R10: ffff90d3df243f10 R11: ffff90d3df25a770 R12:
> 0000000200001dc7
> [ 3197.496495] R13: ffff90d1317a6c00 R14: ffffffffc07d8290 R15:
> ffff90d1317a6c18
> [ 3197.505043] ? tipc_subscrb_release_cb+0x30/0x30 [tipc]
> [ 3197.511689] ? tipc_subscrb_release_cb+0x30/0x30 [tipc]
> [ 3197.518319] tipc_subscrp_timeout+0x25/0x90 [tipc]
> [ 3197.524511] call_timer_fn+0x30/0x130
> [ 3197.529573] run_timer_softirq+0x3fb/0x450
> [ 3197.535073] ? lapic_next_deadline+0x26/0x30
> [ 3197.540684] ? clockevents_program_event+0xd8/0x110
> [ 3197.546832] __do_softirq+0xe4/0x2bb
> [ 3197.551680] do_softirq_own_stack+0x2a/0x40
> [ 3197.557086] </IRQ>
> [ 3197.560362] do_softirq.part.17+0x50/0x60
> [ 3197.565561] __local_bh_enable_ip+0x50/0x60
> [ 3197.570937] _raw_spin_unlock_bh+0x1e/0x20
> [ 3197.576228] tipc_sk_rcv+0x15a/0x440 [tipc]
> [ 3197.581602] tipc_node_xmit+0x147/0x190 [tipc]
> [ 3197.587245] __tipc_sendstream+0x1a9/0x2d0 [tipc]
> [ 3197.593144] ? aa_sk_perm+0x48/0x1a0
> [ 3197.597909] tipc_sendstream+0x35/0x50 [tipc]
> [ 3197.603461] tipc_send_packet+0x17/0x20 [tipc]
> [ 3197.609101] sock_sendmsg+0x3e/0x50
> [ 3197.613767] kernel_sendmsg+0x2b/0x30
> [ 3197.618609] tipc_send_to_sock+0x12f/0x290 [tipc]
> [ 3197.624497] ? __switch_to_asm+0x40/0x70
> [ 3197.629601] ? __switch_to_asm+0x34/0x70
> [ 3197.634697] ? __switch_to_asm+0x34/0x70
> [ 3197.639781] ? __switch_to_asm+0x40/0x70
> [ 3197.644846] ? __switch_to_asm+0x34/0x70
> [ 3197.649909] tipc_send_work+0x23/0x40 [tipc]
> [ 3197.655317] process_one_work+0x1de/0x410
> [ 3197.660470] worker_thread+0x32/0x410
> [ 3197.665274] kthread+0x121/0x140
> [ 3197.669636] ? process_one_work+0x410/0x410
> [ 3197.674955] ? kthread_create_worker_on_cpu+0x70/0x70
> [ 3197.681146] ? do_syscall_64+0x73/0x130
> [ 3197.686118] ? SyS_exit_group+0x14/0x20
> [ 3197.691094] ret_from_fork+0x35/0x40
> pe 26. lokak. 2018 klo 6.38 Ying Xue ([email protected]) kirjoitti:
> >
> > Hi Juhamatti,
> >
> > This is a good test plan indeed. If you encounter any problem during the
> > testing, please share it here.
> >
> > From my view, TIPC had fully supported networking nemespace at least
> > three years ago and its sockets could be fully isolated between containers.
> >
> > Thanks,
> > Ying
> >
> > On 10/26/2018 11:12 AM, [email protected] wrote:
> > > Hello,
> > >
> > > I'm planning to test TIPC scalability in more detail, especially
> > > regarding the binding table and topology (service tracking) service
> > > with large number of nodes (~1k) and sockets (100-1k). Is it enough to
> > > do it with LXC containers with isolated namespaces to get realistic
> > > results? My concern here is that if TIPC implementation would not
> > > provide full isolation to sockets e.g. for performance reasons inside
> > > the same kernel, then this may not provide results matching to real
> > > environments. VMs are the other choice, but they tend to be heavy. Are
> > > the namespaces fully isolated inside the kernel?
> > >
> > > Thanks for the info,
> > > --
> > > Juhamatti
> > >
> > >
> > > _______________________________________________
> > > tipc-discussion mailing list
> > > [email protected]
> > > https://lists.sourceforge.net/lists/listinfo/tipc-discussion
> > >
_______________________________________________
tipc-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/tipc-discussion
