Hi everyone!
I had this panic in the recent days.
Attached the ddb output (traces, ps) and the dmesg.
Unfortunately I wasn't around when it happened, I have no clue what went on
with the machine then.
Reading through the traces it has some in6* stuff in it, this box has IPv6 set
up, if this is helpful.
Dani
OpenBSD 6.5-stable (GENERIC.MP) #1: Thu May 16 16:36:18 CEST 2019
daniell@firefly:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 4261203968 (4063MB)
avail mem = 4122423296 (3931MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xdffd7020 (7 entries)
bios0: vendor coreboot version "v4.0.24" date 02/04/2019
bios0: PC Engines apu4
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S2 S3 S4 S5
acpi0: tables DSDT FACP SSDT APIC HEST SSDT SSDT HPET
acpi0: wakeup devices PWRB(S4) PBR4(S4) PBR5(S4) PBR6(S4) PBR7(S4) PBR8(S4)
UOH1(S3) UOH3(S3) UOH5(S3) XHC0(S4)
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD GX-412TC SOC, 998.27 MHz, 16-30-01
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TOPEXT,DBKP,PERFTSC,PCTRL3,ITSC,BMI1,XSAVEOPT
cpu0: 32KB 64b/line 2-way I-cache, 32KB 64b/line 8-way D-cache, 2MB 64b/line
16-way L2 cache
cpu0: ITLB 32 4KB entries fully associative, 8 4MB entries fully associative
cpu0: DTLB 40 4KB entries fully associative, 8 4MB entries fully associative
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD GX-412TC SOC, 998.13 MHz, 16-30-01
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TOPEXT,DBKP,PERFTSC,PCTRL3,ITSC,BMI1,XSAVEOPT
cpu1: 32KB 64b/line 2-way I-cache, 32KB 64b/line 8-way D-cache, 2MB 64b/line
16-way L2 cache
cpu1: ITLB 32 4KB entries fully associative, 8 4MB entries fully associative
cpu1: DTLB 40 4KB entries fully associative, 8 4MB entries fully associative
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 2 (application processor)
cpu2: AMD GX-412TC SOC, 998.13 MHz, 16-30-01
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TOPEXT,DBKP,PERFTSC,PCTRL3,ITSC,BMI1,XSAVEOPT
cpu2: 32KB 64b/line 2-way I-cache, 32KB 64b/line 8-way D-cache, 2MB 64b/line
16-way L2 cache
cpu2: ITLB 32 4KB entries fully associative, 8 4MB entries fully associative
cpu2: DTLB 40 4KB entries fully associative, 8 4MB entries fully associative
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 3 (application processor)
cpu3: AMD GX-412TC SOC, 998.13 MHz, 16-30-01
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TOPEXT,DBKP,PERFTSC,PCTRL3,ITSC,BMI1,XSAVEOPT
cpu3: 32KB 64b/line 2-way I-cache, 32KB 64b/line 8-way D-cache, 2MB 64b/line
16-way L2 cache
cpu3: ITLB 32 4KB entries fully associative, 8 4MB entries fully associative
cpu3: DTLB 40 4KB entries fully associative, 8 4MB entries fully associative
cpu3: smt 0, core 3, package 0
ioapic0 at mainbus0: apid 4 pa 0xfec00000, version 21, 24 pins
ioapic1 at mainbus0: apid 5 pa 0xfec20000, version 21, 32 pins, remapped
acpihpet0 at acpi0: 14318180 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (PBR4)
acpiprt2 at acpi0: bus 2 (PBR5)
acpiprt3 at acpi0: bus 3 (PBR6)
acpiprt4 at acpi0: bus 4 (PBR7)
acpiprt5 at acpi0: bus 5 (PBR8)
acpicpu0 at acpi0: C2(0@400 io@0x1771), C1(@1 halt!), PSS
acpicpu1 at acpi0: C2(0@400 io@0x1771), C1(@1 halt!), PSS
acpicpu2 at acpi0: C2(0@400 io@0x1771), C1(@1 halt!), PSS
acpicpu3 at acpi0: C2(0@400 io@0x1771), C1(@1 halt!), PSS
acpibtn0 at acpi0: PWRB
acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001
acpicmos0 at acpi0
cpu0: 998 MHz: speeds: 1000 800 600 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "AMD AMD64 16h Root Complex" rev 0x00
pchb1 at pci0 dev 2 function 0 "AMD AMD64 16h Host" rev 0x00
ppb0 at pci0 dev 2 function 1 "AMD AMD64 16h PCIE" rev 0x00: msi
pci1 at ppb0 bus 1
em0 at pci1 dev 0 function 0 "Intel I211" rev 0x03: msi, address
ppb1 at pci0 dev 2 function 2 "AMD AMD64 16h PCIE" rev 0x00: msi
pci2 at ppb1 bus 2
em1 at pci2 dev 0 function 0 "Intel I211" rev 0x03: msi, address
ppb2 at pci0 dev 2 function 3 "AMD AMD64 16h PCIE" rev 0x00: msi
pci3 at ppb2 bus 3
em2 at pci3 dev 0 function 0 "Intel I211" rev 0x03: msi, address
ppb3 at pci0 dev 2 function 4 "AMD AMD64 16h PCIE" rev 0x00: msi
pci4 at ppb3 bus 4
em3 at pci4 dev 0 function 0 "Intel I211" rev 0x03: msi, address
ppb4 at pci0 dev 2 function 5 "AMD AMD64 16h PCIE" rev 0x00: msi
pci5 at ppb4 bus 5
athn0 at pci5 dev 0 function 0 "Atheros AR9281" rev 0x01: apic 5 int 16
athn0: AR9280 rev 2 (2T2R), ROM rev 22, address
ccp0 at pci0 dev 8 function 0 "AMD Cryptographic Co-processor v3" rev 0x00
xhci0 at pci0 dev 16 function 0 "AMD Bolton xHCI" rev 0x11: msi, xHCI 1.0
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "AMD xHCI root hub" rev 3.00/1.00
addr 1
ahci0 at pci0 dev 17 function 0 "AMD Hudson-2 SATA" rev 0x40: apic 4 int 19,
AHCI 1.3
ahci0: port 0: 6.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, SATA SSD, SBFM> SCSI3 0/direct fixed
naa.0000000000000000
sd0: 57241MB, 512 bytes/sector, 117231408 sectors, thin
ehci0 at pci0 dev 18 function 0 "AMD Hudson-2 USB2" rev 0x39: apic 4 int 18
usb1 at ehci0: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "AMD EHCI root hub" rev 2.00/1.00
addr 1
ehci1 at pci0 dev 19 function 0 "AMD Hudson-2 USB2" rev 0x39: apic 4 int 18
usb2 at ehci1: USB revision 2.0
uhub2 at usb2 configuration 1 interface 0 "AMD EHCI root hub" rev 2.00/1.00
addr 1
piixpm0 at pci0 dev 20 function 0 "AMD Hudson-2 SMBus" rev 0x42: SMBus disabled
pcib0 at pci0 dev 20 function 3 "AMD Hudson-2 LPC" rev 0x11
sdhc0 at pci0 dev 20 function 7 "AMD Bolton SD/MMC" rev 0x01: apic 4 int 16
sdhc0: SDHC 2.0, 50 MHz base clock
sdmmc0 at sdhc0: 4-bit, sd high-speed, mmc high-speed, dma
pchb2 at pci0 dev 24 function 0 "AMD AMD64 16h Link Cfg" rev 0x00
pchb3 at pci0 dev 24 function 1 "AMD AMD64 16h Address Map" rev 0x00
pchb4 at pci0 dev 24 function 2 "AMD AMD64 16h DRAM Cfg" rev 0x00
km0 at pci0 dev 24 function 3 "AMD AMD64 16h Misc Cfg" rev 0x00
pchb5 at pci0 dev 24 function 4 "AMD AMD64 16h CPU Power" rev 0x00
pchb6 at pci0 dev 24 function 5 "AMD AMD64 16h Misc Cfg" rev 0x00
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
com2 at isa0 port 0x3e8/8 irq 5: ns16550a, 16 byte fifo
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
wbsio0 at isa0 port 0x2e/2: NCT5104D rev 0x53
vmm0 at mainbus0: SVM/RVI
uhidev0 at uhub0 port 3 configuration 1 interface 0 "American Power Conversion
Back-UPS XS 1400U FW:926.T1 .I USB FW:T1" rev 1.10/1.06 addr 2
uhidev0: iclass 3/0, 123 report ids
upd0 at uhidev0
uhub3 at uhub1 port 1 configuration 1 interface 0 "Advanced Micro Devices
product 0x7900" rev 2.00/0.18 addr 2
uhub4 at uhub2 port 1 configuration 1 interface 0 "Advanced Micro Devices
product 0x7900" rev 2.00/0.18 addr 2
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (338ef9ca14372769.a) swap on sd0b dump on sd0b
WARNING: / was not properly unmounted
pppoe0: received unexpected PADO
pppoe0: received unexpected PADO
pppoe0: received unexpected PADO
cannot forward src fe80:b::20e:c6ff:feb3:e9ad, dst 2001:4860:4860::6464, nxt
17, rcvif 11, outif 10
__mtx_enter_try(ffffffff81d81cc0) at __mtx_enter_try+0x11
end trace frame: 0xffff80002252b050, count: 0
ddb{1}> trace
__mtx_enter_try(ffffffff81d81cc0) at __mtx_enter_try+0x11
__mtx_enter(ffffffff81d81cc0) at __mtx_enter+0x3a
uvm_pmr_getpages(1,0,100000,1,0,1) at uvm_pmr_getpages+0x1cd
uvm_pglistalloc(1000,0,ffffffff,0,0,ffff80002252b268) at uvm_pglistalloc+0x27b
km_alloc(1000,ffff80002252b2e8,ffffffff81c32d48,ffff80002252b300) at km_alloc+0
x269
pool_multi_alloc(ffffffff81d34950,2,ffff80002252b4d4) at pool_multi_alloc+0xb0
m_pool_alloc(ffffffff81d34950,2,ffff80002252b4d4) at m_pool_alloc+0x8a
pool_p_alloc(ffffffff81d34950,2,ffff80002252b4d4) at pool_p_alloc+0x5a
pool_do_get(ffffffff81d34950,2,ffff80002252b4d4) at pool_do_get+0xd3
pool_get(ffffffff81d34950,2) at pool_get+0x93
m_copym(fffffd80d32d6b00,0,3b9aca00,2) at m_copym+0xe4
rtm_sendup(fffffd81193eb7c8,fffffd80d32d6b00,1) at rtm_sendup+0x52
route_input(fffffd80d32d6b00,0,18) at route_input+0x127
rtm_send(fffffd811a905940,2,0,0) at rtm_send+0x12e
rtdeletemsg(fffffd811a905940,ffff80000040b000,0) at rtdeletemsg+0x118
rtflushclone1(fffffd811a905940,fffffd811a905550,0) at rtflushclone1+0x87
rtable_walk_helper(fffffd811d90c7a0,ffff80002252ce40) at rtable_walk_helper+0x5
3
art_table_walk(ffff8000000a58c0,fffffd811ad5f468,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x1d9
art_table_walk(ffff8000000a58c0,fffffd811ad5f448,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5f3a8,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5f388,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5fb68,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5fba8,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5fbc8,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5fbe8,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5fc08,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5fc88,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5f728,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811983b9b0,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811983b270,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811983b230,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811983b1f0,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d202600,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2028e0,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2028c0,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2028a0,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d202880,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2027c0,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2027a0,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d202700,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2026e0,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2628,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2608,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b25c8,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2588,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2568,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2508,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d90b540,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d90bf60,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_walk(ffff8000000a58c0,ffffffff81a094d0,ffff80002252ce40) at art_walk+0xd3
rtable_walk(0,18,ffffffff813d28c0,fffffd811a905550) at rtable_walk+0xa2
rtrequest_delete(ffff80002252cf38,4,ffff80000040b000,ffff80002252cff8,0) at rtr
equest_delete+0x115
rtdeletemsg(fffffd811a905550,ffff80000040b000,0) at rtdeletemsg+0xfc
rt_ifa_purge_walker(fffffd811a905550,ffff800000cdaa00,0) at rt_ifa_purge_walker
+0x27
rtable_walk_helper(fffffd811d90ca90,ffff80002252db60) at rtable_walk_helper+0x5
3
art_table_walk(ffff8000000a58c0,fffffd811d2028e0,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x1d9
art_table_walk(ffff8000000a58c0,fffffd811d2028c0,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2028a0,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d202880,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2027c0,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2027a0,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d202700,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2026e0,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2628,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2608,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b25c8,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2588,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2568,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2508,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d90b540,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d90bf60,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_walk(ffff8000000a58c0,ffffffff81a094d0,ffff80002252db60) at art_walk+0xd3
rtable_walk(0,18,ffffffff813d35a0,ffff800000cdaa00) at rtable_walk+0xa2
rt_ifa_purge(ffff800000cdaa00) at rt_ifa_purge+0x7f
in6_unlink_ifa(ffff800000cdaa00,ffff80000040b000) at in6_unlink_ifa+0x324
in6_purgeaddr(ffff800000cdaa00) at in6_purgeaddr+0x11e
in6_ioctl_change_ifaddr(81206919,ffff800000cbce00,ffff80000040b000) at in6_ioct
l_change_ifaddr+0x156
ifioctl(fffffd81180a2dd8,81206919,ffff800000cbce00,ffff8000222ae018) at ifioctl
+0x2b7
sys_ioctl(ffff8000222ae018,ffff80002252df00,ffff80002252df70) at sys_ioctl+0x3c
4
syscall(ffff80002252dfe0) at syscall+0x305
Xsyscall(6,36,421d5b03200,36,421d5b03200,d) at Xsyscall+0x128
end of kernel
end trace frame: 0x7f7fffff7430, count: -81
ddb{1}> mach ddbcpu 0
Stopped at x86_ipi_db+0x12: leave
ddb{0}> trace
x86_ipi_db(ffffffff81cbaff0) at x86_ipi_db+0x12
x86_ipi_handler() at x86_ipi_handler+0x80
Xresume_lapic_ipi(7,ffffffff81cbaff0,ffff80000007e600,0,ffff80000007e600,ffff80
00222c6660) at Xresume_lapic_ipi+0x23
___mp_lock(ffffffff81d3473c) at ___mp_lock+0x62
intr_handler(ffff8000222c6660,ffff80000007e600) at intr_handler+0x44
Xintr_ioapic_level0_untramp(0,ffffffff81cbaff0,7f7ffffe1918,0,790,ffff8000ffff8
e18) at Xintr_ioapic_level0_untramp+0x1a3
_kernel_lock(ffff8000222c6810,ffff8000222c6810) at _kernel_lock+0xa2
Xsyscall(0,79,4,79,3,37853638cc0) at Xsyscall+0x128
end of kernel
end trace frame: 0x7f7ffffe1930, count: -8
ddb{0}> mach ddbcpu 1
Stopped at __mtx_enter_try+0x11: pushq %r11
ddb{1}> trace
__mtx_enter_try(ffffffff81d81cc0) at __mtx_enter_try+0x11
__mtx_enter(ffffffff81d81cc0) at __mtx_enter+0x3a
uvm_pmr_getpages(1,0,100000,1,0,1) at uvm_pmr_getpages+0x1cd
uvm_pglistalloc(1000,0,ffffffff,0,0,ffff80002252b268) at uvm_pglistalloc+0x27b
km_alloc(1000,ffff80002252b2e8,ffffffff81c32d48,ffff80002252b300) at km_alloc+0
x269
pool_multi_alloc(ffffffff81d34950,2,ffff80002252b4d4) at pool_multi_alloc+0xb0
m_pool_alloc(ffffffff81d34950,2,ffff80002252b4d4) at m_pool_alloc+0x8a
pool_p_alloc(ffffffff81d34950,2,ffff80002252b4d4) at pool_p_alloc+0x5a
pool_do_get(ffffffff81d34950,2,ffff80002252b4d4) at pool_do_get+0xd3
pool_get(ffffffff81d34950,2) at pool_get+0x93
m_copym(fffffd80d32d6b00,0,3b9aca00,2) at m_copym+0xe4
rtm_sendup(fffffd81193eb7c8,fffffd80d32d6b00,1) at rtm_sendup+0x52
route_input(fffffd80d32d6b00,0,18) at route_input+0x127
rtm_send(fffffd811a905940,2,0,0) at rtm_send+0x12e
rtdeletemsg(fffffd811a905940,ffff80000040b000,0) at rtdeletemsg+0x118
rtflushclone1(fffffd811a905940,fffffd811a905550,0) at rtflushclone1+0x87
rtable_walk_helper(fffffd811d90c7a0,ffff80002252ce40) at rtable_walk_helper+0x5
3
art_table_walk(ffff8000000a58c0,fffffd811ad5f468,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x1d9
art_table_walk(ffff8000000a58c0,fffffd811ad5f448,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5f3a8,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5f388,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5fb68,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5fba8,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5fbc8,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5fbe8,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5fc08,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5fc88,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811ad5f728,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811983b9b0,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811983b270,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811983b230,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811983b1f0,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d202600,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2028e0,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2028c0,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2028a0,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d202880,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2027c0,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2027a0,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d202700,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2026e0,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2628,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2608,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b25c8,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2588,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2568,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2508,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d90b540,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d90bf60,ffffffff81a094d0,ffff80002252c
e40) at art_table_walk+0x25c
art_walk(ffff8000000a58c0,ffffffff81a094d0,ffff80002252ce40) at art_walk+0xd3
rtable_walk(0,18,ffffffff813d28c0,fffffd811a905550) at rtable_walk+0xa2
rtrequest_delete(ffff80002252cf38,4,ffff80000040b000,ffff80002252cff8,0) at rtr
equest_delete+0x115
rtdeletemsg(fffffd811a905550,ffff80000040b000,0) at rtdeletemsg+0xfc
rt_ifa_purge_walker(fffffd811a905550,ffff800000cdaa00,0) at rt_ifa_purge_walker
+0x27
rtable_walk_helper(fffffd811d90ca90,ffff80002252db60) at rtable_walk_helper+0x5
3
art_table_walk(ffff8000000a58c0,fffffd811d2028e0,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x1d9
art_table_walk(ffff8000000a58c0,fffffd811d2028c0,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2028a0,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d202880,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2027c0,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2027a0,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d202700,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2026e0,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2628,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2608,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b25c8,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2588,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2568,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d2b2508,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d90b540,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_table_walk(ffff8000000a58c0,fffffd811d90bf60,ffffffff81a094d0,ffff80002252d
b60) at art_table_walk+0x25c
art_walk(ffff8000000a58c0,ffffffff81a094d0,ffff80002252db60) at art_walk+0xd3
rtable_walk(0,18,ffffffff813d35a0,ffff800000cdaa00) at rtable_walk+0xa2
rt_ifa_purge(ffff800000cdaa00) at rt_ifa_purge+0x7f
in6_unlink_ifa(ffff800000cdaa00,ffff80000040b000) at in6_unlink_ifa+0x324
in6_purgeaddr(ffff800000cdaa00) at in6_purgeaddr+0x11e
in6_ioctl_change_ifaddr(81206919,ffff800000cbce00,ffff80000040b000) at in6_ioct
l_change_ifaddr+0x156
ifioctl(fffffd81180a2dd8,81206919,ffff800000cbce00,ffff8000222ae018) at ifioctl
+0x2b7
sys_ioctl(ffff8000222ae018,ffff80002252df00,ffff80002252df70) at sys_ioctl+0x3c
4
syscall(ffff80002252dfe0) at syscall+0x305
Xsyscall(6,36,421d5b03200,36,421d5b03200,d) at Xsyscall+0x128
end of kernel
end trace frame: 0x7f7fffff7430, count: -81
ddb{1}> mach ddbcpu 2
Stopped at x86_ipi_db+0x12: leave
ddb{2}> trace
x86_ipi_db(ffff800022009ff0) at x86_ipi_db+0x12
x86_ipi_handler() at x86_ipi_handler+0x80
Xresume_lapic_ipi(0,ffff800022009ff0,f572e460000,0,ffff8000222afc38,1) at Xresu
me_lapic_ipi+0x23
_kernel_lock(ffff80002250a030,1) at _kernel_lock+0xa2
usertrap(ffff80002250a030) at usertrap+0x23c
recall_trap(6,dbdbdbdbdbdbdbdb,0,7,f574396ca50,f572e460bc0) at recall_trap+0x8
end of kernel
end trace frame: 0x7f7ffffd5570, count: -6
ddb{2}> mach ddbcpu 3
Stopped at x86_ipi_db+0x12: leave
ddb{3}> trace
x86_ipi_db(ffff800022012ff0) at x86_ipi_db+0x12
x86_ipi_handler() at x86_ipi_handler+0x80
Xresume_lapic_ipi(0,ffff800022012ff0,7f7ffffe13e8,0,10,ffff8000ffff5c30) at Xre
sume_lapic_ipi+0x23
_kernel_lock(ffff80002251c8d0,ffff80002251c8d0) at _kernel_lock+0xa2
Xsyscall(6,1,5,1,7f7ffffe1498,0) at Xsyscall+0x128
end of kernel
end trace frame: 0x7f7ffffe1400, count: -5
ddb{3}> ps
PID TID PPID UID S FLAGS WAIT COMMAND
* 4079 370381 31440 0 7 0x2 pkill
31440 205146 88152 0 3 0x10008a pause ksh
83573 459777 61283 0 7 0x100002 ksh
88152 400736 20067 0 3 0x10008a pause ksh
72017 516683 49765 51 3 0x1000b2 poll ping
61283 326917 43290 0 3 0x10008a pause ksh
20067 350232 69408 0 3 0x10008a pause ksh
49765 80839 36129 0 3 0x10008a pause sh
36129 52062 53905 0 3 0x10008a pause sh
69408 193858 48172 0 3 0x10008a pause sh
53905 66245 72263 0 3 0x100090 piperd cron
48172 474809 72263 0 3 0x100090 piperd cron
20871 322420 1 0 7 0 dhcpcd
21325 20376 56473 0 3 0x100083 ttyin ksh
56473 5502 68950 1000 3 0x10008b pause ksh
37714 293353 48570 0 3 0x83 poll systat
48570 12541 68950 1000 3 0x10008b pause ksh
8927 235365 93066 1000 3 0x100083 poll top
93066 371140 68950 1000 3 0x10008b pause ksh
13211 511386 86231 0 3 0x100083 ttyin ksh
86231 106883 68950 1000 3 0x10008b pause ksh
68950 179406 1 1000 3 0x100080 kqread tmux
29085 364537 94870 1000 3 0x100083 kqread tmux
94870 3765 28406 1000 3 0x10008b pause ksh
28406 432782 83692 1000 3 0x90 select sshd
83692 289813 6425 0 3 0x92 poll sshd
98606 91992 45517 97 3 0x100090 kqread nsd
45517 26091 93753 97 3 0x100090 poll nsd
93753 320795 1 97 3 0x100090 kqread nsd
43290 95692 3682 0 3 0x10008a pause sh
25563 374932 1 0 3 0x100083 ttyin getty
72263 356500 1 0 3 0x100098 poll cron
83271 69483 1 0 3 0x100080 nanosleep sensorsd
1804 251180 1 0 3 0x80 kqread apmd
3682 220180 1 0 3 0x100089 pause sh
24078 42533 1 652 3 0x80 nanosleep collectd
24078 344914 1 652 3 0x4000080 fsleep collectd
24078 409916 1 652 3 0x4000080 fsleep collectd
24078 25750 1 652 3 0x4000080 fsleep collectd
24078 473497 1 652 3 0x4000080 fsleep collectd
24078 434741 1 652 3 0x4000080 fsleep collectd
24078 328906 1 652 3 0x4000080 fsleep collectd
24078 440139 1 652 3 0x4000080 fsleep collectd
24078 371474 1 652 3 0x4000080 fsleep collectd
24078 42341 1 652 3 0x4000080 fsleep collectd
24078 164299 1 652 3 0x4000080 fsleep collectd
82853 139978 1 0 2 0x80 miniupnpd
63285 31626 51757 95 3 0x100092 kqread smtpd
8665 478988 51757 103 3 0x100092 kqread smtpd
30636 101346 51757 95 3 0x100092 kqread smtpd
95351 301768 51757 95 3 0x100092 kqread smtpd
30347 98473 51757 95 3 0x100092 kqread smtpd
74847 147376 51757 95 3 0x100092 kqread smtpd
51757 272927 1 0 3 0x100080 kqread smtpd
12714 167662 52281 94 2 0x100092 rad
62058 23094 52281 94 3 0x100092 kqread rad
52281 129274 1 0 3 0x100080 kqread rad
96988 345992 1 77 3 0x100090 poll dhcpd
6425 110241 1 0 3 0x80 select sshd
67141 67233 15692 83 3 0x100092 poll ntpd
15692 371062 53558 83 3 0x100092 poll ntpd
53558 147154 1 0 3 0x100080 poll ntpd
19514 472683 1 53 3 0x90 kqread unbound
58539 415632 61131 73 7 0x100010 syslogd
61131 212940 1 0 3 0x100082 netio syslogd
60218 136903 59394 115 2 0x100092 slaacd
88780 340861 59394 115 3 0x100092 kqread slaacd
59394 178142 1 0 3 0x100080 kqread slaacd
43343 113098 0 0 3 0x14200 pgzero zerothread
27072 217154 0 0 3 0x14200 aiodoned aiodoned
9166 457962 0 0 3 0x14200 syncer update
55960 86858 0 0 3 0x14200 cleaner cleaner
5233 375165 0 0 3 0x14200 reaper reaper
30310 511990 0 0 3 0x14200 pgdaemon pagedaemon
1626 388493 0 0 3 0x14200 bored crynlk
90203 415345 0 0 3 0x14200 bored crypto
39320 123054 0 0 3 0x14200 bored sensors
37769 413891 0 0 3 0x14200 mmctsk sdmmc0
99326 237010 0 0 3 0x14200 usbtsk usbtask
10067 254965 0 0 3 0x14200 usbatsk usbatsk
82977 348772 0 0 3 0x40014200 acpi0 acpi0
93989 82949 0 0 3 0x40014200 idle3
31552 455631 0 0 3 0x40014200 idle2
41089 290051 0 0 3 0x40014200 idle1
71758 483581 0 0 3 0x14200 bored softnet
75768 181704 0 0 2 0x14200 systqmp
92174 491142 0 0 3 0x14200 bored systq
4610 401933 0 0 3 0x40014200 bored softclock
59548 56710 0 0 3 0x40014200 idle0
78199 503470 0 0 3 0x14200 bored smr
1 104744 0 0 3 0x82 wait init
0 0 -1 0 3 0x10200 scheduler swapper
ddb{3}>
