>Synopsis: Crash at ieee80211_node_checkrssi followed by "boot dump" crash
>Category: kernel amd64
>Environment:
System : OpenBSD 6.2
Details : OpenBSD 6.2-current (GENERIC.MP) #275: Mon Dec 11
20:31:14 MST 2017
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
Architecture: OpenBSD.amd64
Machine : amd64
>Description:
The first reboot after upgrading to the latest snapshot failed during network
initialization.
After getting some information with "ps" and "trace" in ddb, I ran "boot dump"
which failed.
Here is the output (possibly with some errors in the transcript).
uvm_fault (0xffffffff81b34848, 0x10001, 0, 1) -> e
kernel: page fault trap, code=0
Stopped at ieee80211_node_checkrssi+0x12: movb 0x2(%rax), %cl
ddb{0}> trace
ieee80211_node_checkrssi(ffff8000001ed530, 10) at ieee80211_node_checkrssi+0x12
ieee80211_input(ffff80000001eda64, ffff80000001af048,ffffffbb,
ffffff000840c00c) at ieee80211_input+0x324
iwn_rx_done(80000000, ffff80000001af000,d) at iwn_rx_done+0x4ed
iwn_notif_intr(800000000) at iwn_notif_intr+0x1c6
iwn_intr(ffff80000001af000) at iwn_intr+0x33e
intr_handler(20, ffff8000000ee680) at intr_handler+0x63
Xintr_ioapic_edge23() at Xintr_ioapic_edge23+0xcc
--- interrupt ---
end of kernel
end trace frame: 0xa546f905c700, count: -7
0x4c48246c894c5024
ddb{0}> ps
FLAGS WAIT COMMAND
0x83 iwncmd ifconfig
0x10008b pause sh
0x100092 kread slaacd
0x100092 kread slaacd
0x80 kread slaacd
0x100083 nanosleep sleep
0x100089 pause sh
0x10008b pause sh
0x14200 pgzero zerothread
0x14200 aiodoned aiodoned
0x14200 syncerr update
0x14200 cleaner cleaner
0x14200 reaper reaper
0x14200 pgdaemon pagedaemon
0x14200 bored srdis
0x14200 bored crynlk
0x14200 bored crypto
0x14200 usbtk usbtask
0x14200 usbatsk usbatsk
0x14200 bored i915-hangcheck
0x14200 bored i915-db
0x14200 bored i915
0x40014200 acpi0 acpi0
0x40014200 idle3
0x40014200 idle2
0x40014200 idle1
0x14200 bored sensors
0x14200 bored softnet
0x14200 bored systqmp
0x14200 bored systq
0x40014200 netlock softclock
0x40014200 idle0
0x14200 bored sbar
0x82 wait init
0x10200 scheduler swapper
#### At this point I want to reboot and type "boot dump"
ddb{0}> boot dump
### The console is flooded with errors, I can't read them all, it's too fast
splassert: ...
uvm_map ...
assertwaitock: want 0 have 7
pool_do_get: want 6 have 7
buf...
### After a while, it drops in DDB again
splassert: pool_do_get: want 6 have 7
splassert: assertwaitok: want 0 have 7
splassert: assertwaitok: want 0 have 7
splassert: pool_do_put: want 6 have 7
splassert: assertwaitok: want 0 have 7
splassert: pool_do_get: want 0 have 7
WARNING: not updating battery clock
splassert: assertwaitok: want 0 have 7
splassert: assertwaitok: want 0 have 7
panic: kernel diagnostic assertion "p->p_stat == SRUN" failed: file
"/usr/src/sys/kern/kern_sched.c", line 312
Stopped at db_enter+0x5: popq %rbp
TID PID UID PRFLAGS PFLAGS CPU
COMMAND
db_enter() at db_enter+0x5
panic() at panic+0x129
__assert(ffffffff814f85b4, ffff800032ad0870, ffffffff81aa0730, ffff800032a84470
at __assert+0x24
sched_chooseproc() at sched_chooseproc+0x233
mi_switch() at mi_switch+0x199
sleep_finish(ffff800032ad0938, 20) at sleep_finish+0x70
tsleep(78,ffff80000000d2000,1,78) at tsleep+0xc4
acpiec_read_1(ffff800032ad0ae0, ffff80000000d2000) at acpiec_read_1+0x167
acpi_gasio(ffff800000f77e20,0,0,ffff800032ad0b00,1,64fe70c3214b8490) at
acpi_gasio+0x424
aml_opreg_ec_handler(5000000000,ffff800032ad0ae0,fffffff815c6719,ffff800032ad0a60,fff8000000f77e20)
at aml_opreg_ec_handler+0x29
aml_rwgen(ffff80000000f77e08,ffff800000071b08,8,0,3c0,64fe70c3214b8490) at
aml_rwgen+0x657
aml_rwfield(0,74,0,ffff8000000071b08,fffff80000000f77e08) at aml_rwfield+0x294
aml_eval(0,ffff80000000002f3cd,ffff80000000f75f88,ffff8000000071b08,5f) at
aml_eval+0x1dd
end trace frame: 0xffff8000032ad0d20, count: 0
### But I can't enter anything in DDB, it keeps crashing
ddb{0}> ^R
panic: kernel diagnostic assertion "__mp_lock_held(&sched_lock, curcpu()) == 0"
failed: file "/usr/src/sys/kern/kern_lock.c", line 80
Stopped at db_enter+0x5: popq %rbp
db_enter() at db_enter+0x5
panic() at panic+0x129
__assert(ffffffff814f85b4, ffff800032ad0388,ffffffff81aa0730,d) at __assert+0x24
_kernel_lock(fffffffff8132d7e1,ffff800032ad0398) at _kernel_lock+0xb1
softintr_dispatch(0) at softintr_dispatch+0x43
Xsoftclock() at Xsoftclock+0x1f
--- interrupt ---
end of kernel
end trace frame: 0x9c5a203abf20eac1, count: 9
0xfffffff8c2c748ff:
### ...and keeps crashking.
### At this point I'm too bored to continue copying the errors...
ddb{0}>
### But then it stops crashing with new errors because of "double fault trap"
panic: netlock: lock not held by this process
Stopped at db_enter+0x5: popq %rbp
db_enter() at db_enter+0x5
panic() at panic+0x129
_rw_exit_write(...) at _rw_exit_write+0x6e
if_downall() at if_downall+0x84
boot(1) at boot+0x86
reboot(4900) at reboot+0x4f
axe_match(...,...,...) at axe_match
db_boot_dump(cmd(...,...,...,...) at db_boot_dump_cmd+0xe
db_command(0,...)at db_command+0x28f
db_command_loop() at db_command_loop+0x96
db_trap() at db_trap+0x137
dbktrap(...,...,...) at db_ktrap+0xe5
trap() at trap+0x505
--- trap (number 1) ---
db_enter() at db_enter+0x5
end trace frame: ..., count: 0
ddb{0}>
panic netlock: lock not hel by this process
kernel: double fault trap, code=0
Faulted in DDB; continuing...
### Nothing new, so I turn it off.
>How-To-Repeat:
The second reboot didn't failed, so I can't reproduce the crash.
>Fix:
Dunno
dmesg:
OpenBSD 6.2-current (GENERIC.MP) #275: Mon Dec 11 20:31:14 MST 2017
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 4062691328 (3874MB)
avail mem = 3932647424 (3750MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.6 @ 0xe0010 (78 entries)
bios0: vendor LENOVO version "6QET47WW (1.17 )" date 07/14/2010
bios0: LENOVO 3680BA5
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SSDT ECDT APIC MCFG HPET ASF! SLIC BOOT SSDT TCPA DMAR
SSDT SSDT SSDT
acpi0: wakeup devices LID_(S3) SLPB(S3) IGBE(S4) EXP1(S4) EXP2(S4) EXP3(S4)
EXP4(S4) EXP5(S4) EHC1(S3) EHC2(S3) HDEF(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpiec0 at acpi0
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5 CPU M 520 @ 2.40GHz, 1197.21 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,AES,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
acpitimer0: recalibrated TSC frequency 2394004739 Hz
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 132MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM) i5 CPU M 520 @ 2.40GHz, 1197.00 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,AES,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 1, core 0, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Core(TM) i5 CPU M 520 @ 2.40GHz, 1197.00 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,AES,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 5 (application processor)
cpu3: Intel(R) Core(TM) i5 CPU M 520 @ 2.40GHz, 1197.00 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,AES,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 1, core 2, package 0
ioapic0 at mainbus0: apid 1 pa 0xfec00000, version 20, 24 pins
, remapped to apid 1
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpihpet0: recalibrated TSC frequency 2393995740 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus -1 (PEG_)
acpiprt2 at acpi0: bus 13 (EXP1)
acpiprt3 at acpi0: bus -1 (EXP2)
acpiprt4 at acpi0: bus -1 (EXP3)
acpiprt5 at acpi0: bus 5 (EXP4)
acpiprt6 at acpi0: bus 2 (EXP5)
acpicpu0 at acpi0: C3(350@245 mwait.3@0x20), C2(500@205 mwait.3@0x10),
C1(1000@3 mwait.1), PSS
acpicpu1 at acpi0: C3(350@245 mwait.3@0x20), C2(500@205 mwait.3@0x10),
C1(1000@3 mwait.1), PSS
acpicpu2 at acpi0: C3(350@245 mwait.3@0x20), C2(500@205 mwait.3@0x10),
C1(1000@3 mwait.1), PSS
acpicpu3 at acpi0: C3(350@245 mwait.3@0x20), C2(500@205 mwait.3@0x10),
C1(1000@3 mwait.1), PSS
acpipwrres0 at acpi0: PUBS, resource for EHC1, EHC2
acpitz0 at acpi0: critical temperature is 100 degC
acpibtn0 at acpi0: LID_
acpibtn1 at acpi0: SLPB
"LEN0018" at acpi0 not configured
"SMO1200" at acpi0 not configured
acpibat0 at acpi0: BAT0 model "42T4837" serial 50934 type LION oem "LGC"
acpiac0 at acpi0: AC unit offline
acpithinkpad0 at acpi0
"PNP0C14" at acpi0 not configured
acpidock0 at acpi0: GDCK not docked (0)
acpivideo0 at acpi0: VID_
acpivout0 at acpivideo0: LCD0
acpivideo1 at acpi0: VID_
cpu0: Enhanced SpeedStep 1197 MHz: speeds: 2400, 2399, 2266, 2133, 1999, 1866,
1733, 1599, 1466, 1333, 1199 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Core Host" rev 0x02
inteldrm0 at pci0 dev 2 function 0 "Intel HD Graphics" rev 0x02
drm0 at inteldrm0
intagp0 at inteldrm0
agp0 at intagp0: aperture at 0xd0000000, size 0x10000000
inteldrm0: msi
inteldrm0: 1280x800, 32bpp
wsdisplay0 at inteldrm0 mux 1: console (std, vt100 emulation)
wsdisplay0: screen 1-5 added (std, vt100 emulation)
"Intel 3400 MEI" rev 0x06 at pci0 dev 22 function 0 not configured
puc0 at pci0 dev 22 function 3 "Intel 3400 KT" rev 0x06: ports: 1 com
com4 at puc0 port 0 apic 1 int 17: ns16550a, 16 byte fifo
com4: probed fifo depth: 0 bytes
em0 at pci0 dev 25 function 0 "Intel 82577LM" rev 0x06: msi, address
f0:de:f1:11:3c:7e
ehci0 at pci0 dev 26 function 0 "Intel 3400 USB" rev 0x06: apic 1 int 23
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00
addr 1
azalia0 at pci0 dev 27 function 0 "Intel 3400 HD Audio" rev 0x06: msi
azalia0: codecs: Conexant/0x5069, Intel/0x2804, using Conexant/0x5069
audio0 at azalia0
ppb0 at pci0 dev 28 function 0 "Intel 3400 PCIE" rev 0x06: msi
pci1 at ppb0 bus 13
ppb1 at pci0 dev 28 function 3 "Intel 3400 PCIE" rev 0x06: msi
pci2 at ppb1 bus 5
ppb2 at pci0 dev 28 function 4 "Intel 3400 PCIE" rev 0x06: msi
pci3 at ppb2 bus 2
iwn0 at pci3 dev 0 function 0 "Intel Centrino Ultimate-N 6300" rev 0x35: msi,
MIMO 3T3R, MoW, address 00:24:d7:46:03:bc
ehci1 at pci0 dev 29 function 0 "Intel 3400 USB" rev 0x06: apic 1 int 19
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00
addr 1
ppb3 at pci0 dev 30 function 0 "Intel 82801BAM Hub-to-PCI" rev 0xa6
pci4 at ppb3 bus 14
pcib0 at pci0 dev 31 function 0 "Intel QM57 LPC" rev 0x06
ahci0 at pci0 dev 31 function 2 "Intel 3400 AHCI" rev 0x06: msi, AHCI 1.3
ahci0: port 0: 3.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, WDC WD1600BEKT-0, 02.0> SCSI3 0/direct
fixed naa.50014ee2059f90bd
sd0: 152627MB, 512 bytes/sector, 312581808 sectors
ichiic0 at pci0 dev 31 function 3 "Intel 3400 SMBus" rev 0x06: apic 1 int 23
iic0 at ichiic0
spdmem0 at iic0 addr 0x50: 2GB DDR3 SDRAM PC3-10600 SO-DIMM
spdmem1 at iic0 addr 0x51: 2GB DDR3 SDRAM PC3-10600 SO-DIMM
itherm0 at pci0 dev 31 function 6 "Intel 3400 Thermal" rev 0x06
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
wsmouse0 at pms0 mux 0
wsmouse1 at pms0 mux 0
pms0: Synaptics touchpad, firmware 7.4, 0x1e0b1 0xb40000
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
aps0 at isa0 port 0x1600/31
pci5 at mainbus0 bus 255
pchb1 at pci5 dev 0 function 0 "Intel QuickPath" rev 0x02
pchb2 at pci5 dev 0 function 1 "Intel QuickPath" rev 0x02
pchb3 at pci5 dev 2 function 0 "Intel QPI Link" rev 0x02
pchb4 at pci5 dev 2 function 1 "Intel QPI Physical" rev 0x02
pchb5 at pci5 dev 2 function 2 "Intel Reserved" rev 0x02
pchb6 at pci5 dev 2 function 3 "Intel Reserved" rev 0x02
vmm0 at mainbus0: VMX/EPT
uhub2 at uhub0 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev
2.00/0.00 addr 2
ugen0 at uhub2 port 3 "UPEK Biometric Coprocessor" rev 1.01/0.02 addr 3
uvideo0 at uhub2 port 6 configuration 1 interface 0 "Chicony Electronics Co.,
Ltd. Integrated Camera" rev 2.00/23.45 addr 4
video0 at uvideo0
uhub3 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching Hub" rev
2.00/0.00 addr 2
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
sd1 at scsibus3 targ 1 lun 0: <OPENBSD, SR CRYPTO, 006> SCSI2 0/direct fixed
sd1: 152624MB, 512 bytes/sector, 312575168 sectors
root on sd1a (23e691dd1ff29ca6.a) swap on sd1b dump on sd1b
usbdevs:
Controller /dev/usb0:
addr 1: high speed, self powered, config 1, EHCI root hub(0x0000),
Intel(0x8086), rev 1.00
port 1 addr 2: high speed, self powered, config 1, Rate Matching Hub(0x0020),
Intel(0x8087), rev 0.00
port 1 powered
port 2 powered
port 3 addr 3: full speed, power 100 mA, config 1, Biometric
Coprocessor(0x2016), UPEK(0x147e), rev 0.02
port 4 powered
port 5 powered
port 6 addr 4: high speed, power 200 mA, config 1, Integrated Camera(0x4816),
Chicony Electronics Co., Ltd.(0x17ef), rev 23.45
port 2 powered
port 3 powered
Controller /dev/usb1:
addr 1: high speed, self powered, config 1, EHCI root hub(0x0000),
Intel(0x8086), rev 1.00
port 1 addr 2: high speed, self powered, config 1, Rate Matching Hub(0x0020),
Intel(0x8087), rev 0.00
port 1 powered
port 2 powered
port 3 powered
port 4 powered
port 5 powered
port 6 powered
port 7 powered
port 8 powered
port 2 powered
port 3 powered
