After upgrading my router to the latest -current snap I've started getting
panics shortly after boot when kern.usercrypto=1. I do not get panics when
leaving usercrypto at its default disabled state. My latest upgrade was
done from a snap about 2 months old to the current snap (so changes from
last stable config would include the big time change). Upgrade was done via
the installer upgrade process and /dev/crypto was rebuilt along with other
devices.
I built a fresh kernel with debug enabled and captured 4 panics, traces,
and ps outputs. The full dmesg of the system (an ALIX.2) is also included
in the first panic.
Let me know if I can provide additional information that would be useful in
tracking it down. I've seen mention that enabling user crypto doesn't
really offer much of a performance benefit so I guess ultimately I can just
leave it disabled.. but seemed worth a try (and a bug is a bug.)
Thanks,
Brian
== Panic #1 & full dmesg - uvm_fault / openvpn ==
PC Engines ALIX.2 v0.99h
640 KB Base Memory
261120 KB Extended Memory
Waiting for HDD ...
01F0 Master 044A ULTIMATE CF CARD 16GB
Phys C/H/S 16383/16/63 Log C/H/S 1943/255/63 LBA
Using drive 0, partition 3.
Loading.....
probing: pc0 com0 com1 pci mem[640K 255M a20=on]
disk: hd0+
>> OpenBSD/i386 BOOT 3.21
switching console to com0
>> OpenBSD/i386 BOOT 3.21
boot>
booting hd0a:/bsd: 11637052+1091692 [52+402064+396246]=0xce69b8
entry point at 0x200120
[ using 798736 bytes of bsd ELF symbol table ]
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
Copyright (c) 1995-2013 OpenBSD. All rights reserved.
http://www.OpenBSD.org
OpenBSD 5.4-current (DEBUG) #0: Sun Sep 8 11:51:42 EDT 2013
[email protected]:/usr/src/sys/arch/i386/compile/DEBUG
cpu0: Geode(TM) Integrated Processor by AMD PCS ("AuthenticAMD" 586-class)
499 MHz
cpu0: FPU,DE,PSE,TSC,MSR,CX8,SEP,PGE,CMOV,CFLUSH,MMX,MMXX,3DNOW2,3DNOW
real mem = 267976704 (255MB)
avail mem = 249384960 (237MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 11/05/08, BIOS32 rev. 0 @ 0xfd088
pcibios0 at bios0: rev 2.1 @ 0xf0000/0x10000
pcibios0: pcibios_get_intr_routing - function not supported
pcibios0: PCI IRQ Routing information unavailable.
pcibios0: PCI bus #0 is the last bus
bios0: ROM list: 0xe0000/0xa800
cpu0 at mainbus0: (uniprocessor)
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 1 function 0 "AMD Geode LX" rev 0x33
glxsb0 at pci0 dev 1 function 2 "AMD Geode LX Crypto" rev 0x00: RNG AES
vr0 at pci0 dev 9 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 10,
address 00:0d:
b9:aa:aa:40
ukphy0 at vr0 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
0x004063, mode
l 0x0034
vr1 at pci0 dev 10 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 11,
address 00:0d
:b9:aa:aa:41
ukphy1 at vr1 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
0x004063, mode
l 0x0034
vr2 at pci0 dev 11 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 15,
address 00:0d
:b9:aa:aa:42
ukphy2 at vr2 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
0x004063, mode
l 0x0034
glxpcib0 at pci0 dev 15 function 0 "AMD CS5536 ISA" rev 0x03: rev 3, 32-bit
3579545H
z timer, watchdog, gpio, i2c
gpio0 at glxpcib0: 32 pins
iic0 at glxpcib0
maxtmp0 at iic0 addr 0x4c: lm86
pciide0 at pci0 dev 15 function 2 "AMD CS5536 IDE" rev 0x01: DMA, channel 0
wired to
compatibility, channel 1 wired to compatibility
wd0 at pciide0 channel 0 drive 0: <ULTIMATE CF CARD 16GB>
wd0: 1-sector PIO, LBA, 15247MB, 31227840 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 4
pciide0: channel 1 ignored (disabled)
ohci0 at pci0 dev 15 function 4 "AMD CS5536 USB" rev 0x02: irq 12, version
1.0, lega
cy support
ehci0 at pci0 dev 15 function 5 "AMD CS5536 USB" rev 0x02: irq 12
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "AMD EHCI root hub" rev 2.00/1.00 addr 1
isa0 at glxpcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
usb1 at ohci0: USB revision 1.0
uhub1 at usb1 "AMD OHCI root hub" rev 1.00/1.00 addr 1
mtrr: K6-family MTRR support (2 registers)
nvram: invalid checksum
vscsi0 at root
scsibus0 at vscsi0: 256 targets
softraid0 at root
scsibus1 at softraid0: 256 targets
root on wd0a (ba9fc330a1419c2f.a) swap on wd0b dump on wd0b
WARNING: / was not properly unmounted
clock: unknown CMOS layout
Automatic boot in progress: starting file system checks.
/dev/rwd0a: FREE BLK COUNT(S) WRONG IN SUPERBLK (SALVAGED)
/dev/rwd0a: SUMMARY INFORMATION BAD (SALVAGED)
/dev/rwd0a: BLK(S) MISSING IN BIT MAPS (SALVAGED)
/dev/rwd0a: 2098 files, 43316 used, 472947 free (235 frags, 59089 blocks,
0.0% fragm
entation)
/dev/rwd0a: MARKING FILE SYSTEM CLEAN
/dev/rwd0g: 11 files, 11 used, 905124 free (20 frags, 113138 blocks, 0.0%
fragmentat
ion)
/dev/rwd0g: MARKING FILE SYSTEM CLEAN
/dev/rwd0d: 4 files, 3 used, 516268 free (20 frags, 64531 blocks, 0.0%
fragmentation
)
/dev/rwd0d: MARKING FILE SYSTEM CLEAN
/dev/rwd0f: file system is clean; not checking
/dev/rwd0e: file system is clean; not checking
setting tty flags
pf enabled
net.inet.ip.forwarding: 0 -> 1
kern.usercrypto: 0 -> 1
kern.userasymcrypto: 0 -> 0
ddb.panic: 1 -> 1
kern.watchdog.period: 0 -> 300
net.inet6.ip6.forwarding: 0 -> 0
net.inet6.ip6.accept_rtadv: 0 -> 1
net.inet6.icmp6.rediraccept: 0 -> 1
starting network
DHCPREQUEST on vr0 to 255.255.255.255 port 67
DHCPACK from 71.85.21.1 (00:01:df:31:aa:81)
bound to 71.85.21.7 -- renewal in 117803 seconds.
IPv6 autoconf: vr0
uvm_fault(0xd0d3f4e0, 0xea0c8000, 0, 1) -> e
kernel: page fault trap, code=0
Stopped at glxsb_crypto_freesession+0x75: movl 0x8(%eax),%eax
ddb> trace
glxsb_crypto_freesession(0,0,2,d0576972,0) at glxsb_crypto_freesession+0x75
crypto_freesession(0,0,0,2809070c,2807d420) at crypto_freesession+0xda
csefree(d1517000,d1517000,0,d1441840,c0) at csefree+0x1b
cryptof_ioctl(d581139c,80046366,f36eee2c,d581e798,80046366) at
cryptof_ioctl+0x
521
sys_ioctl(d581e798,f36eef44,f36eef64,d067ebc1,d581e798) at sys_ioctl+0x459
syscall() at syscall+0x40f
--- syscall (number 176) ---
0x80046366:
ddb> ps
PID PPID PGRP UID S FLAGS WAIT COMMAND
* 4126 12043 12043 0 7 0x4002 openvpn
22961 1 22961 77 3 0x180 poll dhclient
31214 1 31214 0 3 0x80 poll dhclient
12043 1 12043 0 3 0x408a pause sh
13 0 0 0 3 0x100200 aiodoned aiodoned
12 0 0 0 3 0x100200 syncer update
11 0 0 0 3 0x100200 cleaner cleaner
10 0 0 0 3 0x100200 reaper reaper
9 0 0 0 3 0x100200 pgdaemon pagedaemon
8 0 0 0 3 0x100200 bored crypto
7 0 0 0 3 0x100200 pftm pfpurge
6 0 0 0 3 0x100200 usbtsk usbtask
5 0 0 0 3 0x100200 usbatsk usbatsk
4 0 0 0 3 0x100200 bored syswq
3 0 0 0 3 0x40100200 idle0
2 0 0 0 3 0x100200 kmalloc kmthread
1 0 1 0 3 0x4080 wait init
0 -1 0 0 3 0x200 scheduler swapper
== Panic #2: Data modified on freelist / openvpn ==
Data modified on freelist: word 145106852 of object 0xd14417a0 size 0x14
previous type ??? (invalid addr 0xa3c00d5f)
panic: Data modified on freelist: word 3 of object 0xd14417a0 size 0x14
previous type ??? (0xa29a7065 != 0xa29a7065)
Stopped at Debugger+0x4: popl %ebp
ddb> trace
Debugger(d0ba182e,f36eeaa8,d0b9ecc0,f36eeaa8,d14417a0) at Debugger+0x4
panic(d0b9ecc0,d0b9ec68,3,d14417a0,14) at panic+0x96
malloc(14,7f,2,0,f36eeb5c) at malloc+0x5d6
bread_cluster(d57981f0,1c,0,4000,f36eec2c) at bread_cluster+0x102
ffs_read(f36eec5c,0,d57981f0,f36eecb4,0) at ffs_read+0x3fa
VOP_READ(d57981f0,f36eecb4,0,d5834000,7f4) at VOP_READ+0x45
uvn_io(d57981f0,f36eed58,1,2,0) at uvn_io+0x25f
uvn_get(d57981f0,70000,0,f36eeea0,f36eeec8) at uvn_get+0x27e
uvm_fault(d5825180,3c006000,0,1,f36eef64) at uvm_fault+0xf05
trap() at trap+0x8db
--- trap (number 0) ---
0x4:
ddb> ps
PID PPID PGRP UID S FLAGS WAIT COMMAND
* 3275 5775 5775 0 7 0x4002 openvpn
27615 1 27615 77 3 0x180 poll dhclient
20411 1 20411 0 3 0x80 poll dhclient
5775 1 5775 0 3 0x408a pause sh
13 0 0 0 3 0x100200 aiodoned aiodoned
12 0 0 0 3 0x100200 syncer update
11 0 0 0 3 0x100200 cleaner cleaner
10 0 0 0 3 0x100200 reaper reaper
9 0 0 0 3 0x100200 pgdaemon pagedaemon
8 0 0 0 3 0x100200 bored crypto
7 0 0 0 3 0x100200 pftm pfpurge
6 0 0 0 3 0x100200 usbtsk usbtask
5 0 0 0 3 0x100200 usbatsk usbatsk
4 0 0 0 3 0x100200 bored syswq
3 0 0 0 3 0x40100200 idle0
2 0 0 0 3 0x100200 kmalloc kmthread
1 0 1 0 3 0x4080 wait init
0 -1 0 0 3 0x200 scheduler swapper
== Panic #3: Data modified on freelist / ssh-keygen ==
IPv6 autoconf: vr0
Data modified on freelist: word 145106972 of object 0xd1441fe0 size 0x20
previous ty
pe ??? (invalid addr 0xa8bb99a4)
panic: Data modified on freelist: word 3 of object 0xd1441fe0 size 0x20
previous typ
e ??? (0x2a09f6c != 0x2a09f6c)
Stopped at Debugger+0x4: popl %ebp
RUN AT LEAST 'trace' AND 'ps' AND INCLUDE OUTPUT WHEN REPORTING THIS PANIC!
DO NOT EVEN BOTHER REPORTING THIS WITHOUT INCLUDING THAT INFORMATION!
ddb> trace
Debugger(d0ba182e,f36eec88,d0b9ecc0,f36eec88,d1441fe0) at Debugger+0x4
panic(d0b9ecc0,d0b9ec68,3,d1441fe0,20) at panic+0x96
malloc(20,4c,1,d1441e60,c0) at malloc+0x5d6
cryptof_ioctl(d581139c,c01c6365,f36eee2c,d581e91c,c01c6365) at
cryptof_ioctl+0x
205
sys_ioctl(d581e91c,f36eef44,f36eef64,d067ebc1,d581e91c) at sys_ioctl+0x459
syscall() at syscall+0x40f
--- syscall (number -791256456) ---
Bad frame pointer: 0xd0d66278
0xc01c6365:
ddb> ps
PID PPID PGRP UID S FLAGS WAIT COMMAND
*32087 18220 18220 0 7 0x4002 ssh-keygen
25304 1 25304 77 3 0x180 poll dhclient
21902 1 21902 0 3 0x80 poll dhclient
18220 1 18220 0 3 0x408a pause sh
13 0 0 0 3 0x100200 aiodoned aiodoned
12 0 0 0 3 0x100200 syncer update
11 0 0 0 3 0x100200 cleaner cleaner
10 0 0 0 3 0x100200 reaper reaper
9 0 0 0 3 0x100200 pgdaemon pagedaemon
8 0 0 0 3 0x100200 bored crypto
7 0 0 0 3 0x100200 pftm pfpurge
6 0 0 0 3 0x100200 usbtsk usbtask
5 0 0 0 3 0x100200 usbatsk usbatsk
4 0 0 0 3 0x100200 bored syswq
3 0 0 0 3 0x40100200 idle0
2 0 0 0 3 0x100200 kmalloc kmthread
1 0 1 0 3 0x4080 wait init
0 -1 0 0 3 0x200 scheduler swapper
== Panic #4: rw_enter / ssh-keygen ==
IPv6 autoconf: vr0
panic: rw_enter: vmmaplk locking against myself
Stopped at Debugger+0x4: popl %ebp
RUN AT LEAST 'trace' AND 'ps' AND INCLUDE OUTPUT WHEN REPORTING THIS PANIC!
DO NOT EVEN BOTHER REPORTING THIS WITHOUT INCLUDING THAT INFORMATION!
ddb> trace
Debugger(d0ba182e,f36ee7a4,d0b9eeb0,f36ee7a4,b0) at Debugger+0x4
panic(d0b9eeb0,d0bba9b0,f36ee7d8,d06730ba,f36ee7e8) at panic+0x96
rw_exit_write(d5825184,2,f257f000,f36ee7f0,d048db1e) at rw_exit_write+0xc1
rw_enter(d5825184,2,3,0,d066745c) at rw_enter+0x69
rw_enter_read(d5825184,0,f36ee878,d05f5d69,d5825180) at rw_enter_read+0x69
vm_map_lock_read_ln(d5825180,d0bb9bd1,6f6,fffffff4,ffffffff) at
vm_map_lock_rea
d_ln+0x21
uvmfault_lookup(f36ee994,0,0,f36ee8ac,d048da32) at uvmfault_lookup+0x93
uvm_fault(d5825180,8a66b000,0,1,d5753b48) at uvm_fault+0x74
trap() at trap+0x8db
--- trap (number 1563538877) ---
0:
ddb> ps
PID PPID PGRP UID S FLAGS WAIT COMMAND
* 2453 4590 4590 0 7 0x4002 ssh-keygen
11890 1 11890 77 3 0x180 poll dhclient
22772 1 22772 0 3 0x80 poll dhclient
4590 1 4590 0 3 0x408a pause sh
13 0 0 0 3 0x100200 aiodoned aiodoned
12 0 0 0 3 0x100200 syncer update
11 0 0 0 3 0x100200 cleaner cleaner
10 0 0 0 3 0x100200 reaper reaper
9 0 0 0 3 0x100200 pgdaemon pagedaemon
8 0 0 0 3 0x100200 bored crypto
7 0 0 0 3 0x100200 pftm pfpurge
6 0 0 0 3 0x100200 usbtsk usbtask
5 0 0 0 3 0x100200 usbatsk usbatsk
4 0 0 0 3 0x100200 bored syswq
3 0 0 0 3 0x40100200 idle0
2 0 0 0 3 0x100200 kmalloc kmthread
1 0 1 0 3 0x4080 wait init
0 -1 0 0 3 0x200 scheduler swapper
