>Synopsis: Semi-reproducible uvm-related crash "bad frame pointer"
>Category: kernel
>Environment:
System : OpenBSD 5.5
Details : OpenBSD 5.5-current (GENERIC.MP) #43: Sat Apr 12 09:39:12
MDT 2014
[email protected]:/usr/src/sys/arch/i386/compile/GENERIC.MP
Architecture: OpenBSD.i386
Machine : i386
>Description:
[ddb output attached]
I have been experiencing a somewhat reliable crash/hang in
OpenBSD recently, on a Thinkpad T60. The system will
usually run fine for a day or two, but I often wake up to
an unresponsive system. It still responds to pings, but
it is unresponsive to keyboard and mouse input.
I suspected a faulty memory module, but the old HDD in the
system was running OpenBSD 5.1-current without issues. It
was when I replaced the HDD and did a fresh install of
OpenBSD 5.5-current that the crash/hang started.
Additionally, ports/sysutils/memtest86+ doesn't find any
issues.
Running a memory heavy program (like ports/games/wesnoth
with a lot of addons, which uses in excess of 700M of RAM)
is much more likely to hang the system outright and has
caused a hang within minutes of rebooting, but usually the
longer the system runs, the more likely it is to crash,
and I can usually run wesnoth fine so long as it's done
immediately after a reboot.
There have also been a few occasions where, prior to
crashing, running a program in the shell will produce an
error of (somewhat paraphrased) "Cannot fork, could not
allocate memory", but running top showed more than enough
free RAM was available.
I set "machdep.kbdreset=2" and broke into ddb manually and
found a "Bad frame pointer" error.
In the interest of full-disclosure, here is my
/etc/sysctl.conf with the knobs I've turned, though I have
tried commenting out said knobs one-by-one and still get
the same crash/hang, so I don't think they're necessarily
to blame, but am willing to be proven wrong.
# grep -v '^#' /etc/sysctl.conf
net.inet.ip.forwarding=1 # 1=Permit forwarding (routing) of IPv4 packets
ddb.console=1 # 1=Permit entry of ddb from the console
kern.pool_debug=0 # 0=Disable pool corruption checks (faster)
machdep.allowaperture=2 # See xf86(4)
machdep.kbdreset=2 # permit entering ddb from consol w/ keyboard
kern.bufcachepercent=90
kern.usermount=1
kern.shminfo.shmall=134217728 # increase shared memory a ton for memory-hogs,
in hw.pagesize units. Currently set to 512 MB.
kern.maxvnodes=1000000 # increase vnode cache to improve
pathname->inode lookups
Additionally, I have been running a Thinkpad X61t with an
identical configuration, which does not seem to crash.
This crashdump was transcribed by hand from a photograph,
so please forgive any errors.
ddb{0} trace
uaddr_free_rbtree_RB_REMOVE_COLOR(d9a5400c,d92bb390,0) at
uaddr_free_rbtree_RB_REMOVE_COLOR+0x5d
uaddr_free_rbtree_RB_REMOVE(d9a5400c,d92bb5e8,8acf721,3,d92bb5e8) at
uaddr_free_rbtree_RB_REMOVE+0xb0
uaddr_bestfit_remove(d0b928c0,d9a54000,d92bb5e8,f5368f3c,1000) at
uaddr_bestfit_remove+0x1c
uvm_map_free_remove(d0b928c0,d9a54000,d92bb5e8,1000,d0b928c0) at
uvm_mapent_free_remove+0x62
uvm_map_freelist_update_clear(d0b928c0,f5368ed8,1000,0,3) at
uvm_map_freelist_update_clear+0x81
uvm_map_kmem_grow(d0b928c0,f5368ed8,1000,201323,1000) at uvm_map_kmem_grow+0x2e
uvm_map(d0b928c0,f5368f3c,1000,0,ffffffff) at uvm_map+0x3b9
uvm_km_thread(d99ef170) at uvm_km_thread+0xfd
Bad frame pointer: 0xd0d3ce28
[most of ps output omitted]
ddb{0} ps
*23487 0 0 0 7 0x4200 kmthread
ddb{0}> show panic
the kernel did not panic
>How-To-Repeat:
The system rarely stays running for more than two or three
days, so waiting will eventually produce a crash/hang.
Running a memory-intensive program is much more likely to
crash/hang the system though.
>Fix:
Unknown
dmesg:
OpenBSD 5.5-current (GENERIC.MP) #43: Sat Apr 12 09:39:12 MDT 2014
[email protected]:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Genuine Intel(R) CPU T2300 @ 1.66GHz ("GenuineIntel" 686-class) 1.67 GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,NXE,SSE3,MWAIT,VMX,EST,TM2,xTPR,PDCM,PERF
real mem = 2682613760 (2558MB)
avail mem = 2626314240 (2504MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 04/01/10, BIOS32 rev. 0 @ 0xfd6b0, SMBIOS
rev. 2.4 @ 0xe0010 (68 entries)
bios0: vendor LENOVO version "79ETE6WW (2.26 )" date 04/01/2010
bios0: LENOVO 2623D9U
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SSDT ECDT TCPA APIC MCFG HPET BOOT SSDT SSDT SSDT SSDT
acpi0: wakeup devices LID_(S3) SLPB(S3) LURT(S3) DURT(S3) EXP0(S4) EXP1(S4)
EXP2(S4) EXP3(S4) PCI1(S4) USB0(S3) USB1(S3) USB2(S3) USB7(S3) HDEF(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpiec0 at acpi0
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 166MHz
cpu0: mwait min=64, max=64, C-substates=0.2.2.2.2, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Genuine Intel(R) CPU T2300 @ 1.66GHz ("GenuineIntel" 686-class) 1.67 GHz
cpu1:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,NXE,SSE3,MWAIT,VMX,EST,TM2,xTPR,PDCM,PERF
ioapic0 at mainbus0: apid 1 pa 0xfec00000, version 20, 24 pins
ioapic0: misconfigured as apic 2, remapped to apid 1
acpimcfg0 at acpi0 addr 0xf0000000, bus 0-63
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (AGP_)
acpiprt2 at acpi0: bus 2 (EXP0)
acpiprt3 at acpi0: bus 3 (EXP1)
acpiprt4 at acpi0: bus 4 (EXP2)
acpiprt5 at acpi0: bus 12 (EXP3)
acpiprt6 at acpi0: bus 21 (PCI1)
acpicpu0 at acpi0: C3, C2, C1, PSS
acpicpu1 at acpi0: C3, C2, C1, PSS
acpipwrres0 at acpi0: PUBS, resource for USB0, USB2, USB7
acpitz0 at acpi0: critical temperature is 127 degC
acpitz1 at acpi0: critical temperature is 99 degC
acpibtn0 at acpi0: LID_
acpibtn1 at acpi0: SLPB
acpibat0 at acpi0: BAT0 model "92P1139" serial 659 type LION oem "Panasonic"
acpibat1 at acpi0: BAT1 not present
acpiac0 at acpi0: AC unit online
acpithinkpad0 at acpi0
acpidock0 at acpi0: GDCK docked (15)
bios0: ROM list: 0xc0000/0xfe00 0xd0000/0x1000 0xd1000/0x1000 0xdc000/0x4000!
0xe0000/0x10000!
cpu0: Enhanced SpeedStep 1663 MHz: speeds: 1667, 1333, 1000 MHz
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "Intel 82945GM Host" rev 0x03
ppb0 at pci0 dev 1 function 0 "Intel 82945GM PCIE" rev 0x03: apic 1 int 16
pci1 at ppb0 bus 1
radeondrm0 at pci1 dev 0 function 0 "ATI Radeon Mobility X1300 M52-64" rev 0x00
drm0 at radeondrm0
radeondrm0: apic 1 int 16
azalia0 at pci0 dev 27 function 0 "Intel 82801GB HD Audio" rev 0x02: msi
azalia0: codecs: Analog Devices AD1981HD, Conexant/0x2bfa, using Analog Devices
AD1981HD
audio0 at azalia0
ppb1 at pci0 dev 28 function 0 "Intel 82801GB PCIE" rev 0x02: apic 1 int 20
pci2 at ppb1 bus 2
em0 at pci2 dev 0 function 0 "Intel 82573L" rev 0x00: msi, address
00:16:41:52:7e:81
ppb2 at pci0 dev 28 function 1 "Intel 82801GB PCIE" rev 0x02: apic 1 int 21
pci3 at ppb2 bus 3
wpi0 at pci3 dev 0 function 0 "Intel PRO/Wireless 3945ABG" rev 0x02: msi, MoW1,
address 00:13:02:20:41:18
ppb3 at pci0 dev 28 function 2 "Intel 82801GB PCIE" rev 0x02: apic 1 int 22
pci4 at ppb3 bus 4
ppb4 at pci0 dev 28 function 3 "Intel 82801GB PCIE" rev 0x02: apic 1 int 23
pci5 at ppb4 bus 12
uhci0 at pci0 dev 29 function 0 "Intel 82801GB USB" rev 0x02: apic 1 int 16
uhci1 at pci0 dev 29 function 1 "Intel 82801GB USB" rev 0x02: apic 1 int 17
uhci2 at pci0 dev 29 function 2 "Intel 82801GB USB" rev 0x02: apic 1 int 18
uhci3 at pci0 dev 29 function 3 "Intel 82801GB USB" rev 0x02: apic 1 int 19
ehci0 at pci0 dev 29 function 7 "Intel 82801GB USB" rev 0x02: apic 1 int 19
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb5 at pci0 dev 30 function 0 "Intel 82801BAM Hub-to-PCI" rev 0xe2
pci6 at ppb5 bus 21
cbb0 at pci6 dev 0 function 0 "TI PCI1510 CardBus" rev 0x00: apic 1 int 16
cardslot0 at cbb0 slot 0 flags 0
cardbus0 at cardslot0: bus 22 device 0 cacheline 0x8, lattimer 0xb0
pcmcia0 at cardslot0
ichpcib0 at pci0 dev 31 function 0 "Intel 82801GBM LPC" rev 0x02: PM disabled
pciide0 at pci0 dev 31 function 1 "Intel 82801GB IDE" rev 0x02: DMA, channel 0
configured to compatibility, channel 1 configured to compatibility
atapiscsi0 at pciide0 channel 0 drive 0
scsibus1 at atapiscsi0: 2 targets
cd0 at scsibus1 targ 0 lun 0: <HL-DT-ST, DVDRAM GSA-U10N, 1.05> ATAPI 5/cdrom
removable
cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
pciide0: channel 1 ignored (disabled)
ahci0 at pci0 dev 31 function 2 "Intel 82801GBM AHCI" rev 0x02: msi, AHCI 1.1
scsibus2 at ahci0: 32 targets
sd0 at scsibus2 targ 0 lun 0: <ATA, INTEL SSDSC2CW24, 400i> SCSI3 0/direct
fixed naa.5001517bb2a98d08
sd0: 228936MB, 512 bytes/sector, 468862128 sectors, thin
ichiic0 at pci0 dev 31 function 3 "Intel 82801GB SMBus" rev 0x02: apic 1 int 23
iic0 at ichiic0
usb1 at uhci0: USB revision 1.0
uhub1 at usb1 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb2 at uhci1: USB revision 1.0
uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb3 at uhci2: USB revision 1.0
uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb4 at uhci3: USB revision 1.0
uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1
isa0 at ichpcib0
isadma0 at isa0
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard
pms0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
aps0 at isa0 port 0x1600/31
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
uhub5 at uhub0 port 6 "IBM product 0x4485" rev 2.00/0.01 addr 2
umass0 at uhub5 port 1 configuration 1 interface 0 "Seagate Backup+ Desk" rev
2.10/1.00 addr 3
umass0: using SCSI over Bulk-Only
scsibus3 at umass0: 2 targets, initiator 0
sd1 at scsibus3 targ 1 lun 0: <Seagate, Backup+ Desk, 050B> SCSI4 0/direct fixed
sd1: 3815447MB, 4096 bytes/sector, 976754645 sectors
uhidev0 at uhub5 port 3 configuration 1 interface 0 "Logitech USB-PS/2 Optical
Mouse" rev 2.00/20.00 addr 4
uhidev0: iclass 3/1
ums0 at uhidev0: 3 buttons, Z dir
wsmouse1 at ums0 mux 0
uhidev1 at uhub5 port 4 configuration 1 interface 0 "Gravis GamePad Pro USB"
rev 1.00/2.00 addr 5
uhidev1: iclass 3/0
uhid0 at uhidev1: input=4, output=0, feature=0
ugen0 at uhub4 port 2 "STMicroelectronics Biometric Coprocessor" rev 1.00/0.01
addr 2
vscsi0 at root
scsibus4 at vscsi0: 256 targets
softraid0 at root
scsibus5 at softraid0: 256 targets
softraid0: sd2 was not shutdown properly
sd2 at scsibus5 targ 1 lun 0: <OPENBSD, SR CRYPTO, 005> SCSI2 0/direct fixed
sd2: 200595MB, 512 bytes/sector, 410819160 sectors
root on sd2a (bfe3b486511fab55.a) swap on sd2b dump on sd2b
WARNING: / was not properly unmounted
drm: initializing kernel modesetting (RV515 0x1002:0x7149 0x17AA:0x2005).
radeondrm0: VRAM: 128M 0x0000000000000000 - 0x0000000007FFFFFF (64M used)
radeondrm0: GTT: 512M 0x0000000008000000 - 0x0000000027FFFFFF
drm: PCIE GART of 512M enabled (table at 0x0000000000040000).
radeondrm0: 1024x768
wsdisplay0 at radeondrm0 mux 1: console (std, vt100 emulation), using wskbd0
wsdisplay0: screen 1-5 added (std, vt100 emulation)
wpi0: radio is disabled by hardware switch
wpi0: could not initialize hardware
usbdevs:
Controller /dev/usb0:
addr 1: high speed, self powered, config 1, EHCI root hub(0x0000),
Intel(0x8086), rev 1.00
port 1 powered
port 2 powered
port 3 powered
port 4 powered
port 5 powered
port 6 addr 2: high speed, self powered, config 1, product 0x4485(0x4485),
IBM(0x04b3), rev 0.01
port 1 addr 3: high speed, self powered, config 1, Backup+ Desk(0xa0a4),
Seagate(0x0bc2), rev 1.00, iSerialNumber NA5KB06L
port 2 powered
port 3 addr 4: low speed, power 98 mA, config 1, USB-PS/2 Optical
Mouse(0xc03d), Logitech(0x046d), rev 20.00
port 4 addr 5: low speed, power 100 mA, config 1, GamePad Pro USB(0x4001),
Gravis(0x0428), rev 2.00
port 7 powered
port 8 powered
Controller /dev/usb1:
addr 1: full speed, self powered, config 1, UHCI root hub(0x0000),
Intel(0x8086), rev 1.00
port 1 powered
port 2 powered
Controller /dev/usb2:
addr 1: full speed, self powered, config 1, UHCI root hub(0x0000),
Intel(0x8086), rev 1.00
port 1 powered
port 2 powered
Controller /dev/usb3:
addr 1: full speed, self powered, config 1, UHCI root hub(0x0000),
Intel(0x8086), rev 1.00
port 1 powered
port 2 powered
Controller /dev/usb4:
addr 1: full speed, self powered, config 1, UHCI root hub(0x0000),
Intel(0x8086), rev 1.00
port 1 powered
port 2 addr 2: full speed, power 100 mA, config 1, Biometric
Coprocessor(0x2016), STMicroelectronics(0x0483), rev 0.01
