>Synopsis: uvm_fault(0xfffffe8420a7c648, 0x10, 0, 1) -> e
>Category: kernel
>Environment:
System : OpenBSD 5.5
Details : OpenBSD 5.5-current (GENERIC) #5: Sat Mar 8 05:13:55 EST
2014
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC
Architecture: OpenBSD.amd64
Machine : amd64
>Description:
"find" on a particular nfs filesystem hangs and eventually reports
'nfs server nas3:/backup: not responding'. A subsequent
"shutdown -fr now" from the serial console results in the
dropping to ddb with the uvm_fault message.
The fault occurs in nfs_reply(nfs_socket.c:772) according to
the instructions at [1]. Here's the related snippet from the
serial console:
# syncing disks... uvm_fault(0xfffffe8004681010, 0x10, 0, 1) -> e
kernel: page fault trap, code=0
Stopped at nfs_reply+0xe6: movq 0x10(%rdx),%rsi
ddb> trace
nfs_reply() at nfs_reply+0xe6
nfs_request() at nfs_request+0x1cb
nfs_readdirrpc() at nfs_readdirrpc+0x1bd
nfs_readdir() at nfs_readdir+0x173
VOP_READDIR() at VOP_READDIR+0x33
sys_getdents() at sys_getdents+0x113
syscall() at syscall+0x165
--- syscall (number 99) ---
end of kernel
end trace frame: 0x7f7fffff0d80, count: -7
0x712883e915a:
>How-To-Repeat:
I repeated the problem with the last amd64 snapshot, ie.:
OpenBSD 5.5 (GENERIC.MP) #315: Wed Mar 5 09:37:46 MST 2014
I then built a kernel with debug symbols that I thought
might help w/ further analysis. The subsequent dmesg and
ddb results are from that kernel. This problem initially
occurred on a bare metal box and is easily duplicated
in this VirtualBox scenario. Here's what I did to cause
the hang (the uvm_fault appears later in the serial
console capture:
i7v64 2503>grep nas3 /etc/fstab
nas3:/backup /nas3/backup nfs rw,noauto,nodev,nosuid,-b 0 0
i7v64 2504>sudo mount -r /nas3/backup
i7v64 2505>df -ih /nas3/backup
Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
nas3:/backup 10.8T 9.3T 1.5T 87% 2041573 180640027 1%
/nas3/backup
i7v64 2506>pushd /nas3/backup/m51
/nas3/backup/m51 ~/OpenBSD/build/packages ~/OpenBSD/build/common
i7v64 2507>time find . -type f
./SDT3DE9.tmp
./SDT7F9D.tmp
./SDTE52.tmp
./SDTEE05.tmp
./MediaID.bin
./SDTEE06.tmp
./SDTC0F1.tmp
./SDT44A9.tmp
./SDTC7C6.tmp
./M51/Backup Set 2010-08-08 142139/Catalogs/GlobalCatalog.wbcat
nfs server nas3:/backup: not responding
FWIW, using the native find on the nfs server yields:
nas3:backup/m51 235>time find . -type f | wc
4105 33298 390826
real 0m1.119s
user 0m0.029s
sys 0m0.020s
>Fix:
None known.
I've appended the serial console showing the shutdown problem and
subsequent ddb session.
[1]<http://www.openbsd.org/faq/faq2.html#Bugs>
dmesg:
OpenBSD 5.5-current (GENERIC) #5: Sat Mar 8 05:13:55 EST 2014
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC
real mem = 117440512 (112MB)
avail mem = 105902080 (100MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.5 @ 0xe1000 (10 entries)
bios0: vendor innotek GmbH version "VirtualBox" date 12/01/2006
bios0: innotek GmbH VirtualBox
acpi0 at bios0: rev 2
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP APIC SSDT
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz, 3487.47 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,SSSE3,NXE,LONG,LAHF,PERF
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: CPU supports MTRRs but not enabled by BIOS
cpu0: apic clock running at 1000MHz
cpu at mainbus0: not configured
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 11, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0
acpibat0 at acpi0: BAT0 not present
acpiac0 at acpi0: AC unit online
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
pciide0 at pci0 dev 1 function 1 "Intel 82371AB IDE" rev 0x01: DMA, channel 0
configured to compatibility, channel 1 configured to compatibility
pciide0: channel 0 disabled (no drives)
atapiscsi0 at pciide0 channel 1 drive 0
scsibus0 at atapiscsi0: 2 targets
cd0 at scsibus0 targ 0 lun 0: <VBOX, CD-ROM, 1.0> ATAPI 5/cdrom removable
cd0(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 2
vga1 at pci0 dev 2 function 0 "InnoTek VirtualBox Graphics Adapter" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
virtio0 at pci0 dev 3 function 0 "Qumranet Virtio Network" rev 0x00: Virtio
Network Device
vio0 at virtio0: address 08:00:27:06:9c:b4
virtio0: apic 2 int 19
"InnoTek VirtualBox Guest Service" rev 0x00 at pci0 dev 4 function 0 not
configured
auich0 at pci0 dev 5 function 0 "Intel 82801AA AC97" rev 0x01: apic 2 int 21,
ICH AC97
ac97: codec id 0x83847600 (SigmaTel STAC9700)
audio0 at auich0
ohci0 at pci0 dev 6 function 0 "Apple Intrepid USB" rev 0x00: apic 2 int 22,
version 1.0
piixpm0 at pci0 dev 7 function 0 "Intel 82371AB Power" rev 0x08: SMBus disabled
ehci0 at pci0 dev 11 function 0 "Intel 82801FB USB" rev 0x00: apic 2 int 19
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ahci0 at pci0 dev 13 function 0 "Intel 82801HBM AHCI" rev 0x02: apic 2 int 21,
AHCI 1.1
ahci0: device on port 0 didn't come ready, TFD: 0x171<ERR>
ahci0: device on port 1 didn't come ready, TFD: 0x171<ERR>
ahci0: device on port 2 didn't come ready, TFD: 0x171<ERR>
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, VBOX HARDDISK, 1.0> SCSI3 0/direct fixed
t10.ATA_VBOX_HARDDISK_VBbcc99a08-744eefa6_
sd0: 12288MB, 512 bytes/sector, 25165824 sectors
sd1 at scsibus1 targ 1 lun 0: <ATA, VBOX HARDDISK, 1.0> SCSI3 0/direct fixed
t10.ATA_VBOX_HARDDISK_VB9da9db2b-7414c638_
sd1: 20480MB, 512 bytes/sector, 41943040 sectors
sd2 at scsibus1 targ 2 lun 0: <ATA, VBOX HARDDISK, 1.0> SCSI3 0/direct fixed
t10.ATA_VBOX_HARDDISK_VBdaa85231-cc9edc4a_
sd2: 511978MB, 512 bytes/sector, 1048530944 sectors
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
usb1 at ohci0: USB revision 1.0
uhub1 at usb1 "Apple OHCI root hub" rev 1.00/1.00 addr 1
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (81831238603d6adb.a) swap on sd0b dump on sd0b
#######################################################################
############# serial console capture #########################
#######################################################################
OpenBSD/amd64 (i7v64.thrush.com) (tty00)
login: root
Password:
Last login: Sat Mar 8 01:05:15 on tty00
OpenBSD 5.5-current (GENERIC) #5: Sat Mar 8 05:13:55 EST 2014
# pgrep -l find
27044 find . -type f
# ps -p 27044
PID TT STAT TIME COMMAND
27044 p0 D+ 0:00.00 find . -type f
# ps -v -p 27044
PID STAT TIME SL RE PAGEIN VSZ RSS LIM TSIZ %CPU %MEM COMMAND
27044 D+ 0:00.00 3 127 0 412 964 100196 28 0.0 0.8 find . -t
# nfsstat -c
Client Info:
Getattr Setattr Lookup Readlink Read Write Create Remove
9 0 30 0 0 0 0 0
Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access
0 0 0 0 0 5 0 12
Mknod Fsstat Fsinfo PathConf Commit
0 11 4 0 0
Rpc Info:
TimedOut Invalid X Replies Retries Requests FrcSync
0 0 0 6 71 0
Cache Info:
Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits Misses
55 9 9 30 0 0 0 0
BioRLHits Misses BioD Hits Misses DirE Hits Misses
0 0 0 0 4 4
# ps -v -p 27044
PID STAT TIME SL RE PAGEIN VSZ RSS LIM TSIZ %CPU %MEM COMMAND
27044 D+ 0:00.00 5 127 0 412 964 100196 28 0.0 0.8 find . -t
# nfsstat -c
Client Info:
Getattr Setattr Lookup Readlink Read Write Create Remove
9 0 30 0 0 0 0 0
Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access
0 0 0 0 0 5 0 12
Mknod Fsstat Fsinfo PathConf Commit
0 11 4 0 0
Rpc Info:
TimedOut Invalid X Replies Retries Requests FrcSync
0 0 0 6 71 0
Cache Info:
Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits Misses
55 9 9 30 0 0 0 0
BioRLHits Misses BioD Hits Misses DirE Hits Misses
0 0 0 0 4 4
# shutdown -fr now
Shutdown NOW!
shutdown: [pid 24137]
*** FINAL System shutdown message from [email protected] ***
System going down IMMEDIATELY
#
System shutdown time has arrived
stopping package daemons:.
/etc/rc.shutdown in progress...
/etc/rc.shutdown complete.
# syncing disks... uvm_fault(0xfffffe8004681010, 0x10, 0, 1) -> e
kernel: page fault trap, code=0
Stopped at nfs_reply+0xe6: movq 0x10(%rdx),%rsi
ddb> trace
nfs_reply() at nfs_reply+0xe6
nfs_request() at nfs_request+0x1cb
nfs_readdirrpc() at nfs_readdirrpc+0x1bd
nfs_readdir() at nfs_readdir+0x173
VOP_READDIR() at VOP_READDIR+0x33
sys_getdents() at sys_getdents+0x113
syscall() at syscall+0x165
--- syscall (number 99) ---
end of kernel
end trace frame: 0x7f7fffff0d80, count: -7
0x712883e915a:
ddb> ps
PID PPID PGRP UID S FLAGS WAIT COMMAND
24137 1 24137 0 2 0x2 reboot
*27044 1 27044 1000 7 0x3 find
8481 0 0 0 3 0x4280 nfsidl nfsio
8231 0 0 0 3 0x4280 nfsidl nfsio
25653 0 0 0 3 0x4280 nfsidl nfsio
6948 0 0 0 3 0x4280 nfsidl nfsio
11008 0 0 0 3 0x4200 aiodoned aiodoned
14695 0 0 0 3 0x4200 syncer update
23434 0 0 0 3 0x4200 cleaner cleaner
14071 0 0 0 3 0x4200 reaper reaper
6349 0 0 0 3 0x4200 pgdaemon pagedaemon
3522 0 0 0 3 0x4200 bored crypto
4964 0 0 0 3 0x4200 pftm pfpurge
11199 0 0 0 3 0x4200 usbtsk usbtask
18345 0 0 0 3 0x4200 usbatsk usbatsk
25742 0 0 0 3 0x40004200 acpi0 acpi0
5110 0 0 0 3 0x4200 bored systq
5399 0 0 0 3 0x4200 bored syswq
11269 0 0 0 3 0x40004200 idle0
1 0 1 0 3 0x82 wait init
0 -1 0 0 3 0x200 scheduler swapper
ddb> show registers
ds 0
es 0
fs 0x3104
gs 0x99a0
rdi 0
rsi 0x14
rbp 0xffff800003a29a60
rbx 0xffff80000032c600
rdx 0
rcx 0x282
rax 0x100200 acpi_pdirpa+0xeeca0
r8 0x1
r9 0xffffffff81a689c0 pci_bus_dma_tag
r10 0x15
r11 0xffffffff81049ef0 ahci_ata_cmd
r12 0
r13 0xfffffe800619b000
r14 0xfffffe800619b000
r15 0xffff80000032c600
rip 0xffffffff8128a786 nfs_reply+0xe6
cs 0x8
rflags 0x10246
rsp 0xffff800003a299b0
ss 0x10
nfs_reply+0xe6: movq 0x10(%rdx),%rsi
ddb> show extents
extent `swap0x0001' (0x0 - 0x3fb18), flags=0
0x0 - 0x0
extent `swap0x0000' (0x0 - 0x90079), flags=0
0x0 - 0x0
0x90078 - 0x90078
extent `swapmap' (0x1 - 0x7fffffff), flags=0
0x1 - 0xcfb93
extent `pcibus' (0x0 - 0xff), flags=0
0x0 - 0x0
extent `pcimem' (0x0 - 0xffffffffffffffff), flags=0
0x0 - 0x7ffffff
0xfff0000 - 0xfffffff
0xe0000000 - 0xe0ffffff
0xf0000000 - 0xf0407fff
0xfffc0000 - 0xffffffff
0x1000000000 - 0xffffffffffffffff
extent `pciio' (0x0 - 0xffffffff), flags=0
0xd000 - 0xd00f
0xd020 - 0xd05f
0xd100 - 0xd249
0xd250 - 0xd259
0xd260 - 0xd26f
0x10000 - 0xffffffff
extent `iomem' (0x0 - 0xffffffffffff), flags=3<NOCOALESCE,FIXED>
0x20000 - 0x9efff
0xa0000 - 0xbffff
0x100000 - 0xffffff
0x1000000 - 0x7ffffff
0xf0404000 - 0xf0404fff
0xf0405000 - 0xf0405fff
0xf0406000 - 0xf0407fff
extent `ioport' (0x0 - 0xffff), flags=3<NOCOALESCE,FIXED>
0x0 - 0xb
0x40 - 0x43
0x60 - 0x60
0x61 - 0x61
0x64 - 0x64
0x80 - 0x8e
0xc0 - 0xd7
0x170 - 0x177
0x1f0 - 0x1f7
0x376 - 0x376
0x3c0 - 0x3cf
0x3d0 - 0x3df
0x3f6 - 0x3f6
0x3f8 - 0x3ff
0x4000 - 0x4001
0x4002 - 0x4003
0x4004 - 0x4005
0x4008 - 0x400b
0x4020 - 0x4020
0x4021 - 0x4021
0x442e - 0x442e
0xd000 - 0xd00f
0xd020 - 0xd03f
0xd100 - 0xd1ff
0xd200 - 0xd23f
ddb> show malloc
Type InUse MemUse HighUse Limit Requests Type Lim Kern Lim
devbuf 1174 670K 799K 17203K 1949 0 0
pcb 13 8K 22K 17203K 205 0 0
routetbl 69 7K 8K 17203K 120 0 0
ifaddr 34 12K 12K 17203K 34 0 0
ioctlops 0 0K 4K 17203K 1498 0 0
mount 8 8K 12K 17203K 12 0 0
NFS mount 0 0K 2K 17203K 4 0 0
vnodes 1555 97K 97K 17203K 1568 0 0
UFS quota 1 32K 32K 17203K 1 0 0
UFS mount 27 79K 79K 17203K 27 0 0
shm 2 1K 1K 17203K 2 0 0
VM map 2 0K 0K 17203K 2 0 0
sem 2 0K 0K 17203K 2 0 0
dirhash 45 8K 8K 17203K 45 0 0
ACPI 2068 230K 397K 17203K 9018 0 0
file desc 1 4K 29K 17203K 151 0 0
proc 12 9K 9K 17203K 12 0 0
VFS cluster 0 0K 0K 17203K 3 0 0
NFS srvsock 1 0K 0K 17203K 1 0 0
NFS daemon 1 16K 16K 17203K 1 0 0
in_multi 22 1K 1K 17203K 22 0 0
ether_multi 4 0K 0K 17203K 4 0 0
ISOFS mount 1 32K 32K 17203K 1 0 0
MSDOSFS mount 1 16K 16K 17203K 1 0 0
ttys 414 281K 281K 17203K 414 0 0
exec 0 0K 2K 17203K 681 0 0
miscfs mount 1 0K 0K 17203K 1 0 0
pfkey data 1 0K 0K 17203K 2 0 0
xform_data 0 0K 0K 17203K 8 0 0
pagedep 1 8K 8K 17203K 1 0 0
inodedep 1 32K 32K 17203K 3 0 0
newblk 1 0K 0K 17203K 1 0 0
indirdep 0 0K 16K 17203K 2 0 0
VM swap 12 244K 244K 17203K 13 0 0
UVM amap 70 6K 217K 17203K 28733 0 0
UVM aobj 3 2K 2K 17203K 11 0 0
USB 31 12K 12K 17203K 35 0 0
USB device 7 1K 1K 17203K 7 0 0
crypto data 1 1K 1K 17203K 1 0 0
NDP 6 0K 0K 17203K 7 0 0
temp 44 6K 158K 17203K 4671 0 0
ddb> show proc
PROC (find) pid=27044 stat=onproc
flags process=3<CONTROLT,EXEC,8STOPPED> proc=0
pri=24, usrpri=52, nice=20
forw=0xffffffffffffffff, list=0xfffffe80061a4248,0xfffffe8004820dc8
process=0xfffffe800481f498 user=0xffff800003a25000, vmspace=0xfffffe8004681
010
estcpu=2, cpticks=11, pctcpu=0.0, swtime=394
user=0, sys=11, intr=0
ddb> show uvmexp
Current UVM status:
pagesize=4096 (0x1000), pagemask=0xfff, pageshift=12
27902 VM pages: 364 active, 2553 inactive, 0 wired, 15061 free
min 10% (25) anon, 10% (25) vnode, 5% (12) vtext
pages 0 anon, 0 vnode, 0 vtext
freemin=930, free-target=1240, inactive-target=0, wired-max=9300
faults=116670, traps=113114, intrs=3444, ctxswitch=50382 fpuswitch=1007
softint=146070, syscalls=317935, kmapent=13
fault counts:
noram=0, noanon=0, pgwait=0, pgrele=0
ok relocks(total)=2816(2816), anget(retries)=72464(0), amapcopy=39034
neighbor anon/obj pg=8148/73789, gets(lock/unlock)=25314/2816
cases: anon=58943, anoncow=13521, obj=22250, prcopy=3064, przero=18891
daemon and swap counts:
woke=0, revs=0, scans=0, obscans=0, anscans=0
busy=0, freed=0, reactivate=0, deactivate=0
pageouts=0, pending=0, nswget=0
nswapdev=2, nanon=0, nanonneeded=0 nfreeanon=0
swpages=850833, swpginuse=0, swpgonly=0 paging=0
kernel pointers:
objs(kern)=0xffffffff81d6bb60
ddb> callout
ticks now: 135723
ticks wheel arg func
1 0d/44 d 0x0 m_cltick
18 0d/61 d 0xffffffff81d46f80 pffasttimo
28 0d/71 d 0xffffffff81d46f40 pfslowtimo
77 0d/120 d 0xffffffff81d47080 if_slowtimo
77 0d/120 d 0xffffffff81d99b40 pckbc_poll
77 0d/120 d 0xffffffff81d471a0 rt_timer_timer
77 0d/120 d 0xffffffff81d46c40 schedcpu
77 0d/120 d 0x0 nd6_timer
77 0d/120 d 0xfffffe800764d008 endtsleep
277 1d/275 d 0xffff8000000fe400 acpi_poll
14650 1d/331 d 0xffffffff81d471e0 arptimer
44277 1d/447 d 0x0 arc4_reinit
224277 2d/517 d 0x0 nd6_slowtimo
ddb> ps /w
PID COMMAND EMUL WAIT-CHANNEL WAIT-MSG
24137 reboot native 0x0
*27044 find native 0x0
8481 nfsio native 0xffffffff81dcaad0 nfsidl
8231 nfsio native 0xffffffff81dcaad0 nfsidl
25653 nfsio native 0xffffffff81dcaad0 nfsidl
6948 nfsio native 0xffffffff81dcaad0 nfsidl
11008 aiodoned native 0xffffffff81dcd900 aiodoned
14695 update native 0xffffffff81daae30 syncer
23434 cleaner native 0xffffffff81dafcc0 cleaner
14071 reaper native 0xffffffff81a494b0 reaper
6349 pagedaemon native 0xffffffff81dcd8f0 pgdaemon
3522 crypto native 0xffff800000157540 bored
4964 pfpurge native 0xffffffff8112ca40 pftm
11199 usbtask native 0xffffffff81d96e44 usbtsk
18345 usbatsk native 0xffffffff81d96e48 usbatsk
25742 acpi0 native 0xffff8000000fe400 acpi0
5110 systq native 0xffffffff81a4a040 bored
5399 syswq native 0xffffffff81a4a000 bored
11269 idle0 native 0x0
1 init native 0xfffffe8007677000 wait
0 swapper native 0xffffffff81da2260 scheduler
ddb> ps /a
PID COMMAND STRUCT PROC * UAREA * VMSPACE/VM_MAP
24137 reboot 0xfffffe8004820db8 0xffff800003a34000 0xfffffe8004681390
*27044 find 0xfffffe8004820498 0xffff800003a25000 0xfffffe8004681010
8481 nfsio 0xfffffe80061a4248 0xffff8000039a0000 0xffffffff81da1f00
8231 nfsio 0xfffffe80061a4000 0xffff80000399b000 0xffffffff81da1f00
25653 nfsio 0xfffffe800653edb8 0xffff800003996000 0xffffffff81da1f00
6948 nfsio 0xfffffe8007579490 0xffff80000396b000 0xffffffff81da1f00
11008 aiodoned 0xfffffe800764ddb8 0xffff80000394d000 0xffffffff81da1f00
14695 update 0xfffffe800764db70 0xffff800003948000 0xffffffff81da1f00
23434 cleaner 0xfffffe800764d928 0xffff800003943000 0xffffffff81da1f00
14071 reaper 0xfffffe800764d6e0 0xffff80000393e000 0xffffffff81da1f00
6349 pagedaemon 0xfffffe800764d498 0xffff800003939000 0xffffffff81da1f00
3522 crypto 0xfffffe800764d250 0xffff800003934000 0xffffffff81da1f00
4964 pfpurge 0xfffffe800764d008 0xffff80000392f000 0xffffffff81da1f00
11199 usbtask 0xfffffe8007678db0 0xffff80000392a000 0xffffffff81da1f00
18345 usbatsk 0xfffffe8007678b68 0xffff800003925000 0xffffffff81da1f00
25742 acpi0 0xfffffe8007678920 0xffff800003920000 0xffffffff81da1f00
5110 systq 0xfffffe80076786d8 0xffff80000391b000 0xffffffff81da1f00
5399 syswq 0xfffffe8007678490 0xffff800003916000 0xffffffff81da1f00
11269 idle0 0xfffffe8007678248 0xffff800003911000 0xffffffff81da1f00
1 init 0xfffffe8007678000 0xffff80000390c000 0xfffffe8007674000
0 swapper 0xffffffff81da2260 0xffffffff81ebd000 0xffffffff81da1f00
ddb> show bcstats
Current Buffer Cache status:
numbufs 1379 busymapped 1, delwri 0
kvaslots 348 avail kva slots 347
bufpages 5511, dirtypages 0
pendingreads 0, pendingwrites 0
ddb> show all pools
Name Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
inpcbpl 360 101 0 101 3 0 3 3 0 8 3
plimitpl 152 34 0 32 1 0 1 1 0 8 0
synpl 248 4 0 4 1 0 1 1 0 8 1
tcpcbpl 560 23 0 23 2 0 2 2 0 8 2
rtmask 32 10 0 0 1 0 1 1 0 8 0
rtentpl 192 35 0 0 2 0 2 2 0 8 0
strpolpl 48 4 0 4 1 0 1 1 0 8 1
strprocpl 2456 8 0 8 2 0 2 2 0 8 2
pfosfp 40 840 0 420 5 0 5 5 0 8 0
pfosfpen 112 1420 0 710 34 13 21 21 0 8 0
pfstitem 24 109 0 95 1 0 1 1 0 8 0
pfstkey 104 109 0 95 1 0 1 1 0 8 0
pfstate 312 109 0 95 3 0 3 3 0 8 1
pfrule 1336 18 0 15 6 0 6 6 0 8 5
tmpfs_node 240 13 0 6 1 0 1 1 0 8 0
tmpfs_dirent 48 12 0 6 1 0 1 1 0 8 0
nfsnodepl 472 32 0 32 4 0 4 4 0 8 4
dirhash 1024 57 0 0 15 0 15 15 0 8 0
dirrem 64 58 0 58 1 0 1 1 0 8 1
mkdir 56 6 0 6 1 0 1 1 0 8 1
diradd 56 59 0 59 1 0 1 1 0 8 1
freefile 48 43 0 43 1 0 1 1 0 8 1
freeblks 192 46 0 46 1 0 1 1 0 8 1
freefrag 64 4 0 4 1 0 1 1 0 8 1
allocindir 104 85 0 85 3 0 3 3 0 8 3
indirdep 56 3 0 3 1 0 1 1 0 8 1
allocdir 128 79 0 79 2 0 2 2 0 8 2
bmsafemap 64 28 0 28 1 0 1 1 0 8 1
newblk 64 164 0 164 1 0 1 1 0 8 1
inodedep 152 101 0 101 3 0 3 3 0 8 3
pagedep 128 30 0 30 1 0 1 1 0 8 1
dino1pl 128 2226 0 46 71 0 71 71 0 8 0
ffsino 216 2226 0 46 122 0 122 122 0 8 0
nchpl 144 3643 0 129 132 0 132 132 0 8 0
vnodes 264 2291 0 0 153 0 153 153 0 8 0
namei 1024 9732 0 9732 2 0 2 2 0 8 2
nfsreqpl 96 71 0 70 1 0 1 1 0 8 0
scxspl 192 2822 0 2822 1 0 1 1 0 8 1
wdcxfer 176 14 0 11 1 0 1 1 0 8 0
ifaddritem 64 10 0 0 1 0 1 1 0 8 0
wqtasks 40 5 0 5 1 0 1 1 0 8 1
sigapl 432 541 0 538 5 0 5 5 0 8 2
knotepl 112 109629 0 109629 3 0 3 3 0 8 3
kqueuepl 320 10 0 10 1 0 1 1 0 8 1
pipepl 120 488 0 488 1 0 1 1 0 8 1
fdescpl 440 542 0 538 5 0 5 5 0 8 3
filepl 120 4076 0 4071 6 0 6 6 0 8 3
lockfpl 88 24 0 24 1 0 1 1 0 8 1
pcredpl 24 558 0 538 1 0 1 1 0 8 0
sessionpl 64 35 0 32 1 0 1 1 0 8 0
pgrppl 40 111 0 108 1 0 1 1 0 8 0
ucredpl 80 376 0 372 1 0 1 1 0 8 0
zombiepl 144 538 0 538 1 0 1 1 0 8 1
processpl 584 558 0 538 9 0 9 9 0 8 3
procpl 584 558 0 538 9 0 9 9 0 8 3
sockpl 472 293 0 293 17 9 8 17 0 8 8
mcl2k 2048 5885 0 5874 25 0 25 25 4 3072 19
mtagpl 72 1 0 1 1 0 1 1 0 8 1
mbpl 256 24099 0 24084 9 0 9 9 1 384 7
bufpl 272 2140 0 761 93 0 93 93 0 8 0
anonpl 16 35476 0 35375 17 1 16 17 0 7 7
amappl 72 28384 0 28316 39 0 39 39 0 75 25
dma4096 4096 3 0 3 1 0 1 1 0 8 1
dma512 512 8 0 5 1 0 1 1 0 8 0
dma256 256 21 0 21 1 0 1 1 0 8 1
dma128 128 3 0 3 1 0 1 1 0 8 1
dma64 64 9 0 9 1 0 1 1 0 8 1
dma32 32 18 0 18 1 0 1 1 0 8 1
dma16 16 3 0 3 1 0 1 1 0 8 1
aobjpl 64 8 0 6 1 0 1 1 0 8 0
uaddrrnd 40 542 0 538 1 0 1 1 0 8 0
uaddrbest 32 2 0 0 1 0 1 1 0 8 0
uaddr 24 542 0 538 1 0 1 1 0 8 0
vmmpekpl 168 2617 0 2612 1 0 1 1 0 8 0
vmmpepl 168 59891 0 59784 175 0 175 175 0 357 165
vmsppl 224 541 0 538 3 0 3 3 0 8 1
pdppl 4096 541 0 538 45 34 11 41 0 8 8
pvpl 32 148537 0 148108 101 0 101 101 0 265 91
pmappl 152 541 0 538 2 0 2 2 0 8 0
extentpl 40 61 0 35 1 0 1 1 0 8 0
phpool 104 459 0 43 12 0 12 12 0 8 0
ddb> show all mounts
flags 400050000<LOCAL,ROOTFS>
vnodecovered 0x0 syncer 0xfffffe80073b6850 data 0xffff800000230800
vfsconf: ops 0xffffffff817e1260 name "ffs" num 1 ref 7 flags 0x1000
statvfs cache: bsize 800 iosize 4000
blocks 83871 free 24315 avail 20122
files 21758 ffiles 18831 favail 18831
f_fsidx {0x400, 0x439c1eb7} owner 0 ctime 0x531af114
syncwrites 19 asyncwrites = 157
syncreads 9051 asyncreads = 0
fstype "ffs" mnton "/" mntfrom "/dev/sd0a" mntspec "/dev/sd0a"
locked vnodes:
0xfffffe8007290b70
flags 400010030<NOSUID,NODEV,LOCAL>
vnodecovered 0xfffffe80073b6a60 syncer 0xfffffe80073b6b68 data 0xffff8000002384
00
vfsconf: ops 0xffffffff817e1260 name "ffs" num 1 ref 7 flags 0x1000
statvfs cache: bsize 800 iosize 4000
blocks 1725207 free 1505297 avail 1419037
files 441726 ffiles 437648 favail 437648
f_fsidx {0x40a, 0xcbe0ca67} owner 0 ctime 0x531af111
syncwrites 2 asyncwrites = 71
syncreads 427 asyncreads = 0
fstype "ffs" mnton "/home" mntfrom "/dev/sd0k" mntspec "/dev/sd0k"
locked vnodes:
flags 10020<NODEV,LOCAL>
vnodecovered 0xfffffe80073b6c70 syncer 0xfffffe80073b6e80 data 0xffff80000014df
80
vfsconf: ops 0xffffffff81a4b600 name "tmpfs" num 19 ref 1 flags 0x1000
statvfs cache: bsize 1000 iosize 1000
blocks 524288 free 524287 avail 524287
files 2097155 ffiles 2097148 favail 2097148
f_fsidx {0x2701, 0x13} owner 0 ctime 0x531af111
syncwrites 0 asyncwrites = 0
syncreads 0 asyncreads = 0
fstype "tmpfs" mnton "/tmp" mntfrom "tmpfs" mntspec "tmpfs"
locked vnodes:
flags 400010021<RDONLY,NODEV,LOCAL>
vnodecovered 0xfffffe80075cf740 syncer 0x0 data 0xffff80000027c400
vfsconf: ops 0xffffffff817e1260 name "ffs" num 1 ref 7 flags 0x1000
statvfs cache: bsize 800 iosize 4000
blocks 1034191 free 704871 avail 653162
files 285822 ffiles 262363 favail 262363
f_fsidx {0x410, 0x4e875ee3} owner 0 ctime 0x531af111
syncwrites 0 asyncwrites = 0
syncreads 4768 asyncreads = 0
fstype "ffs" mnton "/usr" mntfrom "/dev/sd1a" mntspec "/dev/sd1a"
locked vnodes:
flags 400010021<RDONLY,NODEV,LOCAL>
vnodecovered 0xfffffe8007290438 syncer 0x0 data 0xffff80000027e000
vfsconf: ops 0xffffffff817e1260 name "ffs" num 1 ref 7 flags 0x1000
statvfs cache: bsize 800 iosize 4000
blocks 4080951 free 3790740 avail 3586693
files 1039358 ffiles 1011229 favail 1011229
f_fsidx {0x41b, 0xd4b880e4} owner 0 ctime 0x531af111
syncwrites 0 asyncwrites = 0
syncreads 380 asyncreads = 0
fstype "ffs" mnton "/usr/local" mntfrom "/dev/sd1l" mntspec "/dev/sd1l"
locked vnodes:
flags 110120<NODEV,ASYNC,LOCAL>
vnodecovered 0xfffffe8007290648 syncer 0xfffffe8007290750 data 0xffff80000027ec
00
vfsconf: ops 0xffffffff817e1260 name "ffs" num 1 ref 7 flags 0x1000
statvfs cache: bsize 800 iosize 4000
blocks 4641967 free 4635191 avail 4403093
files 1195262 ffiles 1193744 favail 1193744
f_fsidx {0x41e, 0x6bfcd055} owner 0 ctime 0x531af111
syncwrites 0 asyncwrites = 2
syncreads 1 asyncreads = 0
fstype "ffs" mnton "/usr/obj" mntfrom "/dev/sd1o" mntspec "/dev/sd1o"
locked vnodes:
flags 400010030<NOSUID,NODEV,LOCAL>
vnodecovered 0xfffffe8007290960 syncer 0xfffffe8007290a68 data 0xffff8000002814
00
vfsconf: ops 0xffffffff817e1260 name "ffs" num 1 ref 7 flags 0x1000
statvfs cache: bsize 800 iosize 4000
blocks 121975 free 107918 avail 101820
files 39038 ffiles 37401 favail 37401
f_fsidx {0x404, 0x75371688} owner 0 ctime 0x531af111
syncwrites 92 asyncwrites = 672
syncreads 4354 asyncreads = 0
fstype "ffs" mnton "/var" mntfrom "/dev/sd0e" mntspec "/dev/sd0e"
locked vnodes:
flags 400010030<NOSUID,NODEV,LOCAL>
vnodecovered 0xfffffe8007290c78 syncer 0x0 data 0xffff800000281800
vfsconf: ops 0xffffffff817e1260 name "ffs" num 1 ref 7 flags 0x1000
statvfs cache: bsize 2000 iosize 10000
blocks 651361 free 593676 avail 561108
files 204798 ffiles 202780 favail 202780
f_fsidx {0x406, 0x7c1af260} owner 0 ctime 0x531af111
syncwrites 2 asyncwrites = 2
syncreads 19 asyncreads = 0
fstype "ffs" mnton "/pub" mntfrom "/dev/sd0g" mntspec "/dev/sd0g"
locked vnodes:
ddb> show all nfsreqs
xid 0xa4626c0 flags 0x22 rexmit 7 procnum 16 proc 0xfffffe8004820498
ddb> show all nfsreqs /f
xid 0xa4626c0 flags 0x22 rexmit 7 procnum 16 proc 0xfffffe8004820498
mreq 0xffffffff817dfb05 mrep 0xfffffe8007683600 md 0x0 nfsmount 0xdeadbeefdeadb
eef vnode 0xffff80000032c600 timer 64294304
ddb> show all nfsnodes
ddb> show all vnodes
tag NON(0) type VBAD(8) mount 0x0 typedata 0x0
data 0x0 usecount 0 writecount 0 holdcnt 0 numoutput 0
tag NON(0) type VBAD(8) mount 0x0 typedata 0x0
data 0x0 usecount 0 writecount 0 holdcnt 0 numoutput 0
tag NON(0) type VBAD(8) mount 0x0 typedata 0x0
data 0x0 usecount 0 writecount 0 holdcnt 0 numoutput 0
tag NON(0) type VBAD(8) mount 0x0 typedata 0x0
data 0x0 usecount 0 writecount 0 holdcnt 0 numoutput 0
[ ... removed 4564 additional lines ...]
tag UFS(1) type VREG(1) mount 0xffff800000230400 typedata 0x0
data 0xfffffe80069727e0 usecount 0 writecount 0 holdcnt 1 numoutput 0
tag UFS(1) type VREG(1) mount 0xffff800000230400 typedata 0x0
data 0xfffffe80069728b8 usecount 2 writecount 0 holdcnt 11 numoutput 0
tag UFS(1) type VCHR(4) mount 0xffff800000230400 typedata 0xffff80000033d200
data 0xfffffe8006972990 usecount 0 writecount 0 holdcnt 0 numoutput 0
tag NON(0) type VBAD(8) mount 0x0 typedata 0x0
data 0x0 usecount 0 writecount 0 holdcnt 0 numoutput 0
tag UFS(1) type VREG(1) mount 0xffff800000230400 typedata 0x0
data 0xfffffe8006972a68 usecount 0 writecount 0 holdcnt 1 numoutput 0
ddb> boot crash
dumping to dev 4,1 offset 4458455
dump 127 127 126 125 124 123 122 121 120 119 118 117 116 115 114 113 112 111
110 109 108 107 106 105 104 103 102 101 100 99 98 97 96 95 94 93 92 91 90 89 88
87 86 85 84 83 82 81 80 79 78 77 76 75 74 73 72 71 70 69 68 67 66 65 64 63 62
61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37 36
35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9
8 7 6 5 4 3 2 1 succeeded
done
rebooting...
>> OpenBSD/amd64 BOOT 3.28
Region 0: type 1 at 0x0 for 639KB
Region 1: type 2 at 0x9fc00 for 1KB
Region 2: type 2 at 0xf0000 for 64KB
Region 3: type 1 at 0x100000 for 130048KB
Region 4: type 3 at 0xfff0000 for 64KB
Region 5: type 2 at 0xfffc0000 for 256KB
Low ram: 639KB High ram: 261056KB
Total free memory: 130687KB
Region 0: type 1 at 0x0 for 639KB
Region 1: type 2 at 0x9fc00 for 1KB
Region 2: type 2 at 0xf0000 for 64KB
Region 3: type 1 at 0x100000 for 130048KB
Region 4: type 3 at 0xfff0000 for 64KB
Region 5: type 2 at 0xfffc0000 for 256KB
Low ram: 639KB High ram: 261056KB
Total free memory: 130687KB
Disk BIOS# Type Cyls Heads Secs Flags Checksum
hd0 0x80 label 1024 255 63 0x2 0xd7e79676
hd1 0x81 label 1024 255 63 0x2 0xd76796b2
hd2 0x82 label 1024 255 63 0x0 0xf0e40e4f
boot> boot -s
booting hd0a:bsd.sp.syms: 7607260+2057772+1097360+0+612768
[100+554472+368993]=0xfbb498
entry point at 0x10001e0 [7205c766, 34000004, 24448b12, 1cd8a304]
[ using 924408 bytes of bsd ELF symbol table ]
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
Copyright (c) 1995-2014 OpenBSD. All rights reserved. http://www.OpenBSD.org
OpenBSD 5.5-current (GENERIC) #5: Sat Mar 8 05:13:55 EST 2014
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC
real mem = 117440512 (112MB)
avail mem = 105902080 (100MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.5 @ 0xe1000 (10 entries)
bios0: vendor innotek GmbH version "VirtualBox" date 12/01/2006
bios0: innotek GmbH VirtualBox
acpi0 at bios0: rev 2
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP APIC SSDT
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz, 3487.47 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,SSSE3,NXE,LONG,LAHF,PERF
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: CPU supports MTRRs but not enabled by BIOS
cpu0: apic clock running at 1000MHz
cpu at mainbus0: not configured
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 11, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0
acpibat0 at acpi0: BAT0 not present
acpiac0 at acpi0: AC unit online
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
pciide0 at pci0 dev 1 function 1 "Intel 82371AB IDE" rev 0x01: DMA, channel 0
configured to compatibility, channel 1 configured to compatibility
pciide0: channel 0 disabled (no drives)
atapiscsi0 at pciide0 channel 1 drive 0
scsibus0 at atapiscsi0: 2 targets
cd0 at scsibus0 targ 0 lun 0: <VBOX, CD-ROM, 1.0> ATAPI 5/cdrom removable
cd0(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 2
vga1 at pci0 dev 2 function 0 "InnoTek VirtualBox Graphics Adapter" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
virtio0 at pci0 dev 3 function 0 "Qumranet Virtio Network" rev 0x00: Virtio
Network Device
vio0 at virtio0: address 08:00:27:06:9c:b4
virtio0: apic 2 int 19
"InnoTek VirtualBox Guest Service" rev 0x00 at pci0 dev 4 function 0 not
configured
auich0 at pci0 dev 5 function 0 "Intel 82801AA AC97" rev 0x01: apic 2 int 21,
ICH AC97
ac97: codec id 0x83847600 (SigmaTel STAC9700)
audio0 at auich0
ohci0 at pci0 dev 6 function 0 "Apple Intrepid USB" rev 0x00: apic 2 int 22,
version 1.0
piixpm0 at pci0 dev 7 function 0 "Intel 82371AB Power" rev 0x08: SMBus disabled
ehci0 at pci0 dev 11 function 0 "Intel 82801FB USB" rev 0x00: apic 2 int 19
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ahci0 at pci0 dev 13 function 0 "Intel 82801HBM AHCI" rev 0x02: apic 2 int 21,
AHCI 1.1
ahci0: device on port 0 didn't come ready, TFD: 0x171<ERR>
ahci0: device on port 1 didn't come ready, TFD: 0x171<ERR>
ahci0: device on port 2 didn't come ready, TFD: 0x171<ERR>
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, VBOX HARDDISK, 1.0> SCSI3 0/direct fixed
t10.ATA_VBOX_HARDDISK_VBbcc99a08-744eefa6_
sd0: 12288MB, 512 bytes/sector, 25165824 sectors
sd1 at scsibus1 targ 1 lun 0: <ATA, VBOX HARDDISK, 1.0> SCSI3 0/direct fixed
t10.ATA_VBOX_HARDDISK_VB9da9db2b-7414c638_
sd1: 20480MB, 512 bytes/sector, 41943040 sectors
sd2 at scsibus1 targ 2 lun 0: <ATA, VBOX HARDDISK, 1.0> SCSI3 0/direct fixed
t10.ATA_VBOX_HARDDISK_VBdaa85231-cc9edc4a_
sd2: 511978MB, 512 bytes/sector, 1048530944 sectors
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
usb1 at ohci0: USB revision 1.0
uhub1 at usb1 "Apple OHCI root hub" rev 1.00/1.00 addr 1
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (81831238603d6adb.a) swap on sd0b dump on sd0b
Enter pathname of shell or RETURN for sh:
# time fsck -yp
/dev/rsd0a: file system is clean; not checking
/dev/rsd0k: file system is clean; not checking
/dev/rsd0e: file system is clean; not checking
/dev/rsd0g: file system is clean; not checking
/dev/rsd1a: file system is clean; not checking
/dev/rsd1l: file system is clean; not checking
/dev/rsd1o: file system is clean; not checking
0m0.01s real 0m0.00s user 0m0.01s system
# exit
swapctl: adding /dev/sd1b as swap device at priority 0
Fast boot: skipping disk checks.
setting tty flags
pf enabled
ddb.panic: 1 -> 1
ddb.console: 0 -> 1
ddb.max_line: 24 -> 0
machdep.allowaperture: 0 -> 2
starting network
DHCPREQUEST on vio0 to 255.255.255.255 port 67
DHCPACK from 10.1.2.18 (14:da:e9:b5:84:cf)
bound to 10.1.2.143 -- renewal in 302400 seconds.
starting early daemons: syslogd pflogd ntpd.
starting RPC daemons:.
dumpoff = 2282728960 (4458455 * 512)
savecore: reboot
savecore: system went down at Sat Mar 8 05:52:13 2014
savecore: writing compressed core to /var/crash/bsd.2.core.Z
130556K 129532K 128508K 127484K 126460K 125436K 124412K 123388K
122364K 121340K 120316K 119292K 118268K 117244K 116220K 115196K 114172K
113148K 112124K 111100K 110076K 109052K 108028K 107004K 105980K
104956K 103932K 102908K 101884K 100860K 99836K 98812K 97788K 96764K
95740K 94716K 93692K 92668K 91644K 90620K 89596K 88572K
87548K 86524K 85500K 84476K 83452K 82428K 81404K 80380K 79356K
78332K 77308K 76284K 75260K 74236K 73212K 72188K 71164K 70140K
69116K 68092K 67068K 66044K 65020K 63996K 62972K 61948K
60924K 59900K 58876K 57852K 56828K 55804K 54780K 53756K 52732K
51708K 50684K 49660K 48636K 47612K 46588K 45564K 44540K 43516K
42492K 41468K 40444K 39420K 38396K 37372K 36348K 35324K
34300K 33276K 32252K 31228K 30204K 29180K 28156K 27132K 26108K
25084K 24060K 23036K 22012K 20988K 19964K 18940!
K 17916K
16892K 15868K 14844K 13820K 12796K 11772K 10748K 9724K
8700K 7676K 6652K 5628K 4604K 3580K 2556K 1532K
508Ksavecore: writing compressed kernel to /var/crash/bsd.2.Z
checking quotas: done.
clearing /tmp
starting pre-securelevel daemons:.
setting kernel security level: kern.securelevel: 0 -> 1
creating runtime link editor directory cache.
preserving editor files.
starting network daemons: sshd smtpd ftpd sndiod.
starting package daemons: darkstat(absent).
starting local daemons: apmd cron.
Sat Mar 8 06:17:52 EST 2014
OpenBSD/amd64 (i7v64.thrush.com) (tty00)
login:
