>Synopsis: Repeatable panic w/ re0 in an updated network
>Category: kernel
>Environment:
System : OpenBSD 5.7
Details : OpenBSD 5.7-current (GENERIC.MP) #987: Sun May 17
20:42:55 MDT 2015
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
Architecture: OpenBSD.amd64
Machine : amd64
>Description:
This machine is used heavily and worked well until I made some
network changes yesterday which resulted in the above panic.
I narrowed the problem to nfs usage w/ the re(4) interface.
Restarting the box w/ msk0 rather than re0 worked correctly in
the same nfs scenario.
The problem occurred after I made network changes including
replacing some unmanaged switches with an HP1820-24g. I
configured the HP1820 and the nas boxes to do link aggregation
and jumbo frames.
The original panic occurred w/ the May 13 snap. I upgraded to
the May 17 snap w/ the same results. My other OpenBSD boxes
don't panic when accessing the same nfs resource. I did
notice (probably unrelated) a network hang w/ a soekris 5501
when mounted w/ udp but no hang w/ a tcp mount.
>How-To-Repeat:
dd if=nfs-file of=/dev/null bs=32k
I've appended the serial console transcript which includes the
dmesg, panic and subsequent ddb session.
>Fix:
Don't know. I was hoping to provide more info by building a
debug kernel; however, ddb 'boot dump' results in a different
panic. I have saved that serial console transcript but didn't
want to further clutter this report.
>> OpenBSD/amd64 BOOT 3.28
Disk BIOS# Type Cyls Heads Secs Flags Checksum
fd0 0x0 *none* 80 2 18 0x4 0x0
hd0 0x80 label 1024 255 63 0x2 0xd7479677
hd1 0x81 label 1024 135 14 0x0 0xf6a8b72a
hd2 0x82 label 1024 13 36 0x0 0xde0dcb05
hd3 0x83 label 1024 87 10 0x0 0xa03fbf9c
Region 0: type 1 at 0x0 for 635KB
Region 1: type 2 at 0x9ec00 for 5KB
Region 2: type 2 at 0xe4000 for 112KB
Region 3: type 1 at 0x100000 for 3406336KB
Region 4: type 3 at 0xcff80000 for 56KB
Region 5: type 4 at 0xcff8e000 for 264KB
Region 6: type 2 at 0xcffd0000 for 64KB
Region 7: type 2 at 0xcffe0000 for 128KB
Region 8: type 2 at 0xff700000 for 9216KB
Region 9: type 1 at 0x100000000 for 4718592KB
Low ram: 635KB High ram: 3406336KB
Total free memory: 8125563KB
boot>
booting hd0a:/bsd: 6652640+2125720+254984+0+598016 [72+559440+371817]=0xa14888
entry point at 0x1000160 [7205c766, 34000004, 24448b12, 9060a304]
[ using 931976 bytes of bsd ELF symbol table ]
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
Copyright (c) 1995-2015 OpenBSD. All rights reserved. http://www.OpenBSD.org
OpenBSD 5.7-current (GENERIC.MP) #987: Sun May 17 20:42:55 MDT 2015
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8304197632 (7919MB)
avail mem = 8048680960 (7675MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.5 @ 0x9f000 (72 entries)
bios0: vendor American Megatrends Inc. version "2701" date 10/08/2010
bios0: ASUSTeK Computer INC. M3A78-EM
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S3 S4 S5
acpi0: tables DSDT FACP APIC MCFG OEMB SRAT HPET SSDT
acpi0: wakeup devices PCE2(S4) PCE3(S4) PCE4(S4) PCE5(S4) PCE6(S4) RLAN(S4)
PCE7(S4) PCE9(S4) PCEA(S4) PCEB(S4) PCEC(S4) SBAZ(S4) PS2M(S4) PS2K(S4)
UAR1(S4) P0PC(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD Phenom(tm) 9550 Quad-Core Processor, 2212.16 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,PAGE1GB,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,ITSC
cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB 64b/line
16-way L2 cache, 2MB 64b/line 32-way L3 cache
cpu0: ITLB 32 4KB entries fully associative, 16 4MB entries fully associative
cpu0: DTLB 48 4KB entries fully associative, 48 4MB entries fully associative
cpu0: AMD erratum 721 detected and fixed
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 201MHz
cpu0: mwait min=64, max=64, C-substates=0.0.0.0.0, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD Phenom(tm) 9550 Quad-Core Processor, 2211.90 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,PAGE1GB,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,ITSC
cpu1: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB 64b/line
16-way L2 cache, 2MB 64b/line 32-way L3 cache
cpu1: ITLB 32 4KB entries fully associative, 16 4MB entries fully associative
cpu1: DTLB 48 4KB entries fully associative, 48 4MB entries fully associative
cpu1: AMD erratum 721 detected and fixed
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 2 (application processor)
cpu2: AMD Phenom(tm) 9550 Quad-Core Processor, 2211.90 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,PAGE1GB,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,ITSC
cpu2: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB 64b/line
16-way L2 cache, 2MB 64b/line 32-way L3 cache
cpu2: ITLB 32 4KB entries fully associative, 16 4MB entries fully associative
cpu2: DTLB 48 4KB entries fully associative, 48 4MB entries fully associative
cpu2: AMD erratum 721 detected and fixed
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 3 (application processor)
cpu3: AMD Phenom(tm) 9550 Quad-Core Processor, 2211.90 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,PAGE1GB,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,ITSC
cpu3: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB 64b/line
16-way L2 cache, 2MB 64b/line 32-way L3 cache
cpu3: ITLB 32 4KB entries fully associative, 16 4MB entries fully associative
cpu3: DTLB 48 4KB entries fully associative, 48 4MB entries fully associative
cpu3: AMD erratum 721 detected and fixed
cpu3: smt 0, core 3, package 0
ioapic0 at mainbus0: apid 4 pa 0xfec00000, version 21, 24 pins
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255
acpihpet0 at acpi0: 14318180 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (P0P1)
acpiprt2 at acpi0: bus -1 (PCE2)
acpiprt3 at acpi0: bus -1 (PCE3)
acpiprt4 at acpi0: bus 2 (PCE4)
acpiprt5 at acpi0: bus -1 (PCE5)
acpiprt6 at acpi0: bus 3 (PCE6)
acpiprt7 at acpi0: bus 4 (P0PC)
acpicpu0 at acpi0: PSS
acpicpu1 at acpi0: PSS
acpicpu2 at acpi0: PSS
acpicpu3 at acpi0: PSS
aibs0 at acpi0 RTMP RVLT RFAN GGRP GITM SITM
acpibtn0 at acpi0: PWRB
cpu0: 2212 MHz: speeds: 2200 1100 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "AMD RS780 Host" rev 0x00
ppb0 at pci0 dev 1 function 0 vendor "Asustek", unknown product 0x9602 rev 0x00
pci1 at ppb0 bus 1
radeondrm0 at pci1 dev 5 function 0 "ATI Radeon HD 3200" rev 0x00
drm0 at radeondrm0
radeondrm0: apic 4 int 18
azalia0 at pci1 dev 5 function 1 "ATI RS780 HD Audio" rev 0x00: msi
azalia0: no supported codecs
ppb1 at pci0 dev 4 function 0 "AMD RS780 PCIE" rev 0x00: msi
pci2 at ppb1 bus 2
mskc0 at pci2 dev 0 function 0 "Marvell Yukon 88E8053" rev 0x20, Yukon-2 EC
rev. A3 (0x2): apic 4 int 16
msk0 at mskc0 port A: address 00:50:43:00:7c:9a
eephy0 at msk0 phy 0: 88E1111 Gigabit PHY, rev. 2
ppb2 at pci0 dev 6 function 0 "AMD RS780 PCIE" rev 0x00: msi
pci3 at ppb2 bus 3
re0 at pci3 dev 0 function 0 "Realtek 8168" rev 0x02: RTL8168C/8111C (0x3c00),
msi, address 00:22:15:2c:7d:0b
rgephy0 at re0 phy 7: RTL8169S/8110S PHY, rev. 2
ahci0 at pci0 dev 17 function 0 "ATI SBx00 SATA" rev 0x00: apic 4 int 22, AHCI
1.1
ahci0: port 0: 3.0Gb/s
ahci0: port 1: 3.0Gb/s
ahci0: port 2: 1.5Gb/s
ahci0: port 3: 3.0Gb/s
ahci0: port 5: 3.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0: <ATA, WDC WD1003FZEX-0, 01.0> SCSI3 0/direct
fixed naa.50014ee2b546ec7f
sd0: 953869MB, 512 bytes/sector, 1953525168 sectors
sd1 at scsibus1 targ 1 lun 0: <ATA, SAMSUNG HD502IJ, 1AA0> SCSI3 0/direct fixed
naa.50000f000b170183
sd1: 476940MB, 512 bytes/sector, 976773168 sectors
sd2 at scsibus1 targ 2 lun 0: <ATA, WDC WD740ADFD-00, 21.0> SCSI3 0/direct
fixed t10.ATA_WDC_WD740ADFD-00NLR5_WD-WMANS2500775
sd2: 70911MB, 512 bytes/sector, 145226112 sectors
sd3 at scsibus1 targ 3 lun 0: <ATA, ST3500320AS, SD15> SCSI3 0/direct fixed
naa.5000c500099f8c18
sd3: 476940MB, 512 bytes/sector, 976773168 sectors
sd4 at scsibus1 targ 5 lun 0: <ATA, WDC WD1001FALS-0, 05.0> SCSI3 0/direct
fixed naa.50014ee0ac0a4aec
sd4: 953869MB, 512 bytes/sector, 1953525168 sectors
ohci0 at pci0 dev 18 function 0 "ATI SB700 USB" rev 0x00: apic 4 int 16,
version 1.0, legacy support
ohci1 at pci0 dev 18 function 1 "ATI SB700 USB" rev 0x00: apic 4 int 16,
version 1.0, legacy support
ehci0 at pci0 dev 18 function 2 "ATI SB700 USB2" rev 0x00: apic 4 int 17
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "ATI EHCI root hub" rev 2.00/1.00 addr 1
ohci2 at pci0 dev 19 function 0 "ATI SB700 USB" rev 0x00: apic 4 int 18,
version 1.0, legacy support
ohci3 at pci0 dev 19 function 1 "ATI SB700 USB" rev 0x00: apic 4 int 18,
version 1.0, legacy support
ehci1 at pci0 dev 19 function 2 "ATI SB700 USB2" rev 0x00: apic 4 int 19
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 "ATI EHCI root hub" rev 2.00/1.00 addr 1
piixpm0 at pci0 dev 20 function 0 "ATI SBx00 SMBus" rev 0x3a: SMI
iic0 at piixpm0
spdmem0 at iic0 addr 0x50: 2GB DDR2 SDRAM ECC PC2-5300CL5
spdmem1 at iic0 addr 0x51: 2GB DDR2 SDRAM ECC PC2-5300CL5
spdmem2 at iic0 addr 0x52: 2GB DDR2 SDRAM ECC PC2-5300CL5
spdmem3 at iic0 addr 0x53: 2GB DDR2 SDRAM ECC PC2-5300CL5
pciide0 at pci0 dev 20 function 1 "ATI SB700 IDE" rev 0x00: DMA, channel 0
configured to compatibility, channel 1 configured to compatibility
azalia1 at pci0 dev 20 function 2 "ATI SBx00 HD Audio" rev 0x00: apic 4 int 16
azalia1: codecs: Realtek ALC888
audio0 at azalia1
pcib0 at pci0 dev 20 function 3 "ATI SB700 ISA" rev 0x00
ppb3 at pci0 dev 20 function 4 "ATI SB600 PCI" rev 0x00
pci4 at ppb3 bus 4
ohci4 at pci0 dev 20 function 5 "ATI SB700 USB" rev 0x00: apic 4 int 18,
version 1.0, legacy support
pchb1 at pci0 dev 24 function 0 "AMD AMD64 10h HyperTransport" rev 0x00
pchb2 at pci0 dev 24 function 1 "AMD AMD64 10h Address Map" rev 0x00
pchb3 at pci0 dev 24 function 2 "AMD AMD64 10h DRAM Cfg" rev 0x00
km0 at pci0 dev 24 function 3 "AMD AMD64 10h Misc Cfg" rev 0x00
pchb4 at pci0 dev 24 function 4 "AMD AMD64 10h Link Cfg" rev 0x00
usb2 at ohci0: USB revision 1.0
uhub2 at usb2 "ATI OHCI root hub" rev 1.00/1.00 addr 1
usb3 at ohci1: USB revision 1.0
uhub3 at usb3 "ATI OHCI root hub" rev 1.00/1.00 addr 1
usb4 at ohci2: USB revision 1.0
uhub4 at usb4 "ATI OHCI root hub" rev 1.00/1.00 addr 1
usb5 at ohci3: USB revision 1.0
uhub5 at usb5 "ATI OHCI root hub" rev 1.00/1.00 addr 1
isa0 at pcib0
isadma0 at isa0
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot): using irq 1
wskbd0 at pckbd0: console keyboard
: using irq 12pcppi0 at isa0 port 0x61
spkr0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
it0 at isa0 port 0x2e/2: IT8712F rev 8, EC port 0x290
usb6 at ohci4: USB revision 1.0
uhub6 at usb6 "ATI OHCI root hub" rev 1.00/1.00 addr 1
uhub7 at uhub3 port 2 "Texas Instruments TUSB2046 hub" rev 1.10/1.25 addr 2
uhidev0 at uhub7 port 1 configuration 1 interface 0 "vendor 0x10d5 KVM A6" rev
1.10/0.04 addr 3
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd1 at ukbd0 mux 1
uhidev1 at uhub7 port 1 configuration 1 interface 1 "vendor 0x10d5 KVM A6" rev
1.10/0.04 addr 3
uhidev1: iclass 3/1, 3 report ids
ums0 at uhidev1 reportid 1: 5 buttons, Z dir
wsmouse0 at ums0 mux 0
uhid0 at uhidev1 reportid 2: input=1, output=0, feature=0
uhid1 at uhidev1 reportid 3: input=2, output=0, feature=0
uhub8 at uhub7 port 2 "Texas Instruments TUSB2046 hub" rev 1.10/1.25 addr 4
uhidev2 at uhub7 port 4 configuration 1 interface 0 "Yubico Yubico Yubikey II"
rev 2.00/2.42 addr 5
uhidev2: iclass 3/1
ukbd1 at uhidev2: 8 variable keys, 6 key codes
wskbd2 at ukbd1 mux 1
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (a29f854045613736.a) swap on sd0b dump on sd0b
radeondrm0: 1024x768
wsdisplay0 at radeondrm0 mux 1: console (std, vt100 emulation), using wskbd0
wskbd1: connecting to wsdisplay0
wskbd2: connecting to wsdisplay0
wsdisplay0: screen 1-5 added (std, vt100 emulation)
Fast boot: skipping disk checks.
setting tty flags
pf enabled
starting network
DHCPREQUEST on re0 to 255.255.255.255
DHCPACK from 10.1.2.1 (00:00:24:c9:29:14)
bound to 10.1.2.33 -- renewal in 302400 seconds.
starting early daemons: syslogd pflogd unbound ntpd.
starting RPC daemons: portmap mountd nfsd.
savecore: no core dump
checking quotas: done.
clearing /tmp
kern.securelevel: 0 -> 1
creating runtime link editor directory cache.
preserving editor files.
starting network daemons: sshd snmpd(failed) lpd smtpd sndiod.
Path to firmware: http://firmware.openbsd.org/firmware/snapshots/
Updating: radeondrm-firmware-20131002p0
starting package daemons: smbd nmbd adsuck(absent) apcupsd.
starting local daemons: apmd cron.
Mon May 18 08:02:46 EDT 2015
OpenBSD/amd64 (x4.thrush.com) (tty00)
login: root
Password:
Last login: Mon May 18 07:51:18 on tty00
OpenBSD 5.7-current (GENERIC.MP) #987: Sun May 17 20:42:55 MDT 2015
You have mail.
# mount | grep nas3
nas3:/work on /dpb/nas3/work type nfs (nodev, nosuid, v3, udp, intr, timeo=100,
retrans=101)
nas3:/backup on /nas3/backup type nfs (nodev, nosuid, v3, udp, intr, timeo=100,
retrans=101)
nas3:/public on /dpb/nas3/public type nfs (nodev, nosuid, read-only, v3, udp,
intr, timeo=100, retrans=101)
# ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 32768
priority: 0
groups: lo
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
inet6 ::1 prefixlen 128
inet 127.0.0.1 netmask 0xff000000
msk0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:50:43:00:7c:9a
priority: 0
media: Ethernet autoselect (none)
status: no carrier
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:22:15:2c:7d:0b
priority: 0
groups: egress int
media: Ethernet autoselect (1000baseT full-duplex,rxpause,txpause)
status: active
inet 10.1.2.33 netmask 0xffffff00 broadcast 10.1.2.255
enc0: flags=0<>
priority: 0
groups: enc
status: active
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33144
priority: 0
groups: pflog
# S=/dpb/nas3/public/OpenBSD/snapshots/amd64
# ls -ltT ${S}/install57.iso
-rw-r--r-- 1 rd rd 230189056 May 17 22:49:43 2015
/dpb/nas3/public/OpenBSD/snapshots/amd64/install57.iso
# dd if=${S}/install57.iso of=/dev/null bs=32k
panic: pool_do_get: mcl8k free list modified: page 0xffffff0001a4e000; item
addr 0xffffff0001a4e000; offset 0x0=0x3c2862a624713d01 != 0xa8056a9a45b16778
Stopped at Debugger+0x9: leave
RUN AT LEAST 'trace' AND 'ps' AND INCLUDE OUTPUT WHEN REPORTING THIS PANIC!
IF RUNNING SMP, USE 'mach ddbcpu <#>' AND 'trace' ON OTHER PROCESSORS, TOO.
DO NOT EVEN BOTHER REPORTING THIS WITHOUT INCLUDING THAT INFORMATION!
ddb{0}> set $lines 0
ddb{0}> trace
Debugger() at Debugger+0x9
panic() at panic+0xfe
pool_runqueue() at pool_runqueue
pool_get() at pool_get+0xb5
m_clget() at m_clget+0x51
re_newbuf() at re_newbuf+0x34
re_rx_list_fill() at re_rx_list_fill+0x35
re_rxeof() at re_rxeof+0x333
re_intr() at re_intr+0x19b
intr_handler() at intr_handler+0x67
Xintr_ioapic_edge18() at Xintr_ioapic_edge18+0xdd
--- interrupt ---
cpu_idle_mwait_cycle() at cpu_idle_mwait_cycle+0x62
cpu_idle_cycle() at cpu_idle_cycle+0x10
end trace frame: 0x0, count: -13
ddb{0}> ps
PID PPID PGRP UID S FLAGS WAIT COMMAND
26916 24107 26916 0 3 0x83 getblk dd
790 4247 22307 1000 3 0x82 nanosleep sleep
16419 7909 7909 1000 3 0x82 nanosleep perl
7909 18528 7909 1000 3 0x8a pause sh
18528 23947 23947 0 3 0x80 piperd cron
4247 22307 22307 1000 3 0x8a pause sh
10000 5347 5347 0 3 0x82 nanosleep perl
24107 1 24107 0 3 0x8b pause ksh
25720 1 25720 0 3 0x83 ttyin getty
17251 1 17251 0 3 0x83 ttyin getty
21816 1 21816 0 3 0x83 ttyin getty
25792 1 25792 0 3 0x83 ttyin getty
27641 1 27641 0 3 0x83 ttyin getty
22307 23331 22307 1000 3 0x8a pause sh
5347 6158 5347 0 3 0x8a pause sh
23331 23947 23947 0 3 0x80 piperd cron
6158 23947 23947 0 3 0x80 piperd cron
23947 1 23947 0 3 0x80 poll cron
8704 1 8704 0 3 0x80 kqread apmd
20169 1 20169 0 7 0 apcupsd
7491 1 20169 0 3 0x4000088 sigwait apcupsd
820 1 20169 0 3 0x4000080 select apcupsd
30745 1 30745 0 3 0x80 poll nmbd
21377 30807 30807 0 3 0x90 poll smbd
30807 1 30807 0 3 0x90 poll smbd
662 1 662 99 3 0x90 poll sndiod
32353 9708 9708 95 3 0x90 kqread smtpd
31775 9708 9708 95 3 0x90 kqread smtpd
6539 9708 9708 95 3 0x90 kqread smtpd
11711 9708 9708 95 3 0x90 kqread smtpd
23602 9708 9708 95 3 0x90 kqread smtpd
12815 9708 9708 103 3 0x90 kqread smtpd
9708 1 9708 0 3 0x80 kqread smtpd
18347 1 18347 0 3 0xb0 select lpd
1534 1 1534 0 3 0x80 select sshd
31949 0 0 0 3 0x14280 nfsidl nfsio
1151 0 0 0 3 0x14280 nfsidl nfsio
20004 0 0 0 3 0x14280 nfsrcvlk nfsio
6326 0 0 0 3 0x14200 netio nfsio
11227 27093 27093 0 3 0x80 nfsd nfsd
2401 27093 27093 0 3 0x80 nfsd nfsd
28351 27093 27093 0 3 0x80 nfsd nfsd
11723 27093 27093 0 3 0x80 nfsd nfsd
27093 1 27093 0 3 0x80 netcon nfsd
6235 1 6235 0 3 0x80 poll mountd
2311 1 2311 28 3 0x90 poll portmap
4706 1 4706 0 3 0x80 poll ntpd
23037 5497 23037 83 3 0x90 poll ntpd
5497 1 5497 83 3 0x90 poll ntpd
27817 1 27817 53 3 0x90 kqread unbound
20949 16915 16915 74 3 0x90 bpf pflogd
16915 1 16915 0 3 0x80 netio pflogd
14497 4216 4216 73 3 0x90 kqread syslogd
4216 1 4216 0 3 0x80 netio syslogd
31135 1 31135 77 3 0x90 poll dhclient
21643 1 21643 0 3 0x80 poll dhclient
8966 1 8966 0 3 0x80 mfsidl mount_mfs
13251 0 0 0 3 0x14200 bored ttm_swap
23958 0 0 0 3 0x14200 pgzero zerothread
8226 0 0 0 3 0x14200 aiodoned aiodoned
11051 0 0 0 3 0x14200 syncer update
19572 0 0 0 3 0x14200 cleaner cleaner
9291 0 0 0 3 0x14200 reaper reaper
31035 0 0 0 3 0x14200 pgdaemon pagedaemon
9434 0 0 0 3 0x14200 bored crypto
13551 0 0 0 3 0x14200 pftm pfpurge
2480 0 0 0 3 0x14200 bored sensors
23352 0 0 0 3 0x14200 usbtsk usbtask
9784 0 0 0 3 0x14200 usbatsk usbatsk
12107 0 0 0 3 0x40014200 acpi0 acpi0
27369 0 0 0 7 0x40014200 idle3
10730 0 0 0 7 0x40014200 idle2
31139 0 0 0 3 0x40014200 idle1
10377 0 0 0 3 0x14200 bored softnet
10567 0 0 0 3 0x14200 bored systqmp
13628 0 0 0 3 0x14200 bored systq
*30240 0 0 0 7 0x40014200 idle0
1 0 1 0 3 0x82 wait init
0 -1 0 0 3 0x10200 scheduler swapper
ddb{0}> show registers
ds 0x39f8 __ALIGN_SIZE+0x29f8
es 0xaf5
fs 0xd
gs 0x39f8 __ALIGN_SIZE+0x29f8
rdi 0x1
rsi 0x282
rbp 0xffff8000220339e8
rbx 0xffffffff816e0468 systqmp+0x16a8
rdx 0
rcx 0xffffffff818819c0 cpu_info_primary
rax 0x1
r8 0xffff800022033908
r9 0
r10 0xffff8000220338b0
r11 0x8
r12 0x100
r13 0xffff8000220339f8
r14 0x2
r15 0x1a
rip 0xffffffff813453a9 Debugger+0x9
cs 0x8
rflags 0x286
rsp 0xffff8000220339d8
ss 0x10
Debugger+0x9: leave
ddb{0}> show malloc
Type InUse MemUse HighUse Limit Requests Type Lim Kern Lim
devbuf 1782 1411K 1412K 78643K 2829 0 0
pcb 78 16K 17K 78643K 136 0 0
routetbl 93 7K 7K 78643K 212 0 0
ifaddr 53 13K 13K 78643K 54 0 0
sysctl 3 2K 2K 78643K 7 0 0
ioctlops 0 0K 4K 78643K 2581 0 0
mount 13 13K 13K 78643K 13 0 0
NFS mount 4 2K 2K 78643K 4 0 0
vnodes 1408 88K 88K 78643K 1422 0 0
UFS quota 1 32K 32K 78643K 1 0 0
UFS mount 34 79K 79K 78643K 34 0 0
shm 2 1K 1K 78643K 2 0 0
VM map 2 0K 0K 78643K 2 0 0
sem 2 0K 0K 78643K 2 0 0
dirhash 57 11K 11K 78643K 57 0 0
ACPI 4826 561K 633K 78643K 27133 0 0
file desc 11 5K 6K 78643K 21 0 0
proc 14 9K 9K 78643K 14 0 0
VFS cluster 0 0K 0K 78643K 2 0 0
MFS node 1 0K 0K 78643K 1 0 0
Export Host 1 0K 0K 78643K 3 0 0
NFS srvsock 1 0K 0K 78643K 1 0 0
NFS daemon 5 16K 16K 78643K 5 0 0
in_multi 14 0K 0K 78643K 14 0 0
ether_multi 1 0K 0K 78643K 1 0 0
ISOFS mount 1 32K 32K 78643K 1 0 0
MSDOSFS mount 1 16K 16K 78643K 1 0 0
ttys 1158 746K 746K 78643K 1158 0 0
exec 0 0K 6K 78643K 880 0 0
pfkey data 1 0K 0K 78643K 2 0 0
pagedep 1 8K 8K 78643K 1 0 0
inodedep 1 32K 32K 78643K 3 0 0
newblk 1 0K 0K 78643K 1 0 0
indirdep 0 0K 16K 78643K 1 0 0
VM swap 7 586K 586K 78643K 7 0 0
UVM amap 6699 778K 810K 78643K 44742 0 0
UVM aobj 12 3K 3K 78643K 12 0 0
USB 111 26K 26K 78643K 113 0 0
USB device 37 2K 2K 78643K 37 0 0
memdesc 1 4K 4K 78643K 1 0 0
crypto data 1 1K 1K 78643K 1 0 0
ip6_options 1 0K 0K 78643K 1 0 0
NDP 6 0K 0K 78643K 6 0 0
temp 77 43K 169K 78643K 4585 0 0
DRM 156 2378K 2382K 78643K 173 0 0
ddb{0}> show proc
PROC (idle0) pid=30240 stat=onproc
flags process=14000<NOZOMBIE,SYSTEM> proc=40000200<SYSTEM,CPUPEG>
pri=0, usrpri=86, nice=20
forw=0x762d790aea3d6084, list=0xffffff021f63b000,0xffffff021f63b480
process=0xffffff021f5de278 user=0xffff80002202f000,
vmspace=0xffffffff818f9280
estcpu=36, cpticks=19064, pctcpu=0.0
user=0, sys=0, intr=2
ddb{0}> show uvmexp
Current UVM status:
pagesize=4096 (0x1000), pagemask=0xfff, pageshift=12
1967082 VM pages: 44632 active, 1770 inactive, 3 wired, 1892855 free (236961
zero)
min 10% (25) anon, 10% (25) vnode, 5% (12) vtext
pages 0 anon, 0 vnode, 0 vtext
freemin=65569, free-target=87425, inactive-target=0, wired-max=655694
faults=178888, traps=175500, intrs=11771, ctxswitch=51965 fpuswitch=86
softint=25664, syscalls=383526, kmapent=18
fault counts:
noram=0, noanon=0, pgwait=0, pgrele=0
ok relocks(total)=4874(4874), anget(retries)=82048(0), amapcopy=64795
neighbor anon/obj pg=9104/67738, gets(lock/unlock)=26524/4874
cases: anon=70576, anoncow=11472, obj=22599, prcopy=3925, przero=70251
daemon and swap counts:
woke=0, revs=0, scans=0, obscans=0, anscans=0
busy=0, freed=0, reactivate=0, deactivate=0
pageouts=0, pending=0, nswget=0
nswapdev=1, nanon=0, nanonneeded=0 nfreeanon=0
swpages=2098749, swpginuse=0, swpgonly=0 paging=0
kernel pointers:
objs(kern)=0xffffffff818c27e0
ddb{0}> callout
ticks now: 20933
ticks wheel arg func
498 4d/1024d 0xffffff01da289470 endtsleep
18 4d/1024d 0xffffff01d9ce0250 tcp_delack
-1 4d/1024d 0xffff80000069aa00 nfs_timer
-1 4d/1024d 0xffffffff81906dc0 net_tick
0 4d/1024d 0x0 setperf_auto
67 0d/8 d 0xffff8000000d1048 if_slowtimo
67 0d/8 d 0xffff8000000fc090 if_slowtimo
67 0d/8 d 0xffffffff818f0b60 pckbc_poll
67 0d/8 d 0xffffffff818a2600 rt_timer_timer
67 0d/8 d 0xffffffff818a20e0 schedcpu
67 0d/8 d 0xffffff021f5cbb20 endtsleep
67 0d/8 d 0x0 nd6_timer
8 0d/205 d 0xffffffff818a2420 pffasttimo
18 0d/215 d 0xffffffff818a23e0 pfslowtimo
33 0d/230 d 0xffffff01db36fd50 endtsleep
34 0d/231 d 0xffffff01db36f8e0 endtsleep
57 0d/254 d 0xffffff01d9ce0020 tcp_timer_2msl
57 0d/254 d 0xffff8000000fc000 re_tick
67 1d/338 d 0xffff800000017400 acpi_poll
67 1d/338 d 0xffff8000000c7d00 sensor_task_tick
67 1d/338 d 0xffff8000000cb400 sensor_task_tick
359 1d/339 d 0xffffff01da465b20 endtsleep
500 1d/339 d 0xffffff01da4656b0 endtsleep
560 1d/339 d 0xffffff01da234240 endtsleep
597 1d/340 d 0xffffff01d8a2a488 tcp_timer_2msl
656 1d/340 d 0xffff8000000cf000 drm_helper_hpd_irq_event
1137 1d/342 d 0xffffff01d8a2a6b8 tcp_timer_2msl
1677 1d/344 d 0xffffff01d8a2a8e8 tcp_timer_2msl
2217 1d/346 d 0xffffff01d8a2ab18 tcp_timer_2msl
2757 1d/348 d 0xffffff01d8a2ad48 tcp_timer_2msl
3297 1d/350 d 0xffffff01d8a2a258 tcp_timer_2msl
3837 1d/352 d 0xffffff01d8a2a028 tcp_timer_2msl
4377 1d/354 d 0xffffff01d9ce08e0 tcp_timer_2msl
4435 1d/355 d 0xffffff01db5be518 realitexpire
4917 1d/356 d 0xffffff01d9ce0b10 tcp_timer_2msl
4958 1d/357 d 0xffffff01da289d50 endtsleep
5457 1d/359 d 0xffffff01d9ce0d40 tcp_timer_2msl
10458 1d/378 d 0xffffffff818a2640 arptimer
39067 1d/490 d 0x0 arc4_reinit
40981 1d/497 d 0xffffff01d9ce3238 endtsleep
339067 2d/517 d 0x0 nd6_slowtimo
341051 2d/517 d 0xffffff01d90bb008 endtsleep
348511 2d/517 d 0xffffff01d90bb478 endtsleep
719998 2d/523 d 0xffffff01d9ce0250 tcp_timer_keep
1421350 2d/534 d 0xffffff01d9f0bb20 endtsleep
8620953 2d/643 d 0xffffff01d9f0b008 endtsleep
8621096 2d/643 d 0xffffff01e2cb1d58 endtsleep
ddb{0}> ps /w
PID COMMAND EMUL WAIT-CHANNEL WAIT-MSG
26916 dd native 0xffffff01d8b6e200 getblk
790 sleep native 0xffffffff818a20a8 nanosleep
16419 perl native 0xffffffff818a20a8 nanosleep
7909 sh native 0xffffff021e992540 pause
18528 cron native 0xffffff01d8369080 piperd
4247 sh native 0xffffff021e9921e0 pause
10000 perl native 0xffffffff818a20a8 nanosleep
24107 ksh native 0xffffff021f63a1b0 pause
25720 getty native 0xffff80000050d410 ttyin
17251 getty native 0xffff8000004e0c10 ttyin
21816 getty native 0xffff8000004e0010 ttyin
25792 getty native 0xffff8000004d2410 ttyin
27641 getty native 0xffff8000004c4810 ttyin
22307 sh native 0xffffff01d9c041d8 pause
5347 sh native 0xffffff01dadb0010 pause
23331 cron native 0xffffff01e2bf8f00 piperd
6158 cron native 0xffffff01e2bf8960 piperd
23947 cron native 0xffffffff81904448 poll
8704 apmd native 0xffffff01dae73b40 kqread
20169 apcupsd native 0x0
7491 apcupsd native 0xffffffff818a2078 sigwait
820 apcupsd native 0xffffffff81904448 select
30745 nmbd native 0xffffffff81904448 poll
21377 smbd native 0xffffffff81904448 poll
30807 smbd native 0xffffffff81904448 poll
662 sndiod native 0xffffffff81904448 poll
32353 smtpd native 0xffffff01dae73a00 kqread
31775 smtpd native 0xffffff01dae73500 kqread
6539 smtpd native 0xffffff01dae73640 kqread
11711 smtpd native 0xffffff01dae738c0 kqread
23602 smtpd native 0xffffff01dae73780 kqread
12815 smtpd native 0xffffff01dae733c0 kqread
9708 smtpd native 0xffffff01dae73280 kqread
18347 lpd native 0xffffffff81904448 select
1534 sshd native 0xffffffff81904448 select
31949 nfsio native 0xffffffff81921500 nfsidl
1151 nfsio native 0xffffffff81921500 nfsidl
20004 nfsio native 0xffff80000069aa40 nfsrcvlk
6326 nfsio native 0xffffff01da4ba6c8 netio
11227 nfsd native 0xffff800000695fc0 nfsd
2401 nfsd native 0xffff800000692fc0 nfsd
28351 nfsd native 0xffff800000695f00 nfsd
11723 nfsd native 0xffff800000679600 nfsd
27093 nfsd native 0xffffff01da50acbe netcon
6235 mountd native 0xffffffff81904448 poll
2311 portmap native 0xffffffff81904448 poll
4706 ntpd native 0xffffffff81904448 poll
23037 ntpd native 0xffffffff81904448 poll
5497 ntpd native 0xffffffff81904448 poll
27817 unbound native 0xffffff01dae73140 kqread
20949 pflogd native 0xffff800000675c00 bpf
16915 pflogd native 0xffffff01daec6398 netio
14497 syslogd native 0xffffff01dae73000 kqread
4216 syslogd native 0xffffff01daec6088 netio
31135 dhclient native 0xffffffff81904448 poll
21643 dhclient native 0xffffffff81904448 poll
8966 mount_mfs native 0xffffff021e7e0b08 mfsidl
13251 ttm_swap native 0xffff8000001b58c0 bored
23958 zerothread native 0xffffffff81924944 pgzero
8226 aiodoned native 0xffffffff81924b00 aiodoned
11051 update native 0xffffffff819028e8 syncer
19572 cleaner native 0xffffffff81905fe0 cleaner
9291 reaper native 0xffffffff8186eaf0 reaper
31035 pagedaemon native 0xffffffff81924af0 pgdaemon
9434 crypto native 0xffff8000000aad80 bored
13551 pfpurge native 0xffffffff81130250 pftm
2480 sensors native 0xffff8000000ca140 bored
23352 usbtask native 0xffffffff818eda84 usbtsk
9784 usbatsk native 0xffffffff818eda88 usbatsk
12107 acpi0 native 0xffff800000017400 acpi0
27369 idle3 native 0x0
10730 idle2 native 0x0
31139 idle1 native 0x0
10377 softnet native 0xffff800000018040 bored
10567 systqmp native 0xffffffff8186f680 bored
13628 systq native 0xffffffff8186f640 bored
*30240 idle0 native 0x0
1 init native 0xffffff021f5de000 wait
0 swapper native 0xffffffff818f9600 scheduler
ddb{0}> ps /a
PID COMMAND STRUCT PROC * UAREA * VMSPACE/VM_MAP
26916 dd 0xffffff01da465478 0xffff800023043000 0xffffff01d8ba6780
790 sleep 0xffffff01da465b20 0xffff800022eb5000 0xffffff01d8ba64b0
16419 perl 0xffffff01d90bb478 0xffff800022fda000 0xffffff01d8ba65a0
7909 sh 0xffffff01d90bb240 0xffff800022fd5000 0xffffff01d8ba61e0
18528 cron 0xffffff01d90bbb20 0xffff800022f47000 0xffffff01d8ba6690
4247 sh 0xffffff01d90bb6b0 0xffff800023072000 0xffffff01d8ba63c0
10000 perl 0xffffff01d90bb008 0xffff800023063000 0xffffff01d8ba60f0
24107 ksh 0xffffff01da289000 0xffff80002305e000 0xffffff01d8ba6000
25720 getty 0xffffff01da465008 0xffff800023059000 0xffffff021f5dd690
17251 getty 0xffffff01e2cb1008 0xffff800023054000 0xffffff021f5dd0f0
21816 getty 0xffffff01d99cfd50 0xffff800022fe7000 0xffffff01db5cc1e8
25792 getty 0xffffff01d99cfb18 0xffff800022fe2000 0xffffff01db5cc0f8
27641 getty 0xffffff01d99cf8e0 0xffff800022f35000 0xffffff01db5cc968
22307 sh 0xffffff01d99cf238 0xffff80002303e000 0xffffff01db5cc878
5347 sh 0xffffff01da289238 0xffff800022fff000 0xffffff01db5ccf08
23331 cron 0xffffff01da4658e8 0xffff800022fcb000 0xffffff01d9158a60
6158 cron 0xffffff01da465240 0xffff800022fbc000 0xffffff021f5dda50
23947 cron 0xffffff01da4656b0 0xffff800022fc6000 0xffffff01d9158c40
8704 apmd 0xffffff01d9ce3238 0xffff800022fc1000 0xffffff021f5dd2d0
20169 apcupsd 0xffffff01d9f0b8e8 0xffff800022fb7000 0xffffff01d9158970
7491 apcupsd 0xffffff01d9f0b240 0xffff800022fb2000 0xffffff01d9158970
820 apcupsd 0xffffff01d90bbd58 0xffff800022fd0000 0xffffff01d9158970
30745 nmbd 0xffffff01da234240 0xffff800022fad000 0xffffff01d9158f10
21377 smbd 0xffffff01da289d50 0xffff800022f88000 0xffffff01d9158e20
30807 smbd 0xffffff01d9f0b008 0xffff800022f7b000 0xffffff01d9158d30
662 sndiod 0xffffff01d9f0bd58 0xffff800022f3a000 0xffffff01d9158b50
32353 smtpd 0xffffff01d9f0b478 0xffff800022f6c000 0xffffff01d9158790
31775 smtpd 0xffffff01d9f0bb20 0xffff800022f67000 0xffffff01d91586a0
6539 smtpd 0xffffff01d9f0b6b0 0xffff800022ed3000 0xffffff01d9158100
11711 smtpd 0xffffff01d9ce3d50 0xffff800022f53000 0xffffff01d91585b0
23602 smtpd 0xffffff01d9ce38e0 0xffff800022f4e000 0xffffff01d91582e0
12815 smtpd 0xffffff01d9ce3470 0xffff800022f5d000 0xffffff01d91581f0
9708 smtpd 0xffffff01d9ce3000 0xffff800022ebf000 0xffffff01d91584c0
18347 lpd 0xffffff01d9ce3b18 0xffff800022f76000 0xffffff01d91583d0
1534 sshd 0xffffff01d9ce36a8 0xffff800022f71000 0xffffff01d9158010
31949 nfsio 0xffffff01da289b18 0xffff800022f30000 0xffffffff818f9280
1151 nfsio 0xffffff01da2898e0 0xffff800022f2b000 0xffffffff818f9280
20004 nfsio 0xffffff01da2896a8 0xffff800022f26000 0xffffffff818f9280
6326 nfsio 0xffffff01da289470 0xffff800022f21000 0xffffffff818f9280
11227 nfsd 0xffffff01db36f6a8 0xffff800022f1c000 0xffffff01db5cc698
2401 nfsd 0xffffff01db36f470 0xffff800022f17000 0xffffff01db5cc5a8
28351 nfsd 0xffffff01da2346b0 0xffff800022f12000 0xffffff01db5cc4b8
11723 nfsd 0xffffff01e2cb18e8 0xffff800022eea000 0xffffff01db5cc2d8
27093 nfsd 0xffffff01da234d58 0xffff800022f0d000 0xffffff01db5cc3c8
6235 mountd 0xffffff01da234b20 0xffff800022f08000 0xffffff01db5cc008
2311 portmap 0xffffff01da2348e8 0xffff800022f03000 0xffffff021f5ddf00
4706 ntpd 0xffffff01da234008 0xffff800022efe000 0xffffff021f5dde10
23037 ntpd 0xffffff01e2cb1240 0xffff800022ef9000 0xffffff021f5ddd20
5497 ntpd 0xffffff01db36fd50 0xffff800022ef4000 0xffffff021f5ddc30
27817 unbound 0xffffff01e2cb1478 0xffff800022eef000 0xffffff021f5ddb40
20949 pflogd 0xffffff01db36f8e0 0xffff800022eba000 0xffffff021f5dd1e0
16915 pflogd 0xffffff01db36fb18 0xffff800022ed8000 0xffffff021f5dd960
14497 syslogd 0xffffff01db36f238 0xffff800022ee5000 0xffffff021f5dd870
4216 syslogd 0xffffff01db36f000 0xffff800022ee0000 0xffffff021f5dd780
31135 dhclient 0xffffff01e2cb1d58 0xffff800022ec9000 0xffffff021f5dd4b0
21643 dhclient 0xffffff01e2cb1b20 0xffff800022ece000 0xffffff021f5dd5a0
8966 mount_mfs 0xffffff01e2cb16b0 0xffff800022ec4000 0xffffff021f5dd3c0
13251 ttm_swap 0xffffff021f511d50 0xffff80002288e000 0xffffffff818f9280
23958 zerothread 0xffffff021f511b18 0xffff800022089000 0xffffffff818f9280
8226 aiodoned 0xffffff021f5118e0 0xffff800022084000 0xffffffff818f9280
11051 update 0xffffff021f5116a8 0xffff80002207f000 0xffffffff818f9280
19572 cleaner 0xffffff021f511470 0xffff80002207a000 0xffffffff818f9280
9291 reaper 0xffffff021f511238 0xffff800022075000 0xffffffff818f9280
31035 pagedaemon 0xffffff021f511000 0xffff800022070000 0xffffffff818f9280
9434 crypto 0xffffff021f5cbd58 0xffff80002206b000 0xffffffff818f9280
13551 pfpurge 0xffffff021f5cbb20 0xffff800022066000 0xffffffff818f9280
2480 sensors 0xffffff021f5cb8e8 0xffff800022061000 0xffffffff818f9280
23352 usbtask 0xffffff021f5cb6b0 0xffff80002205c000 0xffffffff818f9280
9784 usbatsk 0xffffff021f5cb478 0xffff800022057000 0xffffffff818f9280
12107 acpi0 0xffffff021f5cb240 0xffff800022052000 0xffffffff818f9280
27369 idle3 0xffffff021f5cb008 0xffff80002204d000 0xffffffff818f9280
10730 idle2 0xffffff021f63bd50 0xffff800022048000 0xffffffff818f9280
31139 idle1 0xffffff021f63bb18 0xffff800022043000 0xffffffff818f9280
10377 softnet 0xffffff021f63b8e0 0xffff80002203e000 0xffffffff818f9280
10567 systqmp 0xffffff021f63b6a8 0xffff800022039000 0xffffffff818f9280
13628 systq 0xffffff021f63b470 0xffff800022034000 0xffffffff818f9280
*30240 idle0 0xffffff021f63b238 0xffff80002202f000 0xffffffff818f9280
1 init 0xffffff021f63b000 0xffff80002202a000 0xffffff021f5dd000
0 swapper 0xffffffff818f9600 0xffffffff81a16000 0xffffffff818f9280
ddb{0}> show bcstats
Current Buffer Cache status:
numbufs 4235 busymapped 2, delwri 2
kvaslots 6553 avail kva slots 6551
bufpages 14170, dirtypages 8
pendingreads 2, pendingwrites 0
ddb{0}> show all pools
Name Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
arp 56 9 0 0 1 0 1 1 0 8 0
inpcbpl 288 196 0 129 6 0 6 6 0 8 0
plimitpl 152 45 0 12 2 0 2 2 0 8 0
sackhl 24 6 0 6 1 0 1 1 0 8 1
tcpqe 32 39 0 39 1 0 1 1 0 8 1
tcpcb 560 76 0 43 6 0 6 6 0 8 0
rtmask 32 9 0 0 1 0 1 1 0 8 0
rtentry 192 33 0 0 2 0 2 2 0 8 0
pfosfp 40 840 0 420 5 0 5 5 0 8 0
pfosfpen 112 1420 0 710 21 0 21 21 0 8 0
pfstitem 24 108 0 74 1 0 1 1 0 8 0
pfstkey 104 108 0 74 2 0 2 2 0 8 0
pfstate 312 108 0 74 6 0 6 6 0 8 3
pfrule 1336 19 0 16 7 0 7 7 0 8 5
nfsnodepl 472 8 0 0 1 0 1 1 0 8 0
dirhash 1024 119 0 0 30 0 30 30 0 8 0
dirrem 64 4 0 4 1 0 1 1 0 8 1
diradd 56 4 0 4 1 0 1 1 0 8 1
freefile 48 3 0 3 1 0 1 1 0 8 1
freeblks 192 12 0 12 1 0 1 1 0 8 1
allocindir 104 1 0 1 1 0 1 1 0 8 1
indirdep 56 1 0 1 1 0 1 1 0 8 1
allocdir 128 10 0 10 1 0 1 1 0 8 1
bmsafemap 64 8 0 8 1 0 1 1 0 8 1
newblk 64 11 0 11 1 0 1 1 0 8 1
inodedep 152 13 0 13 1 0 1 1 0 8 1
pagedep 128 5 0 5 1 0 1 1 0 8 1
dino1pl 128 2454 0 106 76 0 76 76 0 8 0
ffsino 240 2454 0 106 139 0 139 139 0 8 0
nchpl 144 4123 0 164 147 0 147 147 0 8 0
uvmvnodes 72 2487 0 0 46 0 46 46 0 8 0
vnodes 200 2487 0 0 131 0 131 131 0 8 0
namei 1024 14216 0 14216 4 0 4 4 0 8 4
nfsreqpl 96 1393 0 1391 1 0 1 1 0 8 0
ohcixfer 224 369 0 358 1 0 1 1 0 8 0
ehcixfer 264 106 0 104 1 0 1 1 0 8 0
drmobjpl 696 10 0 0 2 0 2 2 0 8 0
scxspl 192 5007 0 5007 2 0 2 2 0 8 2
sigapl 432 464 0 412 7 0 7 7 0 8 0
knotepl 112 449 0 377 3 0 3 3 0 8 0
kqueuepl 320 10 0 0 1 0 1 1 0 8 0
pipepl 120 484 0 461 2 0 2 2 0 8 0
fdescpl 440 465 0 412 7 0 7 7 0 8 0
filepl 120 4966 0 4730 8 0 8 8 0 8 0
lockfpl 88 708 0 690 1 0 1 1 0 8 0
sessionpl 64 36 0 5 1 0 1 1 0 8 0
pgrppl 40 46 0 14 1 0 1 1 0 8 0
ucredpl 96 325 0 277 2 0 2 2 0 8 0
zombiepl 144 412 0 412 1 0 1 1 0 8 1
processpl 632 488 0 412 14 0 14 14 0 8 0
procpl 568 490 0 412 12 0 12 12 0 8 0
sockpl 392 319 0 187 14 0 14 14 0 8 0
mcl8k 8192 2077 0 2072 8 0 8 8 8 6144 3
mcl8k: pool(0xffffffff81905290:mcl8k): free list modified: page
0xffffff0001a42000; item ordinal 0; addr 0xffffff0001a42000 (p
0xffffff021f7d6000); offset 0x0=0xecd2cbe8ce5e5f46
pool(mcl8k): free list modified: page 0xffffff0001a42000; item ordinal 0; addr
0xffffff0001a42000 (p 0xffffff021f7d6000); offset 0x0=0xb24fc372
mcl8k: pool(0xffffffff81905290:mcl8k): page inconsistency: page
0xffffff0001a42000; item ordinal 1; addr 0x4db7e8a7180b103
mcl8k: pool(0xffffffff81905290:mcl8k): free list modified: page
0xffffff0001a4e000; item ordinal 0; addr 0xffffff0001a4e000 (p
0xffffff021f7d6000); offset 0x0=0x3c2862a624713d01
pool(mcl8k): free list modified: page 0xffffff0001a4e000; item ordinal 0; addr
0xffffff0001a4e000 (p 0xffffff021f7d6000); offset 0x0=0xd2fea097
mcl8k: pool(0xffffffff81905290:mcl8k): page inconsistency: page
0xffffff0001a4e000; item ordinal 1; addr 0xccddaefdf31f1ba6
mcl2k 2048 11 0 11 4 0 4 4 4 3072 4
mtagpl 72 1 0 1 1 0 1 1 0 8 1
mbufpl 256 12937 0 12898 5 0 5 5 1 384 1
bufpl 256 5326 0 874 279 0 279 279 0 8 0
anonpl 16 85648 0 44094 175 0 175 175 0 494 2
amappl 72 44145 0 37500 151 0 151 151 0 75 15
dma4096 4096 5 0 5 1 0 1 1 0 8 1
dma512 512 31 0 26 1 0 1 1 0 8 0
dma256 256 58 0 58 1 0 1 1 0 8 1
dma128 128 1 0 1 1 0 1 1 0 8 1
dma64 64 10 0 10 1 0 1 1 0 8 1
dma32 32 53 0 53 1 0 1 1 0 8 1
dma16 16 9 0 9 1 0 1 1 0 8 1
aobjpl 64 11 0 0 1 0 1 1 0 8 0
uaddrrnd 24 465 0 412 1 0 1 1 0 8 0
uaddrbest 32 2 0 0 1 0 1 1 0 8 0
uaddr 24 465 0 412 1 0 1 1 0 8 0
vmmpekpl 168 3235 0 3218 1 0 1 1 0 8 0
vmmpepl 168 162332 0 153089 550 0 550 550 0 357 133
vmsppl 240 464 0 412 4 0 4 4 0 8 0
pdppl 4096 464 0 412 57 2 55 57 0 8 3
pvpl 32 265682 0 245488 297 5 292 297 0 265 115
pmappl 168 464 0 412 3 0 3 3 0 8 0
extentpl 40 114 0 54 1 0 1 1 0 8 0
phpool 112 576 0 2 17 0 17 17 0 8 0
ddb{0}> show all pools /a
Name Address Allocator
arp 0xffffffff81907e00 0xffffffff8186f790
inpcbpl 0xffffffff8190c000 0xffffffff8186f790
plimitpl 0xffffffff818fd660 0xffffffff8186f790
swp vnd 0xffffffff81925540 0xffffffff8186f790
swp vnx 0xffffffff819253e0 0xffffffff8186f790
syncache 0xffffffff8191e660 0xffffffff8186f790
sackhl 0xffffffff8191eb60 0xffffffff8186f790
tcpqe 0xffffffff819209a0 0xffffffff8186f790
tcpcb 0xffffffff8191ecc0 0xffffffff8186f790
ipq 0xffffffff8190c600 0xffffffff8186f790
ipqe 0xffffffff8190c8c0 0xffffffff8186f790
rttmr 0xffffffff81907960 0xffffffff8186f790
rtmask 0xffffffff819077a0 0xffffffff8186f790
rtentry 0xffffffff81907ac0 0xffffffff8186f790
cryptodesc 0xffffffff819213a0 0xffffffff8186f790
cryptop 0xffffffff81921240 0xffffffff8186f790
mppekey 0xffffffff81906fc0 0xffffffff8186f790
ppxss 0xffffffff81907340 0xffffffff8186f790
strpolpl 0xffffffff818f8220 0xffffffff8186f790
strprocpl 0xffffffff818f8380 0xffffffff8186f790
pfstscr 0xffffffff818f34e0 0xffffffff8186f790
pffrag 0xffffffff818f3640 0xffffffff8186f790
pffrent 0xffffffff818f3360 0xffffffff8186f790
pfosfp 0xffffffff818f5420 0xffffffff8186f790
pfosfpen 0xffffffff818f52c0 0xffffffff8186f790
pfrkcounters 0xffffffff818f4ce0 0xffffffff8186f790
pfrke_cost 0xffffffff818f4a10 0xffffffff8186f790
pfrke_route 0xffffffff818f48c8 0xffffffff8186f790
pfrke_plain 0xffffffff818f4780 0xffffffff8186f790
pfrktable 0xffffffff818f4b60 0xffffffff8186f790
hfscintsc 0xffffffff818f5860 0xffffffff8186f790
hfscclass 0xffffffff818f5700 0xffffffff8186f790
pfqueue 0xffffffff818f2cc0 0xffffffff8186f790
pfruleitem 0xffffffff818f3200 0xffffffff8186f790
pfstitem 0xffffffff818f2f40 0xffffffff8186f790
pfstkey 0xffffffff818f2080 0xffffffff8186f790
pfstate 0xffffffff818f2a00 0xffffffff8186f790
pfsnitem 0xffffffff818f1f00 0xffffffff8186f790
pfsrctr 0xffffffff818f30a0 0xffffffff8186f790
pfrule 0xffffffff818f2b60 0xffffffff8186f790
sysvmsgpl 0xffffffff81904600 0xffffffff8186f790
semupl 0xffffffff819047a0 0xffffffff8186f790
semapl 0xffffffff81904900 0xffffffff8186f790
shmpl 0xffffffff81904a80 0xffffffff8186f790
tmpfs_node 0xffffffff81906c40 0xffffffff8186f790
tmpfs_dirent 0xffffffff81906ae0 0xffffffff8186f790
fmsg 0xffffffff818f1be0 0xffffffff8186f790
udfdspl 0xffffffff818f8f00 0xffffffff8186f790
udfndpl 0xffffffff818f8c40 0xffffffff8186f790
udftrpl 0xffffffff818f8da0 0xffffffff8186f790
nfsnodepl 0xffffffff81921520 0xffffffff8186f790
ext2dinopl 0xffffffff81923b40 0xffffffff8186f790
ext2inopl 0xffffffff819239e0 0xffffffff8186f790
dirhash 0xffffffff819237c0 0xffffffff8186f790
newdirblk 0xffffffff81922f40 0xffffffff8186f790
dirrem 0xffffffff81923640 0xffffffff8186f790
mkdir 0xffffffff819227c0 0xffffffff8186f790
diradd 0xffffffff81922500 0xffffffff8186f790
freefile 0xffffffff81922660 0xffffffff8186f790
freeblks 0xffffffff81923220 0xffffffff8186f790
freefrag 0xffffffff81922c00 0xffffffff8186f790
allocindir 0xffffffff81922de0 0xffffffff8186f790
indirdep 0xffffffff819230c0 0xffffffff8186f790
allocdir 0xffffffff81922aa0 0xffffffff8186f790
bmsafemap 0xffffffff819223a0 0xffffffff8186f790
newblk 0xffffffff819234e0 0xffffffff8186f790
inodedep 0xffffffff81922920 0xffffffff8186f790
pagedep 0xffffffff81923380 0xffffffff8186f790
dino2pl 0xffffffff81921f60 0xffffffff8186f790
dino1pl 0xffffffff819220c0 0xffffffff8186f790
ffsino 0xffffffff81922220 0xffffffff8186f790
nchpl 0xffffffff81906060 0xffffffff8186f790
uvmvnodes 0xffffffff81906540 0xffffffff8186f790
vnodes 0xffffffff819066a0 0xffffffff8186f790
namei 0xffffffff81906260 0xffffffff8186f790
nfsreqpl 0xffffffff819216e0 0xffffffff8186f790
ndscpl 0xffffffff81921ac0 0xffffffff8186f790
ohcixfer 0xffff800000168000 0xffffffff8186f790
wdcxfer 0xffffffff818f0980 0xffffffff8186f790
ehcixfer 0xffff800000151a00 0xffffffff8186f790
pfiaddrpl 0xffffffff818f55a0 0xffffffff8186f790
drmobjpl 0xffff8000000cf3d8 0xffffffff8186f790
scsiplug 0xffffffff8192bc00 0xffffffff8186f790
scxspl 0xffffffff8192baa0 0xffffffff8186f790
sigapl 0xffffffff818fda20 0xffffffff8186f790
knotepl 0xffffffff818f9da0 0xffffffff8186f790
kqueuepl 0xffffffff818f9c40 0xffffffff8186f790
pipepl 0xffffffff81904480 0xffffffff8186f790
fdescpl 0xffffffff818f9980 0xffffffff8186f790
filepl 0xffffffff818f9ae0 0xffffffff8186f790
lockfpl 0xffffffff819063c0 0xffffffff8186f790
sessionpl 0xffffffff818fcf60 0xffffffff8186f790
pgrppl 0xffffffff818fd240 0xffffffff8186f790
ucredpl 0xffffffff818fd3a0 0xffffffff8186f790
zombiepl 0xffffffff818fd500 0xffffffff8186f790
processpl 0xffffffff818fce00 0xffffffff8186f790
procpl 0xffffffff818fd0c0 0xffffffff8186f790
sosppl 0xffffffff81905bc0 0xffffffff8186f790
sockpl 0xffffffff81905a60 0xffffffff8186f790
mcl64k 0xffffffff819057b0 0xffffffff8186f7b0
mcl16k 0xffffffff81905668 0xffffffff8186f7b0
mcl12k 0xffffffff81905520 0xffffffff8186f7b0
mcl9k 0xffffffff819053d8 0xffffffff8186f7b0
mcl8k 0xffffffff81905290 0xffffffff8186f7b0
mcl4k 0xffffffff81905148 0xffffffff8186f790
mcl2k 0xffffffff81905000 0xffffffff8186f790
mtagpl 0xffffffff81905900 0xffffffff8186f790
mbufpl 0xffffffff81904ea0 0xffffffff8186f790
bufpl 0xffffffff81905e60 0xffffffff8186f790
anonpl 0xffffffff81924500 0xffffffff8186f790
amappl 0xffffffff819243a0 0xffffffff8186f790
dma65536 0xffffffff81903e40 0xffffffff8186f7b0
dma32768 0xffffffff81903cf8 0xffffffff8186f7b0
dma16384 0xffffffff81903bb0 0xffffffff8186f7b0
dma8192 0xffffffff81903a68 0xffffffff8186f7b0
dma4096 0xffffffff81903920 0xffffffff8186f790
dma2048 0xffffffff819037d8 0xffffffff8186f790
dma1024 0xffffffff81903690 0xffffffff8186f790
dma512 0xffffffff81903548 0xffffffff8186f790
dma256 0xffffffff81903400 0xffffffff8186f790
dma128 0xffffffff819032b8 0xffffffff8186f790
dma64 0xffffffff81903170 0xffffffff8186f790
dma32 0xffffffff81903028 0xffffffff8186f790
dma16 0xffffffff81902ee0 0xffffffff8186f790
aobjpl 0xffffffff819247c0 0xffffffff8186f790
uaoeltpl 0xffffffff81924660 0xffffffff8186f790
uaddrrnd 0xffffffff81923e00 0xffffffff8186f790
uaddrpivot 0xffffffff81923ca0 0xffffffff8186f790
uaddrbest 0xffffffff81923f80 0xffffffff8186f790
uaddrhint 0xffffffff819240e0 0xffffffff8186f790
uaddr 0xffffffff81924240 0xffffffff8186f790
vmmpekpl 0xffffffff81924e20 0xffffffff8186f790
vmmpepl 0xffffffff81924b60 0xffffffff8186f790
vmsppl 0xffffffff81924cc0 0xffffffff8186f790
pdppl 0xffffffff8192b3c0 0xffffffff8186f790
pvpl 0xffffffff8192b100 0xffffffff8186f790
pmappl 0xffffffff8192b260 0xffffffff8186f790
extentpl 0xffffffff81902960 0xffffffff8186f790
phpool 0xffffffff81902d80 0xffffffff8186f790
ddb{0}> show extents
extent `swap0x0000' (0x0 - 0x20063d), flags=0
0x0 - 0x0
0x20063c - 0x20063c
extent `swapmap' (0x1 - 0x7fffffff), flags=0
0x1 - 0x20063e
extent `ppb2 pcipmem' (0x0 - 0xffffffffffffffff), flags=0
0x0 - 0xfaefffff
0xfaff0000 - 0xffffffffffffffff
extent `ppb2 pcimem' (0x0 - 0xffffffffffffffff), flags=0
0x0 - 0xfbefffff
0xfbfc0000 - 0xfbfdffff
0xfbfff000 - 0xffffffffffffffff
extent `ppb2 pciio' (0x0 - 0xffffffff), flags=0
0x0 - 0xdfff
0xe800 - 0xe8ff
0xf000 - 0xffffffff
extent `ppb1 pcimem' (0x0 - 0xffffffffffffffff), flags=0
0x0 - 0xfbdfffff
0xfbefc000 - 0xffffffffffffffff
extent `ppb1 pciio' (0x0 - 0xffffffff), flags=0
0x0 - 0xcfff
0xd800 - 0xd8ff
0xe000 - 0xffffffff
extent `drmext' (0x40000000 - 0x7fffffffffffffff), flags=2<NOCOALESCE>
extent `ppb0 pcipmem' (0x0 - 0xffffffffffffffff), flags=0
0x0 - 0xffffffffffffffff
extent `ppb0 pcimem' (0x0 - 0xffffffffffffffff), flags=0
0x0 - 0xfbcfffff
0xfbde8000 - 0xfbdebfff
0xfbdf0000 - 0xffffffffffffffff
extent `ppb0 pciio' (0x0 - 0xffffffff), flags=0
0x0 - 0xc0ff
0xd000 - 0xffffffff
extent `pcibus' (0x0 - 0xff), flags=0
0x0 - 0x4
extent `pcimem' (0x0 - 0xffffffffffffffff), flags=0
0x0 - 0xdfffffff
0xfaf00000 - 0xfaffffff
0xfbbf4000 - 0xfbbf7fff
0xfbbf9000 - 0xfbbf9fff
0xfbbfa800 - 0xfbbfa8ff
0xfbbfb000 - 0xfbbff0ff
0xfbbff800 - 0xfbbffbff
0xfbc00000 - 0xfbffffff
0xff700000 - 0xffffffff
0x40000000000 - 0xffffffffffffffff
extent `pciio' (0x0 - 0xffffffff), flags=0
0x7000 - 0x700f
0x8000 - 0x8003
0x9000 - 0x9007
0xa000 - 0xa003
0xb000 - 0xb007
0xc000 - 0xefff
0xff00 - 0xff0f
0x10000 - 0xffffffff
extent `iomem' (0x0 - 0xffffffffffff), flags=3<NOCOALESCE,FIXED>
0x23000 - 0x9dfff
0xa0000 - 0xbffff
0x100000 - 0xffffff
0x1000000 - 0xcff7ffff
0xd0040000 - 0xd013ffff
0xd0140000 - 0xd0140fff
0xd0142000 - 0xd0441fff
0xfbbf4000 - 0xfbbf7fff
0xfbbf9000 - 0xfbbf9fff
0xfbbfa800 - 0xfbbfa8ff
0xfbbfb000 - 0xfbbfbfff
0xfbbfc000 - 0xfbbfcfff
0xfbbfd000 - 0xfbbfdfff
0xfbbfe000 - 0xfbbfefff
0xfbbff000 - 0xfbbff0ff
0xfbbff800 - 0xfbbffbff
0xfbdf0000 - 0xfbdfffff
0xfbefc000 - 0xfbefffff
0xfed00000 - 0xfed003ff
0x100000000 - 0x21fffffff
extent `ioport' (0x0 - 0xffff), flags=3<NOCOALESCE,FIXED>
0x0 - 0xb
0x2e - 0x2f
0x40 - 0x43
0x60 - 0x60
0x61 - 0x61
0x64 - 0x64
0x80 - 0x8e
0xb0 - 0xb0
0xc0 - 0xd7
0x170 - 0x177
0x1f0 - 0x1f7
0x290 - 0x297
0x376 - 0x376
0x378 - 0x37b
0x3c0 - 0x3cf
0x3d0 - 0x3df
0x3f0 - 0x3f5
0x3f6 - 0x3f6
0x3f7 - 0x3f7
0x3f8 - 0x3ff
0x800 - 0x801
0x802 - 0x803
0x804 - 0x805
0x808 - 0x80b
0x820 - 0x823
0x824 - 0x827
0x8ff - 0x8ff
0xb00 - 0xb0f
0xc000 - 0xc0ff
0xe800 - 0xe8ff
0xff00 - 0xff0f
ddb{0}> boot sync
syncing disks... panic: assertwaitok: non-zero mutex count: 1
Stopped at Debugger+0x9: leave
RUN AT LEAST 'trace' AND 'ps' AND INCLUDE OUTPUT WHEN REPORTING THIS PANIC!
IF RUNNING SMP, USE 'mach ddbcpu <#>' AND 'trace' ON OTHER PROCESSORS, TOO.
DO NOT EVEN BOTHER REPORTING THIS WITHOUT INCLUDING THAT INFORMATION!
ddb{0}> show registers
ds 0x17f8 __ALIGN_SIZE+0x7f8
es 0x3240 __ALIGN_SIZE+0x2240
fs 0xaf5
gs 0xd
rdi 0x1
rsi 0x296
rbp 0xffff800022033230
rbx 0xffffffff816e17f8 systqmp+0x2a38
rdx 0
rcx 0xffffffff818819c0 cpu_info_primary
rax 0x1
r8 0xffff800022033150
r9 0
r10 0xffff8000220330f8
r11 0x8
r12 0x104
r13 0xffff800022033240
r14 0x4
r15 0
rip 0xffffffff813453a9 Debugger+0x9
cs 0x8
rflags 0x282
rsp 0xffff800022033220
ss 0x10
Debugger+0x9: leave
ddb{0}> trace
Debugger() at Debugger+0x9
panic() at panic+0xfe
assertwaitok() at assertwaitok+0x52
bufq_wait() at bufq_wait+0x2d
bwrite() at bwrite+0xf6
VOP_BWRITE() at VOP_BWRITE+0x38
ffs_fsync() at ffs_fsync+0x13f
VOP_FSYNC() at VOP_FSYNC+0x3c
ffs_sync_vnode() at ffs_sync_vnode+0x6a
vfs_mount_foreach_vnode() at vfs_mount_foreach_vnode+0x35
ffs_sync() at ffs_sync+0x89
sys_sync() at sys_sync+0x87
vfs_syncwait() at vfs_syncwait+0x50
vfs_shutdown() at vfs_shutdown+0x36
boot() at boot+0x148
reboot() at reboot+0x26
db_boot_sync_cmd() at db_boot_sync_cmd+0x12
db_command() at db_command+0x13a
db_command_loop() at db_command_loop+0xb0
db_trap() at db_trap+0xe4
kdb_trap() at kdb_trap+0xf0
trap() at trap+0x15d
--- trap (number 1) ---
Debugger() at Debugger+0x9
panic() at panic+0xfe
pool_runqueue() at pool_runqueue
pool_get() at pool_get+0xb5
m_clget() at m_clget+0x51
re_newbuf() at re_newbuf+0x34
re_rx_list_fill() at re_rx_list_fill+0x35
re_rxeof() at re_rxeof+0x333
re_intr() at re_intr+0x19b
intr_handler() at intr_handler+0x67
Xintr_ioapic_edge18() at Xintr_ioapic_edge18+0xdd
--- interrupt ---
cpu_idle_mwait_cycle() at cpu_idle_mwait_cycle+0x62
cpu_idle_cycle() at cpu_idle_cycle+0x10
end trace frame: 0x0, count: -35
ddb{0}> ps
PID PPID PGRP UID S FLAGS WAIT COMMAND
26916 24107 26916 0 3 0x83 getblk dd
790 4247 22307 1000 3 0x82 nanosleep sleep
16419 7909 7909 1000 3 0x82 nanosleep perl
7909 18528 7909 1000 3 0x8a pause sh
18528 23947 23947 0 3 0x80 piperd cron
4247 22307 22307 1000 3 0x8a pause sh
10000 5347 5347 0 3 0x82 nanosleep perl
24107 1 24107 0 3 0x8b pause ksh
25720 1 25720 0 3 0x83 ttyin getty
17251 1 17251 0 3 0x83 ttyin getty
21816 1 21816 0 3 0x83 ttyin getty
25792 1 25792 0 3 0x83 ttyin getty
27641 1 27641 0 3 0x83 ttyin getty
22307 23331 22307 1000 3 0x8a pause sh
5347 6158 5347 0 3 0x8a pause sh
23331 23947 23947 0 3 0x80 piperd cron
6158 23947 23947 0 3 0x80 piperd cron
23947 1 23947 0 3 0x80 poll cron
8704 1 8704 0 3 0x80 kqread apmd
20169 1 20169 0 7 0 apcupsd
7491 1 20169 0 3 0x4000088 sigwait apcupsd
820 1 20169 0 3 0x4000080 select apcupsd
30745 1 30745 0 3 0x80 poll nmbd
21377 30807 30807 0 3 0x90 poll smbd
30807 1 30807 0 3 0x90 poll smbd
662 1 662 99 3 0x90 poll sndiod
32353 9708 9708 95 3 0x90 kqread smtpd
31775 9708 9708 95 3 0x90 kqread smtpd
6539 9708 9708 95 3 0x90 kqread smtpd
11711 9708 9708 95 3 0x90 kqread smtpd
23602 9708 9708 95 3 0x90 kqread smtpd
12815 9708 9708 103 3 0x90 kqread smtpd
9708 1 9708 0 3 0x80 kqread smtpd
18347 1 18347 0 3 0xb0 select lpd
1534 1 1534 0 3 0x80 select sshd
31949 0 0 0 3 0x14280 nfsidl nfsio
1151 0 0 0 3 0x14280 nfsidl nfsio
20004 0 0 0 3 0x14280 nfsrcvlk nfsio
6326 0 0 0 3 0x14200 netio nfsio
11227 27093 27093 0 3 0x80 nfsd nfsd
2401 27093 27093 0 3 0x80 nfsd nfsd
28351 27093 27093 0 3 0x80 nfsd nfsd
11723 27093 27093 0 3 0x80 nfsd nfsd
27093 1 27093 0 3 0x80 netcon nfsd
6235 1 6235 0 3 0x80 poll mountd
2311 1 2311 28 3 0x90 poll portmap
4706 1 4706 0 3 0x80 poll ntpd
23037 5497 23037 83 3 0x90 poll ntpd
5497 1 5497 83 3 0x90 poll ntpd
27817 1 27817 53 3 0x90 kqread unbound
20949 16915 16915 74 3 0x90 bpf pflogd
16915 1 16915 0 3 0x80 netio pflogd
14497 4216 4216 73 3 0x90 kqread syslogd
4216 1 4216 0 3 0x80 netio syslogd
31135 1 31135 77 3 0x90 poll dhclient
21643 1 21643 0 3 0x80 poll dhclient
8966 1 8966 0 3 0x80 mfsidl mount_mfs
13251 0 0 0 3 0x14200 bored ttm_swap
23958 0 0 0 3 0x14200 pgzero zerothread
8226 0 0 0 3 0x14200 aiodoned aiodoned
11051 0 0 0 3 0x14200 syncer update
19572 0 0 0 3 0x14200 cleaner cleaner
9291 0 0 0 3 0x14200 reaper reaper
31035 0 0 0 3 0x14200 pgdaemon pagedaemon
9434 0 0 0 3 0x14200 bored crypto
13551 0 0 0 3 0x14200 pftm pfpurge
2480 0 0 0 3 0x14200 bored sensors
23352 0 0 0 3 0x14200 usbtsk usbtask
9784 0 0 0 3 0x14200 usbatsk usbatsk
12107 0 0 0 3 0x40014200 acpi0 acpi0
27369 0 0 0 7 0x40014200 idle3
10730 0 0 0 7 0x40014200 idle2
31139 0 0 0 3 0x40014200 idle1
10377 0 0 0 3 0x14200 bored softnet
10567 0 0 0 3 0x14200 bored systqmp
13628 0 0 0 3 0x14200 bored systq
*30240 0 0 0 7 0x40014200 idle0
1 0 1 0 3 0x82 wait init
0 -1 0 0 3 0x10200 scheduler swapper
ddb{0}> boot reboot
panic: mtx_enter: locking against myself
Stopped at Debugger+0x9: leave
RUN AT LEAST 'trace' AND 'ps' AND INCLUDE OUTPUT WHEN REPORTING THIS PANIC!
IF RUNNING SMP, USE 'mach ddbcpu <#>' AND 'trace' ON OTHER PROCESSORS, TOO.
DO NOT EVEN BOTHER REPORTING THIS WITHOUT INCLUDING THAT INFORMATION!
ddb{0}> show registers
ds 0x2b88 __ALIGN_SIZE+0x1b88
es 0x2f00 __ALIGN_SIZE+0x1f00
fs 0xf85b __ALIGN_SIZE+0xe85b
gs 0xf85b __ALIGN_SIZE+0xe85b
rdi 0x1
rsi 0x296
rbp 0xffff800022032b78
rbx 0xffffffff81327de1 mtx_enter+0x61
rdx 0
rcx 0
rax 0x1
r8 0xffff800022032a98
r9 0xffff8000000fc000
r10 0xb
r11 0xffffffff81086c30 comcnputc
r12 0x104
r13 0xffff800022032b88
r14 0xffff8000000fc000
r15 0xd
rip 0xffffffff813453a9 Debugger+0x9
cs 0x8
rflags 0x282
rsp 0xffff800022032b68
ss 0x10
Debugger+0x9: leave
ddb{0}> trace
Debugger() at Debugger+0x9
panic() at panic+0xfe
mtx_enter() at mtx_enter+0x61
m_extfree() at m_extfree+0x7e
m_free() at m_free+0x55
m_freem() at m_freem+0x15
re_stop() at re_stop+0x17b
re_ioctl() at re_ioctl+0x17a
if_downall() at if_downall+0x8a
boot() at boot+0xe4
reboot() at reboot+0x26
db_boot_reboot_cmd() at db_boot_reboot_cmd+0x12
db_command() at db_command+0x13a
db_command_loop() at db_command_loop+0xb0
db_trap() at db_trap+0xe4
kdb_trap() at kdb_trap+0xf0
trap() at trap+0x15d
--- trap (number 1) ---
Debugger() at Debugger+0x9
panic() at panic+0xfe
assertwaitok() at assertwaitok+0x52
bufq_wait() at bufq_wait+0x2d
bwrite() at bwrite+0xf6
VOP_BWRITE() at VOP_BWRITE+0x38
ffs_fsync() at ffs_fsync+0x13f
VOP_FSYNC() at VOP_FSYNC+0x3c
ffs_sync_vnode() at ffs_sync_vnode+0x6a
vfs_mount_foreach_vnode() at vfs_mount_foreach_vnode+0x35
ffs_sync() at ffs_sync+0x89
sys_sync() at sys_sync+0x87
vfs_syncwait() at vfs_syncwait+0x50
vfs_shutdown() at vfs_shutdown+0x36
boot() at boot+0x148
reboot() at reboot+0x26
db_boot_sync_cmd() at db_boot_sync_cmd+0x12
db_command() at db_command+0x13a
db_command_loop() at db_command_loop+0xb0
db_trap() at db_trap+0xe4
kdb_trap() at kdb_trap+0xf0
trap() at trap+0x15d
--- trap (number 1) ---
Debugger() at Debugger+0x9
panic() at panic+0xfe
pool_runqueue() at pool_runqueue
pool_get() at pool_get+0xb5
m_clget() at m_clget+0x51
re_newbuf() at re_newbuf+0x34
re_rx_list_fill() at re_rx_list_fill+0x35
re_rxeof() at re_rxeof+0x333
re_intr() at re_intr+0x19b
intr_handler() at intr_handler+0x67
Xintr_ioapic_edge18() at Xintr_ioapic_edge18+0xdd
--- interrupt ---
cpu_idle_mwait_cycle() at cpu_idle_mwait_cycle+0x62
cpu_idle_cycle() at cpu_idle_cycle+0x10
end trace frame: 0x0, count: -52
ddb{0}>
