Hi,
On our server running with OpenBSD 5.5 calling netstat sometimes lead to an
uvm_fault in sysctl_file().
I can trigger the problem by running some of our tests which create/delete a lot
of files, open/close sockets etc. and call netstat -anv.
Attached file contains dmesg, trace and ps output.
Greetings,
Matthias
ddb{3}> dmesg
OpenBSD 5.5 (GENERIC.MP) #315: Wed Mar 5 09:37:46 MST 2014
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 6416826368 (6119MB)
avail mem = 6237401088 (5948MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.6 @ 0x99c00 (88 entries)
bios0: vendor American Megatrends Inc. version "2.0c" date (null)
bios0: Supermicro AS0600
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S4 S5
acpi0: tables DSDT FACP APIC MCFG SLIT SPMI OEMB HPET SSDT
acpi0: wakeup devices NPE1(S4) NPE2(S4) NPE3(S4) NPE4(S4) NPE5(S4) NPE6(S4) NPE
7(S4) NPE8(S4) NPE9(S4) NPEA(S4) P0P1(S4) USB0(S4) USB1(S4) USB2(S4) USB5(S4) E
USB(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU X5672 @ 3.20GHz, 3200.56 MHz
cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,SMX,
EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF,PERF,I
TSC
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 136MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1.0, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU X5672 @ 3.20GHz, 3264.12 MHz
cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,SMX,
EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF,PERF,I
TSC
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 18 (application processor)
cpu2: Intel(R) Xeon(R) CPU X5672 @ 3.20GHz, 3264.12 MHz
cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,SMX,
EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF,PERF,I
TSC
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 9, package 0
cpu3 at mainbus0: apid 20 (application processor)
cpu3: Intel(R) Xeon(R) CPU X5672 @ 3.20GHz, 3264.12 MHz
cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C
FLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,VMX,SMX,
EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF,PERF,I
TSC
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 10, package 0
ioapic0 at mainbus0: apid 1 pa 0xfec00000, version 20, 24 pins
ioapic0: misconfigured as apic 4, remapped to apid 1
ioapic1 at mainbus0: apid 3 pa 0xfec8a000, version 20, 24 pins
ioapic1: misconfigured as apic 6, remapped to apid 3
ioapic2 at mainbus0: apid 5 pa 0xfec9a000, version 20, 24 pins
ioapic2: misconfigured as apic 8, remapped to apid 5
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (NPE1)
acpiprt2 at acpi0: bus -1 (NPE2)
acpiprt3 at acpi0: bus 2 (NPE3)
acpiprt4 at acpi0: bus -1 (NPE4)
acpiprt5 at acpi0: bus 3 (NPE5)
acpiprt6 at acpi0: bus -1 (NPE6)
acpiprt7 at acpi0: bus 4 (NPE7)
acpiprt8 at acpi0: bus -1 (NPE8)
acpiprt9 at acpi0: bus 5 (NPE9)
acpiprt10 at acpi0: bus -1 (NPEA)
acpiprt11 at acpi0: bus 6 (P0P1)
acpiprt12 at acpi0: bus -1 (P0P4)
acpiprt13 at acpi0: bus -1 (P0P5)
acpiprt14 at acpi0: bus -1 (P0P6)
acpiprt15 at acpi0: bus -1 (P0P7)
acpiprt16 at acpi0: bus -1 (P0P8)
acpiprt17 at acpi0: bus -1 (P0P9)
acpiprt18 at acpi0: bus 128 (BR50)
acpiprt19 at acpi0: bus 130 (NPE1)
acpiprt20 at acpi0: bus -1 (NPE2)
acpiprt21 at acpi0: bus 131 (NPE3)
acpiprt22 at acpi0: bus -1 (NPE4)
acpiprt23 at acpi0: bus 132 (NPE5)
acpiprt24 at acpi0: bus -1 (NPE6)
acpiprt25 at acpi0: bus 133 (NPE7)
acpiprt26 at acpi0: bus -1 (NPE8)
acpiprt27 at acpi0: bus 134 (NPE9)
acpiprt28 at acpi0: bus -1 (NPEA)
acpicpu0 at acpi0: PSS
acpicpu1 at acpi0: PSS
acpicpu2 at acpi0: PSS
acpicpu3 at acpi0: PSS
acpibtn0 at acpi0: SLPB
acpibtn1 at acpi0: PWRB
ipmi at mainbus0 not configured
cpu0: Enhanced SpeedStep 3200 MHz: speeds: 3201, 3200, 3067, 2933, 2800, 2667, 2
533, 2400, 2267, 2133, 2000, 1867, 1733, 1600 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 5520 Host" rev 0x22
ppb0 at pci0 dev 1 function 0 "Intel X58 PCIE" rev 0x22: msi
pci1 at ppb0 bus 1
em0 at pci1 dev 0 function 0 "Intel 82576" rev 0x01: msi, address 00:25:90:69:3
9:52
em1 at pci1 dev 0 function 1 "Intel 82576" rev 0x01: msi, address 00:25:90:69:3
9:53
ppb1 at pci0 dev 3 function 0 "Intel X58 PCIE" rev 0x22: msi
pci2 at ppb1 bus 2
ppb2 at pci0 dev 5 function 0 "Intel X58 PCIE" rev 0x22: msi
pci3 at ppb2 bus 3
ppb3 at pci0 dev 7 function 0 "Intel X58 PCIE" rev 0x22: msi
pci4 at ppb3 bus 4
ppb4 at pci0 dev 9 function 0 "Intel X58 PCIE" rev 0x22: msi
pci5 at ppb4 bus 5
mpii0 at pci5 dev 0 function 0 "Symbios Logic SAS2008" rev 0x03: msi
mpii0: SMC2008-IR, firmware 11.0.0.0 IR, MPI 2.0
scsibus0 at mpii0: 258 targets
sd0 at scsibus0 targ 0 lun 0: <LSI, Logical Volume, 3000> SCSI4 0/direct fixed n
aa.600508e00000000083dd277ea32cda09
sd0: 139236MB, 512 bytes/sector, 285155328 sectors
"Intel X58 IOxAPIC" rev 0x22 at pci0 dev 19 function 0 not configured
"Intel X58 Misc" rev 0x22 at pci0 dev 20 function 0 not configured
"Intel X58 GPIO" rev 0x22 at pci0 dev 20 function 1 not configured
"Intel X58 RAS" rev 0x22 at pci0 dev 20 function 2 not configured
"Intel X58 Throttle" rev 0x22 at pci0 dev 20 function 3 not configured
"Intel X58 QuickData" rev 0x22 at pci0 dev 22 function 0 not configured
"Intel X58 QuickData" rev 0x22 at pci0 dev 22 function 1 not configured
"Intel X58 QuickData" rev 0x22 at pci0 dev 22 function 2 not configured
"Intel X58 QuickData" rev 0x22 at pci0 dev 22 function 3 not configured
"Intel X58 QuickData" rev 0x22 at pci0 dev 22 function 4 not configured
"Intel X58 QuickData" rev 0x22 at pci0 dev 22 function 5 not configured
"Intel X58 QuickData" rev 0x22 at pci0 dev 22 function 6 not configured
"Intel X58 QuickData" rev 0x22 at pci0 dev 22 function 7 not configured
uhci0 at pci0 dev 26 function 0 "Intel 82801JI USB" rev 0x00: apic 1 int 16
uhci1 at pci0 dev 26 function 1 "Intel 82801JI USB" rev 0x00: apic 1 int 21
uhci2 at pci0 dev 26 function 2 "Intel 82801JI USB" rev 0x00: apic 1 int 19
ehci0 at pci0 dev 26 function 7 "Intel 82801JI USB" rev 0x00: apic 1 int 18
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
uhci3 at pci0 dev 29 function 0 "Intel 82801JI USB" rev 0x00: apic 1 int 23
uhci4 at pci0 dev 29 function 1 "Intel 82801JI USB" rev 0x00: apic 1 int 19
uhci5 at pci0 dev 29 function 2 "Intel 82801JI USB" rev 0x00: apic 1 int 18
ehci1 at pci0 dev 29 function 7 "Intel 82801JI USB" rev 0x00: apic 1 int 23
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb5 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0x90
pci6 at ppb5 bus 6
vga1 at pci6 dev 4 function 0 "Matrox MGA G200eW" rev 0x0a
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
pcib0 at pci0 dev 31 function 0 "Intel 82801JIR LPC" rev 0x00
ahci0 at pci0 dev 31 function 2 "Intel 82801JI AHCI" rev 0x00: msi, AHCI 1.2
scsibus1 at ahci0: 32 targets
cd0 at scsibus1 targ 0 lun 0: <TEAC, DV-W28SS-V, 1.0A> ATAPI 5/cdrom removable
ichiic0 at pci0 dev 31 function 3 "Intel 82801JI SMBus" rev 0x00: apic 1 int 18
iic0 at ichiic0
iic0: addr 0x2e 00=40 words 00=4040 01=0000 02=0000 03=0000 04=0000 05=0000 06=
0000 07=0000
nvt0 at iic0 addr 0x2f: W83795ADG
usb2 at uhci0: USB revision 1.0
uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb3 at uhci1: USB revision 1.0
uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb4 at uhci2: USB revision 1.0
uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb5 at uhci3: USB revision 1.0
uhub5 at usb5 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb6 at uhci4: USB revision 1.0
uhub6 at usb6 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb7 at uhci5: USB revision 1.0
uhub7 at usb7 "Intel UHCI root hub" rev 1.00/1.00 addr 1
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
wbsio0 at isa0 port 0x2e/2: W83627DHG-P rev 0x73
lm1 at wbsio0 port 0xa10/8: W83627DHG
pci7 at mainbus0 bus 128
ppb6 at pci7 dev 0 function 0 vendor "Intel", unknown product 0x3420 rev 0x22
pci8 at ppb6 bus 129
ppb7 at pci7 dev 1 function 0 "Intel X58 PCIE" rev 0x22: msi
pci9 at ppb7 bus 130
ppb8 at pci7 dev 3 function 0 "Intel X58 PCIE" rev 0x22: msi
pci10 at ppb8 bus 131
em2 at pci10 dev 0 function 0 "Intel 82576" rev 0x01: msi, address 90:e2:ba:0a:
dc:68
em3 at pci10 dev 0 function 1 "Intel 82576" rev 0x01: msi, address 90:e2:ba:0a:
dc:69
ppb9 at pci7 dev 5 function 0 "Intel X58 PCIE" rev 0x22: msi
pci11 at ppb9 bus 132
ppb10 at pci7 dev 7 function 0 "Intel X58 PCIE" rev 0x22: msi
pci12 at ppb10 bus 133
ppb11 at pci7 dev 9 function 0 "Intel X58 PCIE" rev 0x22: msi
pci13 at ppb11 bus 134
"Intel X58 IOxAPIC" rev 0x22 at pci7 dev 19 function 0 not configured
"Intel X58 Misc" rev 0x22 at pci7 dev 20 function 0 not configured
"Intel X58 GPIO" rev 0x22 at pci7 dev 20 function 1 not configured
"Intel X58 RAS" rev 0x22 at pci7 dev 20 function 2 not configured
"Intel X58 Throttle" rev 0x22 at pci7 dev 20 function 3 not configured
"Intel X58 QuickData" rev 0x22 at pci7 dev 22 function 0 not configured
"Intel X58 QuickData" rev 0x22 at pci7 dev 22 function 1 not configured
"Intel X58 QuickData" rev 0x22 at pci7 dev 22 function 2 not configured
"Intel X58 QuickData" rev 0x22 at pci7 dev 22 function 3 not configured
"Intel X58 QuickData" rev 0x22 at pci7 dev 22 function 4 not configured
"Intel X58 QuickData" rev 0x22 at pci7 dev 22 function 5 not configured
"Intel X58 QuickData" rev 0x22 at pci7 dev 22 function 6 not configured
"Intel X58 QuickData" rev 0x22 at pci7 dev 22 function 7 not configured
umass0 at uhub0 port 2 configuration 1 interface 0 "Generic Mass Storage Device
" rev 2.00/1.00 addr 2
umass0: using SCSI over Bulk-Only
scsibus2 at umass0: 2 targets, initiator 0
sd1 at scsibus2 targ 1 lun 0: <Generic, Storage Device, 0.00> SCSI2 0/direct re
movable serial.048d1336000000000006
ugen0 at uhub2 port 1 "SCM Microsystems Inc. SCR3310 v2.0 USB SC Reader" rev 2.
00/2.04 addr 2
vscsi0 at root
scsibus3 at vscsi0: 256 targets
softraid0 at root
scsibus4 at softraid0: 256 targets
root on sd0a (5964bf704a771eed.a) swap on sd0b dump on sd0b
ddb{3}> trace
sysctl_file() at sysctl_file+0x180
kern_sysctl() at kern_sysctl+0x686
sys___sysctl() at sys___sysctl+0x1f4
syscall() at syscall+0x24f
--- syscall (number 202) ---
end of kernel
end trace frame: 0xc0277bc1760, count: -4
0xc0280b3b74a:
ddb{3}> ps
PID PPID PGRP UID S FLAGS WAIT COMMAND
1414 6219 6219 2315 2 0x1 bash
8635 20340 20340 2315 3 0xb3 sysctllk netstat
6074 167 167 2315 2 0 perl
4725 21966 21966 2315 2 0x33 netstat
*11703 6845 6845 2315 7 0x33 netstat
7010 8407 8407 2315 3 0xb3 sysctllk netstat
25625 20047 20047 2315 3 0x82 sysctllk unbound-control
10159 32627 32627 2315 2 0x33 netstat
20047 6580 20047 2315 3 0x8a pause sh
23923 18590 15748 2315 7 0 perl
18590 6889 15748 2315 3 0x80 piperd perl
14389 4295 27391 2315 2 0 perl
3089 32408 13808 2315 2 0 perl
32408 5393 13808 2315 3 0x80 piperd perl
5094 29930 5094 2315 3 0xb3 poll systat
25641 15217 1663 2315 7 0 perl
15217 5729 1663 2315 3 0x80 piperd perl
6580 14783 22414 2315 3 0x80 wait perl
14783 27919 22414 2315 3 0x80 piperd perl
20340 29930 20340 2315 3 0x81 wait bash
32627 29930 32627 2315 3 0x81 wait bash
6219 29930 6219 2315 3 0x81 wait bash
6845 29930 6845 2315 3 0x81 wait bash
8407 29930 8407 2315 3 0x81 wait bash
6238 31336 27391 2315 3 0x80 select perl
4295 31336 27391 2315 3 0x80 select perl
21966 29930 21966 2315 3 0x81 wait bash
32270 4043 13808 2315 3 0x80 netcon perl
5393 4043 13808 2315 3 0x80 select perl
27919 24262 22414 2315 3 0x80 select perl
8902 24262 22414 2315 3 0x80 netcon perl
24262 22414 22414 2315 3 0x82 select perl
30364 22414 22414 2315 3 0x82 select perl
22414 4461 22414 2315 3 0x80 select perl
4461 22427 30457 2315 3 0x82 wait perl
22427 30457 30457 2315 3 0x82 wait perl
31336 27391 27391 2315 3 0x82 select perl
3796 27391 27391 2315 3 0x82 select perl
27391 27201 27391 2315 7 0 perl
27201 10815 17416 2315 3 0x82 wait perl
10815 17416 17416 2315 3 0x82 wait perl
6889 9632 15748 2315 3 0x80 select perl
23074 9632 15748 2315 3 0x80 select perl
30849 23215 1663 2315 3 0x80 select perl
5729 23215 1663 2315 3 0x80 select perl
9632 15748 15748 2315 3 0x82 select perl
23215 1663 1663 2315 3 0x82 select perl
26558 15748 15748 2315 3 0x82 select perl
15748 812 15748 2315 3 0x80 select perl
24369 1663 1663 2315 3 0x82 select perl
1663 11795 1663 2315 2 0 perl
812 28542 11739 2315 3 0x82 wait perl
28542 11739 11739 2315 3 0x82 wait perl
11795 26949 2710 2315 3 0x82 wait perl
26949 2710 2710 2315 3 0x82 wait perl
26103 167 167 2315 3 0x2 netio perl
8633 167 167 2315 3 0x82 select perl
167 10932 167 2315 3 0x80 wait perl
10932 10026 963 2315 3 0x82 wait perl
10026 963 963 2315 3 0x82 wait perl
24896 1490 1490 2315 3 0x83 kqread tail
2710 1 2710 2315 3 0x80 select perl
30457 1 30457 2315 3 0x80 select perl
11739 1 11739 2315 3 0x80 select perl
963 1 963 2315 3 0x80 select perl
1490 17751 1490 2315 3 0x83 wait perl
4043 13808 13808 2315 3 0x82 select perl
7708 13808 13808 2315 3 0x82 select perl
13808 28485 13808 2315 3 0x80 select perl
28485 12248 11571 2315 3 0x82 wait perl
12248 11571 11571 2315 3 0x82 wait perl
4796 9173 9173 2315 3 0x82 pipewr tidyp
21149 29723 29723 2315 3 0x82 pipewr tidyp
19212 31984 9173 2315 3 0x80 select perl
1968 31984 9173 2315 3 0x80 netcon perl
16903 2211 29723 2315 3 0x80 select perl
7079 2211 29723 2315 3 0x80 select perl
29721 24403 29721 2433 3 0x83 ttyin bash
24403 23949 23949 2433 3 0x90 select sshd
23949 12920 23949 0 3 0x92 poll sshd
9745 28698 9745 2345 3 0x83 select vim
31984 9173 9173 2315 3 0x82 select perl
24114 9173 9173 2315 3 0x82 select perl
9173 15622 9173 2315 3 0x80 pipewr perl
15622 4323 19604 2315 3 0x82 wait perl
4323 19604 19604 2315 3 0x82 wait perl
2211 29723 29723 2315 3 0x82 select perl
28906 29723 29723 2315 3 0x82 select perl
29723 32763 29723 2315 3 0x80 pipewr perl
32763 26744 30760 2315 3 0x82 wait perl
26744 30760 30760 2315 3 0x82 wait perl
26928 21318 21318 2315 3 0x83 kqread tail
17416 1 17416 2315 3 0x80 select perl
11571 1 11571 2315 3 0x80 select perl
30760 1 30760 2315 3 0x80 select perl
19604 1 19604 2315 3 0x80 select perl
21318 16069 21318 2315 3 0x83 wait perl
10033 5910 10033 2345 3 0x83 select vim
14334 28936 14334 2257 3 0x83 ttyin less
29340 17659 29340 2257 3 0x83 ttyin less
7245 12602 7245 2257 3 0x83 ttyin less
28936 7367 28936 2257 3 0x8b pause ksh
7367 26385 26385 2257 3 0x90 select sshd
26385 12920 26385 0 3 0x92 poll sshd
12602 21545 12602 2257 3 0x8b pause ksh
21545 11655 11655 2257 3 0x90 select sshd
11655 12920 11655 0 3 0x92 poll sshd
27751 20026 27751 2136 3 0x83 ttyin bash
20026 9950 20026 2136 3 0x80 select screen
9950 12779 9950 2136 3 0x8b pause screen
12779 13632 12779 2136 3 0x83 wait bash
13632 14558 14558 2136 3 0x90 select sshd
14558 12920 14558 0 3 0x92 poll sshd
17659 3899 17659 2257 3 0x8b pause ksh
3899 20214 20214 2257 3 0x90 select sshd
20214 12920 20214 0 3 0x92 poll sshd
26955 25650 26955 2445 3 0x83 ttyin bash
25650 30961 30961 2445 3 0x90 select sshd
30961 12920 30961 0 3 0x92 poll sshd
10830 6075 10830 2136 3 0x83 ttyin bash
6075 1424 1424 2136 3 0x90 select sshd
1424 12920 1424 0 3 0x92 poll sshd
5910 8042 5910 2345 3 0x83 wait bash
8042 26268 26268 2345 3 0x90 select sshd
26268 12920 26268 0 3 0x92 poll sshd
28698 21237 28698 2345 3 0x83 wait bash
21237 7913 7913 2345 3 0x90 select sshd
7913 12920 7913 0 3 0x92 poll sshd
29930 12883 29930 2315 3 0x83 wait bash
12883 14747 14747 2315 3 0x90 select sshd
17751 20896 17751 2315 3 0x83 wait bash
16069 6597 16069 2315 3 0x83 wait bash
6597 26963 26963 2315 3 0x90 select sshd
20896 25451 25451 2315 3 0x90 select sshd
14747 12920 14747 0 3 0x92 poll sshd
26963 12920 26963 0 3 0x92 poll sshd
25451 12920 25451 0 3 0x92 poll sshd
13050 1 13050 0 3 0x83 ttyin getty
1130 1 1130 0 3 0x83 ttyin getty
19893 1 19893 0 3 0x83 ttyin getty
23196 1 23196 0 3 0x83 ttyin getty
12471 1 12471 0 3 0x83 ttyin getty
27519 1 27519 0 3 0x83 ttyin getty
27753 1 27753 0 3 0x80 select cron
20959 1 20959 0 3 0x80 nanosleep sensorsd
491 1 491 99 3 0x90 poll sndiod
16331 6405 6405 67 3 0x90 netcon httpd
30630 6405 6405 67 3 0x90 netcon httpd
28191 6405 6405 67 3 0x90 netcon httpd
23846 6405 6405 67 3 0x90 netcon httpd
7592 6405 6405 67 3 0x90 netcon httpd
6405 1 6405 0 3 0x80 select httpd
16437 1 16437 0 3 0xb0 select sendmail
14879 17122 17122 90 3 0x90 kqread ospf6d
13615 17122 17122 90 3 0x90 kqread ospf6d
17122 1 17122 0 3 0x80 kqread ospf6d
31843 4938 4938 85 3 0x90 kqread ospfd
404 4938 4938 85 3 0x90 kqread ospfd
4938 1 4938 0 3 0x80 kqread ospfd
12920 1 12920 0 3 0x80 select sshd
18657 0 0 0 3 0x4280 nfsidl nfsio
5120 0 0 0 3 0x4280 nfsidl nfsio
18105 0 0 0 3 0x4280 nfsidl nfsio
3296 0 0 0 3 0x4280 nfsidl nfsio
4477 1 4477 0 3 0x80 poll ntpd
2933 27973 2933 83 3 0x90 poll ntpd
27973 1 27973 83 3 0x90 poll ntpd
15665 18236 18236 73 2 0x90 syslogd
18236 1 18236 0 3 0x80 netio syslogd
16482 1 16482 0 3 0x80 mfsidl mount_mfs
4113 0 0 0 3 0x4200 aiodoned aiodoned
3866 0 0 0 3 0x4200 syncer update
3792 0 0 0 3 0x4200 cleaner cleaner
13133 0 0 0 3 0x4200 reaper reaper
7987 0 0 0 3 0x4200 pgdaemon pagedaemon
225 0 0 0 3 0x4200 bored crypto
15026 0 0 0 3 0x4200 pftm pfpurge
27229 0 0 0 3 0x4200 usbtsk usbtask
15362 0 0 0 3 0x4200 usbatsk usbatsk
7481 0 0 0 3 0x40004200 acpi0 acpi0
17800 0 0 0 3 0x40004200 idle3
14740 0 0 0 3 0x40004200 idle2
13786 0 0 0 3 0x40004200 idle1
15327 0 0 0 3 0x4200 bored sensors
15735 0 0 0 3 0x4200 bored systq
24634 0 0 0 3 0x4200 bored syswq
1608 0 0 0 3 0x40004200 idle0
1 0 1 0 3 0x82 wait init
0 -1 0 0 3 0x200 scheduler swapper
