On Mon, Jul 30, 2012 at 11:25 AM, Gregory Edigarov <[email protected]> wrote:
> On 07/30/2012 06:02 PM, Bennett Samowich wrote:
>>
>> The problem exists even if I use the system's "/usr/bin/false" and
>> "/usr/bin/true" commands.
>> The problem exists even when PF is disabled or the only rule is "pass in".
>>
>> That being said the script itself is a simple host lookup against the
>> IP addresses to ensure the DNS server is actually resolving. Again,
>> just using "/usr/bin/false" or "/usr/bin/true" produces the same drop
>> in throughput.
>>
>> An example of the drop looks like this:
>> 9913Mbps
>> 9913Mbps
>> 7253Mbps <--- script interval point
>> 9913Mbps
>> 9913Mbps
>> ...etc...
>>
>> When the script is an actual shell script rather than /usr/bin/false,
>> the throughput drops spans the three seconds surrounding the time the
>> script runs.
>> 9913Mbps
>> 9913Mbps
>> 4321Mbps
>> 7253Mbps <--- script interval point
>> 5162Mbps
>> 9913Mbps
>> 9913Mbps
>>
>> # relayd.conf (somewhat sterilized):
>> table <dns-servers> { 192.168.1.1, 192.168.1.2 }
>> redirect dns-udp {
>> listen on 192.168.100.1 udp port 53
>> forward to <dns-servers> port 53 \
>> check script "/usr/bin/false" \
>> timeout 4000 \
>> interval 15 \
>> mode roundrobin
>> }
>> redirect dns-tcp {
>> listen on 192.168.100.1 port 53
>> forward to <dns-servers> port 53 \
>> check script "/usr/bin/false" \
>> timeout 4000 \
>> interval 15 \
>> mode roundrobin
>> }
>
> looks reasonable to me.
> what 'top' shows during the problems?
> what 'top' shows when there is no problems?
> what 'vmstat -i' shows?
> what's in your dmesg?
>
> I guess you have an SSD or something like this, may be a flash drive?
> I.e. I suspect have something really slow for your disk device, which
> perhaps gives you a high interrupt rate.
>
Drives are 2 x 250GB 1/2 height SATA drives mirrored.
# dmesg
OpenBSD 5.1 (GENERIC.MP) #207: Sun Feb 12 09:42:14 MST 2012
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 12873994240 (12277MB)
avail mem = 12517126144 (11937MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.6 @ 0xcf79c000 (83 entries)
bios0: vendor Dell Inc. version "1.2.6" date 07/17/2009
bios0: Dell Inc. PowerEdge R710
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC SPCR HPET DMAR MCFG WD__ SLIC ERST HEST
BERT EINJ SRAT TCPA SSDT
acpi0: wakeup devices PCI0(S5)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 16 (boot processor)
cpu0: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.35 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: apic clock running at 133MHz
cpu1 at mainbus0: apid 0 (application processor)
cpu1: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF
cpu1: 256KB 64b/line 8-way L2 cache
cpu2 at mainbus0: apid 18 (application processor)
cpu2: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF
cpu2: 256KB 64b/line 8-way L2 cache
cpu3 at mainbus0: apid 2 (application processor)
cpu3: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF
cpu3: 256KB 64b/line 8-way L2 cache
cpu4 at mainbus0: apid 20 (application processor)
cpu4: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF
cpu4: 256KB 64b/line 8-way L2 cache
cpu5 at mainbus0: apid 4 (application processor)
cpu5: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF
cpu5: 256KB 64b/line 8-way L2 cache
cpu6 at mainbus0: apid 22 (application processor)
cpu6: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz
cpu6:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF
cpu6: 256KB 64b/line 8-way L2 cache
cpu7 at mainbus0: apid 6 (application processor)
cpu7: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz
cpu7:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF
cpu7: 256KB 64b/line 8-way L2 cache
cpu8 at mainbus0: apid 17 (application processor)
cpu8: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz
cpu8:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF
cpu8: 256KB 64b/line 8-way L2 cache
cpu9 at mainbus0: apid 1 (application processor)
cpu9: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz
cpu9:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF
cpu9: 256KB 64b/line 8-way L2 cache
cpu10 at mainbus0: apid 19 (application processor)
cpu10: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz
cpu10:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF
cpu10: 256KB 64b/line 8-way L2 cache
cpu11 at mainbus0: apid 3 (application processor)
cpu11: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz
cpu11:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF
cpu11: 256KB 64b/line 8-way L2 cache
cpu12 at mainbus0: apid 21 (application processor)
cpu12: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz
cpu12:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF
cpu12: 256KB 64b/line 8-way L2 cache
cpu13 at mainbus0: apid 5 (application processor)
cpu13: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz
cpu13:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF
cpu13: 256KB 64b/line 8-way L2 cache
cpu14 at mainbus0: apid 23 (application processor)
cpu14: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz
cpu14:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF
cpu14: 256KB 64b/line 8-way L2 cache
cpu15 at mainbus0: apid 7 (application processor)
cpu15: Intel(R) Xeon(R) CPU X5570 @ 2.93GHz, 2926.00 MHz
cpu15:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,NXE,LONG,LAHF
cpu15: 256KB 64b/line 8-way L2 cache
ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 20, 24 pins
ioapic1 at mainbus0: apid 1 pa 0xfec80000, version 20, 24 pins
ioapic1: misconfigured as apic 0, remapped to apid 1
acpihpet0 at acpi0: 14318179 Hz
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (PEX1)
acpiprt2 at acpi0: bus 2 (PEX3)
acpiprt3 at acpi0: bus 3 (PEX4)
acpiprt4 at acpi0: bus 4 (PEX5)
acpiprt5 at acpi0: bus 5 (PEX6)
acpiprt6 at acpi0: bus 6 (PEX7)
acpiprt7 at acpi0: bus 7 (PEX9)
acpiprt8 at acpi0: bus -1 (PEXA)
acpiprt9 at acpi0: bus -1 (SBEX)
acpiprt10 at acpi0: bus 8 (COMP)
acpicpu0 at acpi0: C3, C1
acpicpu1 at acpi0: C3, C1
acpicpu2 at acpi0: C3, C1
acpicpu3 at acpi0: C3, C1
acpicpu4 at acpi0: C3, C1
acpicpu5 at acpi0: C3, C1
acpicpu6 at acpi0: C3, C1
acpicpu7 at acpi0: C3, C1
acpicpu8 at acpi0: C3, C1
acpicpu9 at acpi0: C3, C1
acpicpu10 at acpi0: C3, C1
acpicpu11 at acpi0: C3, C1
acpicpu12 at acpi0: C3, C1
acpicpu13 at acpi0: C3, C1
acpicpu14 at acpi0: C3, C1
acpicpu15 at acpi0: C3, C1
ipmi at mainbus0 not configured
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 5520 Host" rev 0x13
ppb0 at pci0 dev 1 function 0 "Intel X58 PCIE" rev 0x13
pci1 at ppb0 bus 1
bnx0 at pci1 dev 0 function 0 "Broadcom BCM5709" rev 0x20: apic 1 int 4
bnx1 at pci1 dev 0 function 1 "Broadcom BCM5709" rev 0x20: apic 1 int 16
ppb1 at pci0 dev 3 function 0 "Intel X58 PCIE" rev 0x13
pci2 at ppb1 bus 2
bnx2 at pci2 dev 0 function 0 "Broadcom BCM5709" rev 0x20: apic 1 int 0
bnx3 at pci2 dev 0 function 1 "Broadcom BCM5709" rev 0x20: apic 1 int 10
ppb2 at pci0 dev 4 function 0 "Intel X58 PCIE" rev 0x13
pci3 at ppb2 bus 3
mpi0 at pci3 dev 0 function 0 "Symbios Logic SAS1068E" rev 0x08: msi
scsibus0 at mpi0: 112 targets
sd0 at scsibus0 targ 0 lun 0: <Dell, VIRTUAL DISK, 1028> SCSI3
0/direct fixed naa.600508e0000000001e5b8ac0d042c509
sd0: 237952MB, 512 bytes/sector, 487325696 sectors
ses0 at scsibus0 targ 8 lun 0: <DP, BACKPLANE, 1.07> SCSI3
13/enclosure services fixed t10.DP_BACKPLANE000000
ppb3 at pci0 dev 5 function 0 "Intel X58 PCIE" rev 0x13: msi
pci4 at ppb3 bus 4
ppb4 at pci0 dev 6 function 0 "Intel X58 PCIE" rev 0x13: msi
pci5 at ppb4 bus 5
ix0 at pci5 dev 0 function 0 "Intel 10GbE SR Dual (82598AF)" rev 0x01:
msi, address 00:1b:21:3f:f0:e1
ix1 at pci5 dev 0 function 1 "Intel 10GbE SR Dual (82598AF)" rev 0x01:
msi, address 00:1b:21:3f:f0:e0
ppb5 at pci0 dev 7 function 0 "Intel X58 PCIE" rev 0x13: msi
pci6 at ppb5 bus 6
ix2 at pci6 dev 0 function 0 "Intel 10GbE SR Dual (82598AF)" rev 0x01:
msi, address 00:1b:21:0e:89:53
ix3 at pci6 dev 0 function 1 "Intel 10GbE SR Dual (82598AF)" rev 0x01:
msi, address 00:1b:21:0e:89:52
ppb6 at pci0 dev 9 function 0 "Intel X58 PCIE" rev 0x13: msi
pci7 at ppb6 bus 7
ix4 at pci7 dev 0 function 0 "Intel 10GbE LR (82598EB)" rev 0x01: msi,
address 00:1b:21:3f:b5:ed
"Intel X58 Misc" rev 0x13 at pci0 dev 20 function 0 not configured
"Intel X58 GPIO" rev 0x13 at pci0 dev 20 function 1 not configured
"Intel X58 RAS" rev 0x13 at pci0 dev 20 function 2 not configured
uhci0 at pci0 dev 26 function 0 "Intel 82801I USB" rev 0x02: apic 0 int 17
uhci1 at pci0 dev 26 function 1 "Intel 82801I USB" rev 0x02: apic 0 int 18
ehci0 at pci0 dev 26 function 7 "Intel 82801I USB" rev 0x02: apic 0 int 19
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
uhci2 at pci0 dev 29 function 0 "Intel 82801I USB" rev 0x02: apic 0 int 21
uhci3 at pci0 dev 29 function 1 "Intel 82801I USB" rev 0x02: apic 0 int 20
ehci1 at pci0 dev 29 function 7 "Intel 82801I USB" rev 0x02: apic 0 int 21
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb7 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0x92
pci8 at ppb7 bus 8
vga1 at pci8 dev 3 function 0 "Matrox MGA G200eW" rev 0x0a
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
pcib0 at pci0 dev 31 function 0 "Intel 82801IB LPC" rev 0x02
pciide0 at pci0 dev 31 function 2 "Intel 82801I SATA" rev 0x02: DMA,
channel 0 configured to native-PCI, channel 1 configured to native-PCI
pciide0: using apic 0 int 23 for native-PCI interrupt
atapiscsi0 at pciide0 channel 0 drive 0
scsibus1 at atapiscsi0: 2 targets
cd0 at scsibus1 targ 0 lun 0: <TEAC, DVD-ROM DV28SV, D.0K> ATAPI
5/cdrom removable
cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 5
usb2 at uhci0: USB revision 1.0
uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb3 at uhci1: USB revision 1.0
uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb4 at uhci2: USB revision 1.0
uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb5 at uhci3: USB revision 1.0
uhub5 at usb5 "Intel UHCI root hub" rev 1.00/1.00 addr 1
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
mtrr: Pentium Pro MTRR support
uhub6 at uhub0 port 3 "Standard Microsystems product 0x2514" rev
2.00/0.00 addr 2
uhidev0 at uhub4 port 1 configuration 1 interface 0 "Avocent Dell
03R874" rev 1.10/1.00 addr 2
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 modifier keys, 6 key codes, country code 33
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
uhidev1 at uhub4 port 1 configuration 1 interface 1 "Avocent Dell
03R874" rev 1.10/1.00 addr 2
uhidev1: iclass 3/1, 3 report ids
ums0 at uhidev1 reportid 1: 5 buttons, Z dir
wsmouse0 at ums0 mux 0
uhid0 at uhidev1 reportid 2: input=2, output=0, feature=0
uhid1 at uhidev1 reportid 3: input=1, output=0, feature=0
uhidev2 at uhub4 port 2 configuration 1 interface 0 "Avocent USB
Composite Device-0" rev 1.10/0.00 addr 3
uhidev2: iclass 3/1
ukbd1 at uhidev2: 8 modifier keys, 6 key codes
wskbd2 at ukbd1 mux 1
wskbd2: connecting to wsdisplay0
uhidev3 at uhub4 port 2 configuration 1 interface 1 "Avocent USB
Composite Device-0" rev 1.10/0.00 addr 3
uhidev3: iclass 3/1
ums1 at uhidev3: 3 buttons, Z dir
wsmouse1 at ums1 mux 0
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (91d0defb08274310.a) swap on sd0b dump on sd0b
bnx0: address 00:26:b9:36:ef:39
brgphy0 at bnx0 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8
bnx1: address 00:26:b9:36:ef:3b
brgphy1 at bnx1 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8
bnx2: address 00:26:b9:36:ef:3d
brgphy2 at bnx2 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8
bnx3: address 00:26:b9:36:ef:3f
brgphy3 at bnx3 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8
carp2: state transition: BACKUP -> MASTER
carp3: state transition: BACKUP -> MASTER
# vmstat -i
interrupt total rate
irq0/clock 134133416 1599
irq0/ipi 1159252 13
irq112/bnx0 1 0
irq113/bnx1 1 0
irq115/bnx3 1 0
irq96/mpi0 59182 0
irq118/ix2 6935983 82
irq119/ix3 6936142 82
irq103/ehci0 1007 0
irq104/uhci2 4645 0
irq104/ehci1 3 0
Total 149229633 1779
# top (normal)
load averages: 0.14, 0.17, 0.15 xge3.nccs.nasa.gov 14:54:52
22 processes: 21 idle, 1 on processor
All CPUs: 0.0% user, 0.0% nice, 0.0% system, 0.1% interrupt, 99.9% idle
Memory: Real: 13M/250M act/tot Free: 11G Cache: 187M Swap: 0K/0K
PID USERNAME PRI NICE SIZE RES STATE WAIT TIME CPU COMMAND
30947 _relayd 2 0 1432K 2372K idle kqread 39:04 0.00% relayd
31379 _relayd 2 0 1432K 2360K idle kqread 39:04 0.00% relayd
3934 _relayd 2 0 1436K 2372K idle kqread 22:38 0.00% relayd
31328 _relayd 2 0 968K 2556K sleep/10 kqread 0:35 0.00% relayd
8356 root 2 0 1548K 3132K sleep/7 kqread 0:19 0.00% relayd
10458 _relayd 2 0 1436K 2384K idle kqread 0:14 0.00% relayd
8338 _relayd 2 0 1436K 2380K idle kqread 0:14 0.00% relayd
580 root 28 0 520K 1320K onproc/10 - 0:08 0.00% top
6742 _pflogd 4 0 760K 440K sleep/7 bpf 0:02 0.00% pflogd
28690 _relayd 2 0 1404K 2848K sleep/7 kqread 0:01 0.00% relayd
16084 root 10 0 1120K 1940K sleep/7 wait 0:00 0.00% bash
23373 _ntp 2 0 700K 1100K idle poll 0:00 0.00% ntpd
28623 root 2 0 696K 504K idle netio 0:00 0.00% pflogd
1 root 10 0 560K 428K idle wait 0:00 0.00% init
27220 root 2 0 308K 804K idle select 0:00 0.00% inetd
6638 root 3 0 396K 912K idle ttyin 0:00 0.00% getty
6520 root 3 0 384K 916K idle ttyin 0:00 0.00% getty
21147 root 3 0 392K 920K idle ttyin 0:00 0.00% getty
# top (during network delay)
load averages: 0.43, 0.23, 0.17 xge3.nccs.nasa.gov 14:55:05
22 processes: 21 idle, 1 on processor
All CPUs: 0.0% user, 0.0% nice, 0.0% system, 0.1% interrupt, 99.9% idle
Memory: Real: 13M/250M act/tot Free: 11G Cache: 187M Swap: 0K/0K
PID USERNAME PRI NICE SIZE RES STATE WAIT TIME CPU COMMAND
30947 _relayd 2 0 1432K 2372K idle kqread 39:04 0.00% relayd
31379 _relayd 2 0 1432K 2360K idle kqread 39:04 0.00% relayd
3934 _relayd 2 0 1436K 2372K idle kqread 22:38 0.00% relayd
31328 _relayd 2 0 968K 2556K sleep/10 kqread 0:35 0.00% relayd
8356 root 2 0 1548K 3132K sleep/7 kqread 0:19 0.00% relayd
10458 _relayd 2 0 1436K 2384K idle kqread 0:14 0.00% relayd
8338 _relayd 2 0 1436K 2380K idle kqread 0:14 0.00% relayd
6742 _pflogd 4 0 760K 440K sleep/7 bpf 0:02 0.00% pflogd
10300 root 35 0 384K 1304K onproc/10 - 0:02 0.00% top
28690 _relayd 2 0 1404K 2848K sleep/7 kqread 0:01 0.00% relayd
16084 root 10 0 1120K 1940K sleep/8 wait 0:00 0.00% bash
23373 _ntp 2 0 700K 1100K idle poll 0:00 0.00% ntpd
28623 root 2 0 696K 504K idle netio 0:00 0.00% pflogd
1 root 10 0 560K 428K idle wait 0:00 0.00% init
27220 root 2 0 308K 804K idle select 0:00 0.00% inetd
6638 root 3 0 396K 912K idle ttyin 0:00 0.00% getty
6520 root 3 0 384K 916K idle ttyin 0:00 0.00% getty
21147 root 3 0 392K 920K idle ttyin 0:00 0.00% getty
