On Tue, 18 Jan 2011 18:51:32 +0700, Claudio Jeker
<cje...@diehard.n-r-g.com> wrote:
On Mon, Jan 17, 2011 at 11:35:02PM -0500, Jason Healy wrote:
I had a few hours to play with a hardware traffic generator today, I
wanted to
try beating up my OpenBSD setup to see what kind of throughput I could
get.
For the curious, I was able to pulverize it with 64 byte packets and it
topped
out at about 165kpps. Throughput was less than physical interface speed
(about 800Mbps). For fun, I cranked the payload size up to 1500 bytes,
but I
couldn't get the box to exceed 1Gbps, even though I had several gigabit
interfaces trunked together. At first, it was a switch problem (the
switch
was sending all the traffic over a single link). However, after I
found out
my switches LACP hash algorithm I was able to spread the traffic out by
randomizing the port numbers.
I then confirmed that 4Gbps of traffic was leaving the switch to the
OpenBSD
box, but only 1Gbps was coming back. Therefore, I'm guessing that the
load-balancing algorithm for OpenBSD does not behave the same way as my
Juniper switching gear. Does anybody know the LACP hash that the trunk
interface in OpenBSD uses to load-balance the outgoing traffic? I
didn't have
time to do more than a cursory test with different port numbers and IP
addresses, so I'm not sure what I might be doing wrong, or if its even
possible to use layer 3/4 info in OpenBSD to hash the traffic. Since
I'm
using the box as a router, layer 2 hashing doesn't help me very much
since the
source MAC is always the same.
I took a peek at the source, but I'm definitely not a C hacker, so
nothing
jumped out at me for computing the hash...
165kpps is fairly low. Please add a dmesg so there is a chance to see
what
is causing this low rate. Modern HW with good nics should handle around
500kpps.
My november 21st i386.MP -current handles 1.3Mpps inbound and 1.3Mpps
outbound packet during rootkits attacks on one of our collocated costumer,
on an 80Mbps traffic, via a vlan interface. CPU is 1% idle, system still
responsive (I get to ssh-ed the machine and see systat).
Btw. trunk is using src & dest MAC addrs, a possible vlan tag and the IP
/
IPv6 src & dst addrs to build the hash. It does not use port numbers.
The function used for this is trunk_hashmbuf().
dmesg:
OpenBSD 4.8-current (GENERIC.MP) #21: Sun Nov 21 03:46:30 WIT 2010
r...@greenrouter-jkt01.mygreenlinks.net:/usr/src/sys/arch/i386/compile/GENERIC.MP
RTC BIOS diagnostic error f<fixed_disk,invalid_time>
cpu0: Intel(R) Xeon(R) CPU X3220 @ 2.40GHz ("GenuineIntel" 686-class) 2.41
GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,S
SE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM
real mem = 2142744576 (2043MB)
avail mem = 2097610752 (2000MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 03/26/07, SMBIOS rev. 2.4 @
0x7fbe4000 (43 entries)
bios0: vendor Intel Corporation version
"S3000.86B.02.00.0054.061120091710" date 06/11/2009
bios0: Intel S3000AH
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S4 S5
acpi0: tables DSDT SLIC FACP APIC WDDT HPET MCFG ASF! SSDT SSDT SSDT SSDT
SSDT HEST BERT ERST EINJ
acpi0: wakeup devices SLPB(S4) P32_(S4) UAR1(S1) PEX4(S4) PEX5(S4)
UHC1(S1) UHC2(S1) UHC3(S1) UHC4(S1) EHCI(S1) AC9M(S4) AZAL(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 266MHz
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU X3220 @ 2.40GHz ("GenuineIntel" 686-class) 2.41
GHz
cpu1:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,S
SE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM
cpu2 at mainbus0: apid 1 (application processor)
cpu2: Intel(R) Xeon(R) CPU X3220 @ 2.40GHz ("GenuineIntel" 686-class) 2.41
GHz
cpu2:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,S
SE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM
cpu3 at mainbus0: apid 3 (application processor)
cpu3: Intel(R) Xeon(R) CPU X3220 @ 2.40GHz ("GenuineIntel" 686-class) 2.41
GHz
cpu3:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,S
SE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM
ioapic0 at mainbus0: apid 5 pa 0xfec00000, version 20, 24 pins
ioapic0: misconfigured as apic 0, remapped to apid 5
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 4 (P32_)
acpiprt2 at acpi0: bus 1 (PEX0)
acpiprt3 at acpi0: bus -1 (PEX1)
acpiprt4 at acpi0: bus -1 (PEX2)
acpiprt5 at acpi0: bus -1 (PEX3)
acpiprt6 at acpi0: bus 2 (PEX4)
acpiprt7 at acpi0: bus 3 (PEX5)
acpicpu0 at acpi0: PSS
acpicpu1 at acpi0: PSS
acpicpu2 at acpi0: PSS
acpicpu3 at acpi0: PSS
acpibtn0 at acpi0: SLPB
bios0: ROM list: 0xc0000/0x9000
cpu0: Enhanced SpeedStep 2401 MHz: speeds: 2394, 1596 MHz
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "Intel E7230 Host" rev 0x00
ppb0 at pci0 dev 28 function 0 "Intel 82801GB PCIE" rev 0x01: apic 5 int
17 (irq 255)
pci1 at ppb0 bus 1
ppb1 at pci0 dev 28 function 4 "Intel 82801G PCIE" rev 0x01: apic 5 int 17
(irq 255)
pci2 at ppb1 bus 2
em0 at pci2 dev 0 function 0 "Intel PRO/1000 PT (82571EB)" rev 0x06: apic
5 int 16 (irq 9), address 00:15:17:86:51:72
em1 at pci2 dev 0 function 1 "Intel PRO/1000 PT (82571EB)" rev 0x06: apic
5 int 17 (irq 10), address 00:15:17:86:51:73
ppb2 at pci0 dev 28 function 5 "Intel 82801G PCIE" rev 0x01: apic 5 int 16
(irq 255)
pci3 at ppb2 bus 3
em2 at pci3 dev 0 function 0 "Intel PRO/1000MT (82573E)" rev 0x03: apic 5
int 17 (irq 10), address 00:15:17:49:03:b3
"Intel 82573E Serial" rev 0x03 at pci3 dev 0 function 3 not configured
"Intel 82573E KCS" rev 0x03 at pci3 dev 0 function 4 not configured
uhci0 at pci0 dev 29 function 0 "Intel 82801GB USB" rev 0x01: apic 5 int
23 (irq 11)
uhci1 at pci0 dev 29 function 1 "Intel 82801GB USB" rev 0x01: apic 5 int
19 (irq 11)
uhci2 at pci0 dev 29 function 2 "Intel 82801GB USB" rev 0x01: apic 5 int
18 (irq 11)
uhci3 at pci0 dev 29 function 3 "Intel 82801GB USB" rev 0x01: apic 5 int
16 (irq 9)
ehci0 at pci0 dev 29 function 7 "Intel 82801GB USB" rev 0x01: apic 5 int
23 (irq 11)
ehci0: timed out waiting for BIOS
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb3 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xe1
pci4 at ppb3 bus 4
skc0 at pci4 dev 0 function 0 "D-Link DGE-530T B1" rev 0x11, Yukon Lite
(0x9): apic 5 int 21 (irq 11)
sk0 at skc0 port A: address 00:1e:58:9a:99:65
eephy0 at sk0 phy 0: 88E1011 Gigabit PHY, rev. 5
skc1 at pci4 dev 1 function 0 "D-Link DGE-530T B1" rev 0x11, Yukon Lite
(0x9): apic 5 int 22 (irq 11)
sk1 at skc1 port A: address 00:1e:58:9a:98:f6
eephy1 at sk1 phy 0: 88E1011 Gigabit PHY, rev. 5
vga1 at pci4 dev 4 function 0 "ATI ES1000" rev 0x02
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
radeondrm0 at vga1: apic 5 int 18 (irq 11)
drm0 at radeondrm0
em3 at pci4 dev 5 function 0 "Intel PRO/1000MT (82541GI)" rev 0x05: apic 5
int 17 (irq 10), address 00:15:17:49:03:b4
ichpcib0 at pci0 dev 31 function 0 "Intel 82801GB LPC" rev 0x01: PM
disabled
pciide0 at pci0 dev 31 function 1 "Intel 82801GB IDE" rev 0x01: DMA,
channel 0 configured to compatibility, channel 1 configured to
compatibility
pciide0: channel 0 disabled (no drives)
pciide0: channel 1 disabled (no drives)
pciide1 at pci0 dev 31 function 2 "Intel 82801GB SATA" rev 0x01: DMA,
channel 0 configured to native-PCI, channel 1 configured to native-PCI
pciide1: using apic 5 int 19 (irq 11) for native-PCI interrupt
wd0 at pciide1 channel 0 drive 0: <ST380215AS>
wd0: 16-sector PIO, LBA48, 76319MB, 156301488 sectors
wd0(pciide1:0:0): using PIO mode 4, Ultra-DMA mode 6
ichiic0 at pci0 dev 31 function 3 "Intel 82801GB SMBus" rev 0x01: apic 5
int 19 (irq 11)
iic0 at ichiic0
adt0 at iic0 addr 0x2e: sch5027 rev 0x69
spdmem0 at iic0 addr 0x50: 1GB DDR2 SDRAM ECC PC2-5300CL5
spdmem1 at iic0 addr 0x52: 1GB DDR2 SDRAM ECC PC2-5300CL5
usb1 at uhci0: USB revision 1.0
uhub1 at usb1 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb2 at uhci1: USB revision 1.0
uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb3 at uhci2: USB revision 1.0
uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb4 at uhci3: USB revision 1.0
uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1
isa0 at ichpcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
mtrr: Pentium Pro MTRR support
vscsi0 at root
scsibus0 at vscsi0: 256 targets
softraid0 at root
root on wd0a swap on wd0b dump on wd0b
WARNING: / was not properly unmounted
Amazing beast,
Insan Praja
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
