Am 21.09.2010 00:43, schrieb Stuart Henderson:
> On 2010-09-20, Andre Keller <a...@list.ak.cx> wrote:
>
>> I have some odd packet loss on a openbsd based router (running -current
>> as of the beginning of september....) .
>>
>> The router has 6 physical interfaces (all em, Intel 82575EB), 4 of them
>> have traffic (about 10-20 Mbps).
>>
>>
>> We did some tuning (mostly with informations from:
>> https://calomel.org/network_performance.html) and could improve the
>> performance:
>>
> grr, that page again.
>
> "As a very general rule, using the on-board network card is going
> to be much slower than an add in PCI card"
>
> "A gigabit network controller built on board using the CPU will
> slow the entire system down. More than likely the system will not
> even be able to sustain 100MB speeds while also pegging the CPU at
> 100%."
>
> and people still use it for kernel tuning advice?
>
As we didn't find any other advices out there we thought it might be
worth giving it a try
>
>> Currently we use the following sysctl tweaks:
>> sysctl kern.maxclusters=122880
>>
> how much?!!
>
yes this might be a bit to much:
[r...@rt01-rc: root]# netstat
-m
9665 mbufs in use:
9642 mbufs allocated to data
14 mbufs allocated to packet headers
9 mbufs allocated to socket names and addresses
83/1970/122880 mbuf 2048 byte clusters in use (current/peak/max)
0/8/122880 mbuf 4096 byte clusters in use (current/peak/max)
0/8/122880 mbuf 8192 byte clusters in use (current/peak/max)
0/8/122880 mbuf 9216 byte clusters in use (current/peak/max)
0/8/122880 mbuf 12288 byte clusters in use (current/peak/max)
0/8/122880 mbuf 16384 byte clusters in use (current/peak/max)
0/8/122880 mbuf 65536 byte clusters in use (current/peak/max)
7288 Kbytes allocated to network (35% in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines
>> sysctl net.inet.ip.ifq.maxlen=1536
>>
> increasing this from the defaults can be useful if you see drops in
> net.inet.ip.ifq.drops, I'm surprised if you have to go that high for
> 4x10-20Mb.
>
yeah we had alot of ifq drops first and after setting this value they
are gone... I read on multiple "tuning tutorial" setting this to
256*iface count makes sense
>> sysctl net.inet.tcp.recvspace=262144
>> sysctl net.inet.tcp.sendspace=262144
>> sysctl net.inet.udp.recvspace=262144
>> sysctl net.inet.udp.sendspace=262144
>>
> the net.inet.*space values HAVE NO EFFECT on routed packets.
>
OK good to know...
>> But still we have about 1300 Ierrs per minute...
>>
>> When we run a simple ping, we can see that something is strange. Where
>> the majority of packets have a rtt of 1ms or less about every tenth
>> package shows a rtt of >250ms...
>>
> missing dmesg.
Not from the machine above but a machine with the exactly same hardware...
OpenBSD 4.8 (GENERIC.MP) #3: Wed Aug 11 19:24:59 CEST 2010
r...@scaramanga.rbnetwork.biz:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 3486973952 (3325MB)
avail mem = 3380334592 (3223MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.5 @ 0xcfedf000 (39 entries)
bios0: vendor Phoenix Technologies LTD version "1.3a" date 11/03/2009
bios0: Supermicro X7SBi
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S4 S5
acpi0: tables DSDT FACP _MAR MCFG APIC BOOT SPCR ERST HEST BERT EINJ
SLIC SSDT SSDT SSDT SSDT SSDT SSDT SSDT SSDT SSDT
acpi0: wakeup devices PXHA(S5) PEX_(S5) LAN_(S5) USB4(S5) USB5(S5)
USB7(S5) ESB2(S5) EXP1(S5) EXP5(S5) EXP6(S5) USB1(S5) USB2(S5) USB3(S5)
USB6(S5) ESB1(S5) PCIB(S5) KBC0(S1) MSE0(S1) COM1(S5) COM2(S5) PWRB(S3)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU X3220 @ 2.40GHz, 2400.43 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG
cpu0: 4MB 64b/line 16-way L2 cache
cpu0: apic clock running at 266MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Xeon(R) CPU X3220 @ 2.40GHz, 2400.09 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG
cpu1: 4MB 64b/line 16-way L2 cache
cpu2 at mainbus0: apid 2 (application processor)
cpu2: Intel(R) Xeon(R) CPU X3220 @ 2.40GHz, 2400.09 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG
cpu2: 4MB 64b/line 16-way L2 cache
cpu3 at mainbus0: apid 3 (application processor)
cpu3: Intel(R) Xeon(R) CPU X3220 @ 2.40GHz, 2400.09 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG
cpu3: 4MB 64b/line 16-way L2 cache
ioapic0 at mainbus0: apid 4 pa 0xfec00000, version 20, 24 pins
ioapic1 at mainbus0: apid 5 pa 0xfecc0000, version 20, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 2 (PXHA)
acpiprt2 at acpi0: bus 3 (PEX_)
acpiprt3 at acpi0: bus 7 (EXP1)
acpiprt4 at acpi0: bus 13 (EXP5)
acpiprt5 at acpi0: bus 15 (EXP6)
acpiprt6 at acpi0: bus 17 (PCIB)
acpicpu0 at acpi0: C3, PSS
acpicpu1 at acpi0: C3, PSS
acpicpu2 at acpi0: C3, PSS
acpicpu3 at acpi0: C3, PSS
acpibtn0 at acpi0: PWRB
acpivideo0 at acpi0: IGD0
ipmi at mainbus0 not configured
cpu0: Enhanced SpeedStep 2400 MHz: speeds: 2400, 1600 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 3200/3210 Host" rev 0x01
ppb0 at pci0 dev 1 function 0 "Intel 3200/3210 PCIE" rev 0x01: apic 4
int 16 (irq 5)
pci1 at ppb0 bus 1
ppb1 at pci1 dev 0 function 0 "Intel PCIE-PCIE" rev 0x09
pci2 at ppb1 bus 2
"Intel IOxAPIC" rev 0x09 at pci1 dev 0 function 1 not configured
ppb2 at pci0 dev 6 function 0 "Intel 3210 PCIE" rev 0x01: apic 4 int 16
(irq 5)
pci3 at ppb2 bus 3
ppb3 at pci3 dev 0 function 0 "PLX PEX 8518" rev 0xac
pci4 at ppb3 bus 4
ppb4 at pci4 dev 1 function 0 "PLX PEX 8518" rev 0xac: apic 4 int 17
(irq 10)
pci5 at ppb4 bus 5
em0 at pci5 dev 0 function 0 "Intel PRO/1000 (82576)" rev 0x01: apic 4
int 17 (irq 10), address 00:25:90:05:54:cc
em1 at pci5 dev 0 function 1 "Intel PRO/1000 (82576)" rev 0x01: apic 4
int 18 (irq 5), address 00:25:90:05:54:cd
ppb5 at pci4 dev 2 function 0 "PLX PEX 8518" rev 0xac: apic 4 int 18
(irq 11)
pci6 at ppb5 bus 6
em2 at pci6 dev 0 function 0 "Intel PRO/1000 (82576)" rev 0x01: apic 4
int 18 (irq 11), address 00:25:90:05:54:ce
em3 at pci6 dev 0 function 1 "Intel PRO/1000 (82576)" rev 0x01: apic 4
int 19 (irq 10), address 00:25:90:05:54:cf
uhci0 at pci0 dev 26 function 0 "Intel 82801I USB" rev 0x02: apic 4 int
16 (irq 5)
uhci1 at pci0 dev 26 function 1 "Intel 82801I USB" rev 0x02: apic 4 int
17 (irq 10)
uhci2 at pci0 dev 26 function 2 "Intel 82801I USB" rev 0x02: apic 4 int
18 (irq 11)
ehci0 at pci0 dev 26 function 7 "Intel 82801I USB" rev 0x02: apic 4 int
18 (irq 11)
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb6 at pci0 dev 28 function 0 "Intel 82801I PCIE" rev 0x02: apic 4 int
16 (irq 5)
pci7 at ppb6 bus 7
ppb7 at pci0 dev 28 function 4 "Intel 82801I PCIE" rev 0x02: apic 4 int
16 (irq 5)
pci8 at ppb7 bus 13
em4 at pci8 dev 0 function 0 "Intel PRO/1000MT (82573E)" rev 0x03: apic
4 int 16 (irq 5), address 00:30:48:f8:cc:e0
ppb8 at pci0 dev 28 function 5 "Intel 82801I PCIE" rev 0x02: apic 4 int
17 (irq 10)
pci9 at ppb8 bus 15
em5 at pci9 dev 0 function 0 "Intel PRO/1000MT (82573L)" rev 0x00: apic
4 int 17 (irq 10), address 00:30:48:f8:cc:e1
uhci3 at pci0 dev 29 function 0 "Intel 82801I USB" rev 0x02: apic 4 int
23 (irq 10)
uhci4 at pci0 dev 29 function 1 "Intel 82801I USB" rev 0x02: apic 4 int
22 (irq 11)
uhci5 at pci0 dev 29 function 2 "Intel 82801I USB" rev 0x02: apic 4 int
18 (irq 11)
ehci1 at pci0 dev 29 function 7 "Intel 82801I USB" rev 0x02: apic 4 int
23 (irq 10)
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb9 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0x92
pci10 at ppb9 bus 17
vga1 at pci10 dev 3 function 0 "ATI ES1000" rev 0x02
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
radeondrm0 at vga1: apic 4 int 22 (irq 11)
drm0 at radeondrm0
pciide0 at pci10 dev 4 function 0 "ITExpress IT8213F" rev 0x00: DMA
(unsupported), channel 0 wired to native-PCI, channel 1 wired to native-PCI
pciide0: using apic 4 int 23 (irq 10) for native-PCI interrupt
pciide0: channel 0 ignored (not responding; disabled or no drives?)
pciide0: channel 1 ignored (not responding; disabled or no drives?)
pcib0 at pci0 dev 31 function 0 "Intel 82801IR LPC" rev 0x02
ahci0 at pci0 dev 31 function 2 "Intel 82801I AHCI" rev 0x02: apic 4 int
17 (irq 10), AHCI 1.2
scsibus0 at ahci0: 32 targets
sd0 at scsibus0 targ 0 lun 0: <ATA, WDC WD5001AALS-0, 05.0> SCSI3
0/direct fixed
sd0: 476940MB, 512 bytes/sec, 976773168 sec total
ichiic0 at pci0 dev 31 function 3 "Intel 82801I SMBus" rev 0x02: apic 4
int 17 (irq 10)
iic0 at ichiic0
lm1 at iic0 addr 0x2d: W83627HF
wbng0 at iic0 addr 0x2f: w83793g
"eeprom" at iic0 addr 0x50 not configured
spdmem0 at iic0 addr 0x52: 2GB DDR2 SDRAM ECC PC2-6400CL5
"Intel 82801I Thermal" rev 0x02 at pci0 dev 31 function 6 not configured
usb2 at uhci0: USB revision 1.0
uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb3 at uhci1: USB revision 1.0
uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb4 at uhci2: USB revision 1.0
uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb5 at uhci3: USB revision 1.0
uhub5 at usb5 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb6 at uhci4: USB revision 1.0
uhub6 at usb6 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb7 at uhci5: USB revision 1.0
uhub7 at usb7 "Intel UHCI root hub" rev 1.00/1.00 addr 1
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
wbsio0 at isa0 port 0x2e/2: W83627HF rev 0x41
lm2 at wbsio0 port 0x290/8: W83627HF
lm1 detached
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
mtrr: Pentium Pro MTRR support
uhidev0 at uhub1 port 3 configuration 1 interface 0 "Peppercon AG
Multidevice" rev 2.00/0.01 addr 2
uhidev0: iclass 3/0
ums0 at uhidev0
ums0: X report 0x0022 not supported
uhidev1 at uhub1 port 3 configuration 1 interface 1 "Peppercon AG
Multidevice" rev 2.00/0.01 addr 2
uhidev1: iclass 3/1
ukbd0 at uhidev1: 8 modifier keys, 6 key codes
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
softraid0 at root
root on sd0a swap on sd0b dump on sd0b
> but try disabling sensor devices or i2c controllers
> (boot -c, disable <somedevice>, quit).
>
I'll try to find out what devices i could disable...
Thank you for your hints...
