On Wed, Mar 18, 2009 at 7:32 AM, <[email protected]> wrote:
> Hi,
>
> ok, I'm back with some tests and results.
> I read a lot about the em driver settings, and this is what I did:
> in /etc/sysctl.conf I added:
> dev.em.0.rx_processing_limit=1600
> dev.em.1.rx_processing_limit=1600
> although I also tried -1 and some smaller values.
>
> in /boot/loader.conf I added:
> hw.em.rxd="4096"
> hw.em.txd="4096"
> and I believe these took care of the errors on the interfaces I used to see.
>
> I also decided to change these in sysctl.conf:
> kern.ipc.somaxconn=1024
> net.inet.ip.intr_queue_maxlen=4096
>
> the first one was a recommendation from a freebsd documentation and the
> second one I changed even though I had net.inet.ip.intr_queue_drops = 0.
> I also tried changing net.isr.direct to "0".
>
> Now, for the important part. The "emX taskq" is back(after reboot), "swi1:
> net" is gone and while I don't have any serious load right now, I can see by
> the percentage of this process that it will hit 100% exactly around 15kpps,
> as usual. And I should remind you that this is still a different server -
> IBM x336.
Just as an FYI and comparison proving that FreeBSD 6.2 handles this
w/out blinking. Unfortunately, I don't have any boxes running 7.x at
this time.
bwm-ng v0.6 (probing every 0.500s), press 'h' for help
input: getifaddrs type: rate
- iface Rx Tx Total
==============================================================================
em0: 3801.54 P/s 4210.02 P/s 8011.56 P/s
em1: 19377.65 P/s 18855.49 P/s 38233.14 P/s
em2: 111.75 P/s 7231.21 P/s 7342.97 P/s
em3: 1.93 P/s 1.93 P/s 3.85 P/s
lo0: 0.00 P/s 0.00 P/s 0.00 P/s
last pid: 67441; load averages: 0.47, 0.53, 0.54
up 241+10:41:12 13:33:00
48 processes: 1 running, 28 sleeping, 19 zombie
CPU states: 0.8% user, 0.0% nice, 1.3% system, 11.5% interrupt, 86.5% idle
Mem: 22M Active, 1298M Inact, 243M Wired, 112M Buf, 1696M Free
Swap: 2048M Total, 2048M Free
This is on an HP DL385G5 with two dual-core Opteron 2218 cpu's and
dual port Intel PCI-e LC fiber cards (as well as a quad port copper
card...the PPS rates above are going over the fiber card). The
operating system version is FreeBSD 6.2 - the same kernel (and some of
the same patches) that pfSense 1.2.0 runs.
# sysctl net.isr
net.isr.direct: 0
net.isr.count: 1942552898
net.isr.directed: 0
net.isr.deferred: 1942552900
net.isr.queued: 31395
net.isr.drop: 0
net.isr.swi_count: 347040539
(all 4 nics running the same settings)
# sysctl dev.em.0
dev.em.0.%desc: Intel(R) PRO/1000 Network Connection Version - 6.2.9
dev.em.0.%driver: em
dev.em.0.%location: slot=0 function=0 handle=\_SB_.PCI0.EXB0.PES5
dev.em.0.%pnpinfo: vendor=0x8086 device=0x105f subvendor=0x8086
subdevice=0x125f class=0x020000
dev.em.0.%parent: pci5
dev.em.0.debug_info: -1
dev.em.0.stats: -1
dev.em.0.rx_int_delay: 0
dev.em.0.tx_int_delay: 66
dev.em.0.rx_abs_int_delay: 66
dev.em.0.tx_abs_int_delay: 66
dev.em.0.rx_processing_limit: 100
net.inet.ip.intr_queue_maxlen: 5000
net.inet.ip.intr_queue_drops: 0
--Bill
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Commercial support available - https://portal.pfsense.org
