I've got a Nehalem single-package box that I'm setting up as a router. It's running Linux 2.6.30, and has version 1.3.16-k2 of the igb driver. Hyperthreading is turned on, so it presents 8 CPUs in 4 cores.
The box has four e1000e ports on the motherboard, and two igb ports on an add-in card. When we route traffic between one igb port and one e1000e port, the box works, but when we route the traffic between both igb ports, it begins to lose packets at an alarming rate, and anything sending a lot of data via TCP becomes completely unusable, such as loading large Web pages. The traffic isn't all that big, no more than 80000 packets per second incoming on one interface, maybe 30 Mbps. My first suspect was SMP affinity, but trying the 20091017 version of irqbalance made no difference. I also tried setting the affinities manually, in a couple of different patterns, also with no effect. I found this app note at Intel's site: http://download.intel.com/design/network/applnots/319935.pdf Which warns that irqbalance might not work, and has this intriguing text: > For IP forwarding, a transmit/receive queue pair should use the > same processor core and reduce any cache synchronization between > different cores. This can be performed by assigning transmit and > receive interrupts to specific cores. But I'm not sure how to interpret the words "transmit/receive queue pair". Are they talking about queue pairs on the same card, such as eth4-rx-0 and eth4-tx-0, or on different cards, for the expected path of the traffic (i.e. if traffic is coming in on eth4 and going out eth5, should I map eth4-rx-0 and eth5-tx-0 to the same CPU)? Also, is Hyperthreading an issue, should I pretend there are only 4 CPUs when setting up affinities or maybe turn Hyperthreading off? /proc/interrupts shows only four queues each for rx and tx regardless of the state of Hyperthreading. Any other advice on how to get IP forwarding working with two igb cards would be much appreciated. Thanks, -- Ed ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel
