On 02/17/2014 02:19 PM, John-Paul Robinson wrote: > Hi, > > I don't know if this topic is appropriate here, please direct me to a > better place if not. > > I've been spending considerable time trying to measure the performance > of our 10G fabric that uses Intel X520 cards. The primary test machine > has dual Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz chip 8-core chips and > 96GB RAM. > > The test machine is now running Ubuntu 12.04.4 server with kernel 3.11.0 > with latest ixgbe driver 3.19.1. > > Using iperf (2.0.5) I see about 9.39Gbs steady inbound transfers (there > are a few glitches where I've seen drop to 7Gbs but it recovers). My > outbound transfers, however, are about 8.83Gbs steady and tend to be > more variable. > > This is the best performance I can get on the server. > > Interestingly when I boot the machine off the live CDROM image for > Ubuntu 12.04.4 desktop, I see nice steady 9.39Gbs in both directions. > This is the best performance i have seen with this card to-date. > > I've spent a lot of time with these cards and in general they have be > very finicky, delivering inconsistent results from test to test, being > very sensitive to driver and kernel versions. > > I've taken them from extremely erratic performance on Ubuntu 12.04.1 > with the stock ixgbe 3.6.7 driver to much higher, more stable > performance simply by updating to ixgbe 3.11.33. It would be nice to > see a stable flatline performance at line speeds on kernel 3.11 with the > 3.19.1 driver. > > I'm wondering if there is a known configuration profile that allows > these cards to perform at line speeds or if there are known issues or > hardware incompatibilities. > > I know there are a lot of subtleties to performance tuning but > performance on other cards in our fabric (btw from Brocade) deliver very > consistent, stable, high performance line speed results over many tests. > > I've been scratching my head for a while and am looking for a fresh > perspective or deeper understanding.
First, check 'dmesg' and make sure your NICs are using at least x8 pci with 5GT/s. Check BIOS and disable 'VT-d' if it is on...it hurts performance by 50% or so. Try using several (5-10) flows in iperf, maybe just use 5-10 instances of iperf so you get good usage of your cores. Thanks, Ben -- Ben Greear <gree...@candelatech.com> Candela Technologies Inc http://www.candelatech.com ------------------------------------------------------------------------------ Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
