Forgive my top post. With the new kernel you may be running into needing faster cleanup to increase tx speed. try increasing the interrupt rate via ethtool -C ethX rx-usecs 10, yes I said rx because there is only one rate control for the interrupt.
You can easily do line rate tx with 82599. The biggest limiter in tx only tests is the amount of data in flight and the time it takes to get acks back. Also please make sure you have run the set_irq_affinity script to bind interrupts to CPUs. -- Jesse Brandeburg > On Feb 17, 2014, at 5:42 PM, "Ben Greear" <gree...@candelatech.com> wrote: > >> On 02/17/2014 02:19 PM, John-Paul Robinson wrote: >> Hi, >> >> I don't know if this topic is appropriate here, please direct me to a >> better place if not. >> >> I've been spending considerable time trying to measure the performance >> of our 10G fabric that uses Intel X520 cards. The primary test machine >> has dual Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz chip 8-core chips and >> 96GB RAM. >> >> The test machine is now running Ubuntu 12.04.4 server with kernel 3.11.0 >> with latest ixgbe driver 3.19.1. >> >> Using iperf (2.0.5) I see about 9.39Gbs steady inbound transfers (there >> are a few glitches where I've seen drop to 7Gbs but it recovers). My >> outbound transfers, however, are about 8.83Gbs steady and tend to be >> more variable. >> >> This is the best performance I can get on the server. >> >> Interestingly when I boot the machine off the live CDROM image for >> Ubuntu 12.04.4 desktop, I see nice steady 9.39Gbs in both directions. >> This is the best performance i have seen with this card to-date. >> >> I've spent a lot of time with these cards and in general they have be >> very finicky, delivering inconsistent results from test to test, being >> very sensitive to driver and kernel versions. >> >> I've taken them from extremely erratic performance on Ubuntu 12.04.1 >> with the stock ixgbe 3.6.7 driver to much higher, more stable >> performance simply by updating to ixgbe 3.11.33. It would be nice to >> see a stable flatline performance at line speeds on kernel 3.11 with the >> 3.19.1 driver. >> >> I'm wondering if there is a known configuration profile that allows >> these cards to perform at line speeds or if there are known issues or >> hardware incompatibilities. >> >> I know there are a lot of subtleties to performance tuning but >> performance on other cards in our fabric (btw from Brocade) deliver very >> consistent, stable, high performance line speed results over many tests. >> >> I've been scratching my head for a while and am looking for a fresh >> perspective or deeper understanding. > > First, check 'dmesg' and make sure your NICs are using at least > x8 pci with 5GT/s. > > Check BIOS and disable 'VT-d' if it is on...it hurts performance by > 50% or so. > > Try using several (5-10) flows in iperf, maybe just use 5-10 > instances of iperf so you get good usage of your cores. > > Thanks, > Ben > > > -- > Ben Greear <gree...@candelatech.com> > Candela Technologies Inc http://www.candelatech.com > > > ------------------------------------------------------------------------------ > Managing the Performance of Cloud-Based Applications > Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. > Read the Whitepaper. > http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk > _______________________________________________ > E1000-devel mailing list > E1000-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/e1000-devel > To learn more about Intel® Ethernet, visit > http://communities.intel.com/community/wired ------------------------------------------------------------------------------ Managing the Performance of Cloud-Based Applications Take advantage of what the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. http://pubads.g.doubleclick.net/gampad/clk?id=121054471&iu=/4140/ostg.clktrk _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
