On 11/20/2012 04:18 PM, Ben Greear wrote: > On 11/20/2012 03:59 PM, Alexander Duyck wrote: >> On 11/20/2012 02:56 PM, Ben Greear wrote: >>> We are trying out some new hardware, but it's not able to go above about >>> 4Gbps in each direction >>> (using modified pktgen). The two ixgbe ports are cabled to each other. >>> >>> Ethernet controller: >>> >>> 03:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ >>> Network Connection (rev 01) >>> 03:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ >>> Network Connection (rev 01) >>> >>> CPU: >>> >>> Intel(R) Xeon(R) CPU E5-2643 0 @ 3.30GHz >>> >>> dmesg shows it is using 5GT/s: >>> >>> ixgbe 0000:03:00.0: (PCI Express:5.0GT/s:Width x8) 00:e0:ed:1e:d4:dc >>> >>> >>> On similar setup (and same kernel) with i7 or e3 processor systems it runs >>> right at 10Gbps bi-directional. >>> >>> Is the E5-2643 just not that fast of a processor, or should I expect >>> better performance? >>> >>> Thanks, >>> Ben >>> >> >> What is the driver and kernel you are testing this with? > > kernel 3.5.7+, standard in-kernel driver. > >> Also, have you checked to make sure the feature set is comparable? For >> instance the E5 can support VT-d. If that is enabled it can have a >> negative impact on I/O performance due to extra locking overhead for >> map/unmap calls on the host. > > I'll go poke around the BIOS and disable the VT-d if I can find it.
Wow, disabling VT-d gives a big improvement! It now runs around 9.3Gbps bi-directional. Still not as good as our E3 or i7 systems, but it's at least closer. Here's the new perf top Samples: 24K of event 'cycles', Event count (approx.): 15591201274 10.61% [ixgbe] [k] ixgbe_poll 6.76% [pktgen] [k] pktgen_thread_worker 6.40% [kernel] [k] timekeeping_get_ns 5.46% [ixgbe] [k] ixgbe_xmit_frame_ring 4.20% libc-2.15.so [.] __memcpy_ssse3_back 3.98% [kernel] [k] do_raw_spin_unlock 3.02% [kernel] [k] skb_copy_bits 2.99% [kernel] [k] build_skb 2.61% perf-2510.map [.] 0x00007f73b2a28476 2.56% [kernel] [k] __netif_receive_skb What CPU(s) do you suggest for high network bandwidth..hopefully pci-e gen3 systems that can push beyond 2 10G NICs at full speed? Thanks, Ben -- Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com ------------------------------------------------------------------------------ Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
