Would 2 TX and 2 RX queues help head-of-line blocking? Would those ACKs actually end up on other queues than the data, and therefore be served quicker?
Cheers, Anna ________________________________________ From: Waskiewicz Jr, Peter P [[email protected]] Sent: 28 June 2012 10:03 To: Fischer, Anna Cc: [email protected] Subject: Re: [E1000-devel] 82599 receive performance On Thu, 2012-06-28 at 08:55 +0000, Anna Fischer wrote: > Hi, I have 2 VCPUs assigned to the VM. I thought more CPUs wouldn't > really help because a single TCP connection doesn't span across cores > anyway? I would have to run multiple connections simultaneously I > think. Do you see 2 Tx and 2 Rx queues in ethtool -S ethX inside your VM or no? The issue you can run into is head-of-line blocking of ACKs being sent if you're using a single Tx and Rx queue. TCP connections can't have their ACKs sent or be received and processed since there's pending data packets needing to be serviced. -PJ > I use the standard packet size, I don't do jumbo frames or anything. I > don't tweak the OS either, I just want to see how it would perform by > default. > > I'm just surprised that CPU isn't maxed out at all. It must be because > polling/NAPI is too slow fetching packets? It must be a timer thing > like this? > Otherwise, is it that hardware resources are limited with just a > single queue and the RX ring buffer cannot be filled/refilled quickly > enough? Why else would it be slow if CPU resources are there. > > When I say maxed out I mean the VCPU of course, so those cores are not > running at full capability. > > Anna > > > ______________________________________________________________________ > From: "Waskiewicz Jr, Peter P" <[email protected]> > Date: Thu, 28 Jun 2012 08:45:23 +0000 > To: Fischer, Anna<[email protected]> > Cc: > [email protected]<[email protected]> > Subject: Re: [E1000-devel] 82599 receive performance > > > Hi Anna, > > On Thu, 2012-06-28 at 08:32 +0000, Fischer, Anna wrote: > > Hi, I'm running a 10G 82599 controller in PCI SR-IOV mode, and the > VF is assigned to a VM (PCI pass-through). I only enable a single VF > (max_vfs=1). On my machine I run KVM and just a single VM. > > The machine has 4 Xeon X5550 cores. > > > > I've tested RX and TX throughput using iperf (TCP). Basically I want > to see how fast I can send and receive packets to/from the VM. > > How many CPU's are provisioned to your VM? Also, what packet size are > you using with iperf? In many cases, a single Rx/Tx queue pair is not > enough to run 10GbE line rate. > > -PJ > > > TX performance comes to about 8.5Gbps. At that point, CPU is maxed > out. > > > > RX performance only comes to about 5.5Gbps. CPU isn't maxed out at > all, and I wonder, if this is the expected RX throughput? I have to > say that I only run a single TX queue and a single RX queue. (I am > testing against something else that also just runs a single queue, so > I need to keep it fair.) I haven't tuned any OS parameters, I run > Fedora Core 16 and a 3.4 Linux kernel. > > > > Thanks, > > Anna > > > > > ------------------------------------------------------------------------------ > > Live Security Virtual Conference > > Exclusive live event will cover all the ways today's security and > > threat landscape has changed and how IT managers can respond. > Discussions > > will include endpoint security, mobile security and the latest in > malware > > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > > _______________________________________________ > > E1000-devel mailing list > > [email protected] > > https://lists.sourceforge.net/lists/listinfo/e1000-devel > > To learn more about Intel® Ethernet, visit > http://communities.intel.com/community/wired > ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
