> I do not see any packet drops on the NIC but I see "overruns" where the > kernel cannot handle the packet load and the buffers on the NIC overrun. > > I can see that ksoftirqd is running at 100% on the CPU, so I guess I am > running > out of CPU cycles. Yep, I agree. If the stack is being overrun then that is what is happening.
Cheers, John > -----Original Message----- > From: Anna Fischer [mailto:a.fisc...@sirrix.com] > Sent: Monday, February 29, 2016 7:16 AM > To: Ronciak, John <john.ronc...@intel.com>; e1000- > de...@lists.sourceforge.net > Subject: Re: Poor IPsec performance and high ksoftirqd load > > > > Am 26.02.2016 um 17:31 schrieb Ronciak, John: > > Hi Anna, > > > >> I have tried with multiple TCP streams as well but the performance > >> drops down even more already with two parallel streams. I use iperf3 > >> for testing on the client side, and then I have some scripts on my > >> IPSec box to monitor packet rates and bandwidth as well. > > So are you running out of CPU cycles or are packets being dropped causing > retransmitted packets? Use netstat and ethtool to look for dropped packets. > > I do not see any packet drops on the NIC but I see "overruns" where the > kernel cannot handle the packet load and the buffers on the NIC overrun. > > I can see that ksoftirqd is running at 100% on the CPU, so I guess I am > running > out of CPU cycles. > > Anna > > > > > Cheers, > > John > > > > > >> -----Original Message----- > >> From: Anna Fischer [mailto:a.fisc...@sirrix.com] > >> Sent: Friday, February 26, 2016 12:10 AM > >> To: Ronciak, John <john.ronc...@intel.com>; e1000- > >> de...@lists.sourceforge.net > >> Subject: Re: Poor IPsec performance and high ksoftirqd load > >> > >> > >> > >> Am 25.02.2016 um 20:40 schrieb Ronciak, John: > >>>> -----Original Message----- > >>>> From: Anna Fischer [mailto:a.fisc...@sirrix.com] > >>>> Sent: Thursday, February 25, 2016 10:53 AM > >>>> To: e1000-devel@lists.sourceforge.net > >>>> Subject: [E1000-devel] Poor IPsec performance and high ksoftirqd > >>>> load > >>>> > >>>> Hi, > >>>> > >>>> I'm running Linux 4.3.3 (Gentoo) and standard igb drivers. I have > >>>> trouble with the performance of IPSec on my platform. The platform > >>>> has Intel Ethernet Controller I211 (1GbE) and Intel Atom CPUs > >>>> (4-core) from the C2000 product family. > >>>> > >>>> My platform only seems to be able to do ~300Mbit/s when receiving > >>>> IPSec packets. When sending IPSec packets it can do ~600Mbit/s. The > >>>> weird thing is, the ksoftirqd handler seems to run at very high CPU > >>>> load. I don't understand where that comes from. I have traced using > >>>> kernel function tracing and I can see that it is taking a lot of > >>>> time do handle do_softirq and functions like igb_poll(). I was > >>>> wondering if it was normal that this takes up so much CPU? Also it > >>>> seems that this is > >> only the case in the following scenario: > >>>> > >>>> The platform receives IPSec packets on eth0, then decrypts them, > >>>> and routes them out via eth1 (plaintext). The packet stream is TCP. > >>>> So for each amount of packets going this way, there is an TCP ACK > >>>> packet going back the other way. E.g. plaintext incoming via eth1, > >>>> IPSec > >> encryption and TX out via eth0. > >>>> > >>>> For me this sounds like a very normal use case of IPSec, but still > >>>> the platform does not seem to be able to handle it. As my CPU load > >>>> is high, but not maxed out, the CPU does not seem to be the > bottleneck. > >>>> So my guess is it might also be the NIC. I do not have any > >>>> configuration options in my igb driver, and I already tried a few > >>>> option > >> with ethtool, but I don't get any improvements. > >>>> Does anyone have a good idea on how to track what is slowing down > >>>> the system? > >>>> > >>>> Many thanks, > >>>> Anna > >>>> > >>>> > >>> Hi Anna, > >>> > >>> So the i211 is a very low end client Ethernet device (PCIe x1 2with > >>> limited > >> buffer space). So it's nowhere near as performant as the server devices. > >> That said, I'm not sure what you are using to test this with. > >> Testing with a single TCP stream can always be problematic when > >> looking at wire bandwidth. Do things change if you start multiple > >> TCP strings? It will use more CPU as the packets need to be > >> decrypted but you should see an increase in throughput, at least > >> until you run out of CPU cycles. The single stream TCP throughput is > >> well known to be an issue as any single TCP connection can only have > >> so much data outstanding before being ACK'd. So using multiple TPC > >> connections changes that and should increase your overall throughput. > Please give that a try. > >>> > >>> Thanks. > >>> > >>> Cheers, > >>> John > >>> > >> > >> I have tried with multiple TCP streams as well but the performance > >> drops down even more already with two parallel streams. I use iperf3 > >> for testing on the client side, and then I have some scripts on my > >> IPSec box to monitor packet rates and bandwidth as well. > >> > >> Anna > > -- > Sirrix AG security technologies - http://www.sirrix.com > Dr. Anna Fischer eMail: a.fisc...@sirrix.com > Tel +49(681) 959 86-139 Mobile +49(176) 47928927 > PGP Key ID 0x96E0708B > > Vorstand: Ammar Alkassar (Vors.), Christian Stüble Vorsitzender des > Aufsichtsrates: Peter Riedel Sitz der Gesellschaft: Homburg/Saar, HRB 3857 > Amtsgericht Saarbrücken > > This message may contain confidential and/or privileged information. If you > are not the addressee, you must not use, copy, disclose or take any action > based on this message or any information herein. If you have received this > message in error, please advise the sender immediately by reply e-mail and > delete this message. ------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
