Re: ipsec impact on performance

On (12/08/15 12:32), Steffen Klassert wrote: > > Would be nice if you could share the results. Comments are Sure, not a problem. Give me some time though, I'm also looking into the skb_cow_data and other memory-management issues that were flagged on this thread. I'll have all this info by

Re: ipsec impact on performance

On Mon, Dec 07, 2015 at 06:27:48AM -0500, Sowmini Varadhan wrote: > On (12/07/15 09:40), Steffen Klassert wrote: > > > > I've pushed it to > > > > https://git.kernel.org/cgit/linux/kernel/git/klassert/linux-stk.git/log/?h=net-next-ipsec-offload > > > > It is just example code, nothing that I

Re: ipsec impact on performance

On Thu, Dec 03, 2015 at 06:38:20AM -0500, Sowmini Varadhan wrote: > On (12/03/15 09:45), Steffen Klassert wrote: > > pcrypt(echainiv(authenc(hmac(sha1-ssse3),cbc-aes-aesni))) > > > > Result: > > > > iperf -c 10.0.0.12 -t 60 > > > >

Re: ipsec impact on performance

On (12/07/15 09:40), Steffen Klassert wrote: > > I've pushed it to > > https://git.kernel.org/cgit/linux/kernel/git/klassert/linux-stk.git/log/?h=net-next-ipsec-offload > > It is just example code, nothing that I would show usually. > But you asked for it, so here is it :) that's fine, I dont

Re: ipsec impact on performance

This article is old (turn of the century) but it may have numbers worth comparing to http://www.freeswan.org/freeswan_trees/CURRENT-TREE/doc/performance.html -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo

Re: ipsec impact on performance

On Wed, Dec 02, 2015 at 07:05:38AM -0500, Sowmini Varadhan wrote: > On (12/02/15 07:53), Steffen Klassert wrote: > > > > I'm currently working on a GRO/GSO codepath for IPsec too. The GRO part > > works already. I decapsulate/decrypt the packets on layer2 with a esp GRO > > callback function and

Re: ipsec impact on performance

On (12/03/15 09:45), Steffen Klassert wrote: > pcrypt(echainiv(authenc(hmac(sha1-ssse3),cbc-aes-aesni))) > > Result: > > iperf -c 10.0.0.12 -t 60 > > Client connecting to 10.0.0.12, TCP port 5001 > TCP window size: 45.0 KByte (default)

Re: ipsec impact on performance

On Thu, Dec 03, 2015 at 06:38:20AM -0500, Sowmini Varadhan wrote: > On (12/03/15 09:45), Steffen Klassert wrote: > > pcrypt(echainiv(authenc(hmac(sha1-ssse3),cbc-aes-aesni))) > > > > Result: > > > > iperf -c 10.0.0.12 -t 60 > > > >

Re: ipsec impact on performance

From: Sowmini Varadhan Date: Tue, 1 Dec 2015 12:59:53 -0500 > I instrumented iperf with and without ipsec, just using esp-null, > and 1 thread, to keep things simple. I'm seeing some pretty dismal > performance numbers with ipsec, and trying to think of ways to >

Re: ipsec impact on performance

On Thu, 2015-12-03 at 14:33 -0500, David Miller wrote: > From: Sowmini Varadhan > Date: Tue, 1 Dec 2015 12:59:53 -0500 > > > I instrumented iperf with and without ipsec, just using esp-null, > > and 1 thread, to keep things simple. I'm seeing some pretty dismal > >

Re: ipsec impact on performance

On (12/03/15 14:33), David Miller wrote: > > Doesn't skb_cow_data() contribute significantly to the ESP base cost, > especially for TCP packets? Indeed. For esp-null, it's about half of the total time spent in esp_output (for one run that I just instrumented with perf tracepoints 2.5 ms

Re: ipsec impact on performance

On (12/02/15 12:41), David Laight wrote: > You are getting 0.7 Gbps with ass-ccm-a-128, scale the esp-null back to > that and it would use 7/18*71 = 27% of the cpu. > So 69% of the cpu in the a-128 case is probably caused by the > encryption itself. > Even if the rest of the code cost nothing

Re: ipsec impact on performance

On Wed, Dec 2, 2015 at 1:12 PM, Sowmini Varadhan wrote: > On (12/02/15 13:07), Tom Herbert wrote: >> That's easy enough to add to flow dissector, but is SPI really >> intended to be used an L4 entropy value? We would need to consider the > > yes. To quote

Re: ipsec impact on performance

On (12/02/15 13:07), Tom Herbert wrote: > That's easy enough to add to flow dissector, but is SPI really > intended to be used an L4 entropy value? We would need to consider the yes. To quote https://en.wikipedia.org/wiki/Security_Parameter_Index "This works like port numbers in TCP and UDP

Re: ipsec impact on performance

On (12/02/15 14:01), Tom Herbert wrote: > No, please don't persist is this myopic "we'll get to IPv6 later" > model! IPv6 is a real protocol, it has significant deployment of the > Internet, and there are now whole data centers that are IPv6 only > (e.g. FB), and there are plenty of use cases of

Re: ipsec impact on performance

On Wed, 2015-12-02 at 16:12 -0500, Sowmini Varadhan wrote: > IPv6 would be an interesting academic exercise Really, you made my day ! -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: ipsec impact on performance

On Wed, Dec 2, 2015 at 12:50 PM, Sowmini Varadhan wrote: > On (12/02/15 12:41), David Laight wrote: >> You are getting 0.7 Gbps with ass-ccm-a-128, scale the esp-null back to >> that and it would use 7/18*71 = 27% of the cpu. >> So 69% of the cpu in the a-128 case is

Re: ipsec impact on performance

On (12/02/15 13:44), Tom Herbert wrote: > > IPv6 would be an interesting academic exercise, but it's going > > to be a while before we get RDS-TCP to go over IPv6. > > > Huh? Who said anything about RDS-TCP? I thought you were trying to > improve IPsec performance... yes, and it would be nice to

Re: ipsec impact on performance

On Wed, Dec 2, 2015 at 1:47 PM, Sowmini Varadhan wrote: > On (12/02/15 13:44), Tom Herbert wrote: >> > IPv6 would be an interesting academic exercise, but it's going >> > to be a while before we get RDS-TCP to go over IPv6. >> > >> Huh? Who said anything about

RE: ipsec impact on performance

From: Sowmini Varadhan > Sent: 01 December 2015 18:37 ... > I was using esp-null merely to not have the crypto itself perturb > the numbers (i.e., just focus on the s/w overhead for now), but here > are the numbers for the stock linux kernel stack > Gbps peak cpu util > esp-null

Re: ipsec impact on performance

On (12/02/15 11:56), David Laight wrote: > > Gbps peak cpu util > > esp-null 1.8 71% > > aes-gcm-c-2561.6 79% > > aes-ccm-a-1280.7 96% > > > > That trend made me think that if we can get esp-null to be as close > > as possible to GSO/GRO, the rest will

Re: ipsec impact on performance

On (12/02/15 07:53), Steffen Klassert wrote: > > I'm currently working on a GRO/GSO codepath for IPsec too. The GRO part > works already. I decapsulate/decrypt the packets on layer2 with a esp GRO > callback function and reinject them into napi_gro_receive(). So in case > the decapsulated packet

RE: ipsec impact on performance

From: Sowmini Varadhan > Sent: 02 December 2015 12:12 > On (12/02/15 11:56), David Laight wrote: > > > Gbps peak cpu util > > > esp-null 1.8 71% > > > aes-gcm-c-2561.6 79% > > > aes-ccm-a-1280.7 96% > > > > > > That trend made me think that if we can get

Re: ipsec impact on performance

On (12/02/15 12:41), David Laight wrote: > > Also what/how are you measuring cpu use. > I'm not sure anything on Linux gives you a truly accurate value > when processes are running for very short periods. I was using mpstat, while running iperf. Should I be using something else? or running it

Re: ipsec impact on performance

On 12/02/2015 03:56 AM, David Laight wrote: From: Sowmini Varadhan Sent: 01 December 2015 18:37 ... I was using esp-null merely to not have the crypto itself perturb the numbers (i.e., just focus on the s/w overhead for now), but here are the numbers for the stock linux kernel stack

Re: ipsec impact on performance

On 12/1/15 10:17 AM, Rick Jones wrote: On 12/01/2015 09:59 AM, Sowmini Varadhan wrote: But these are all still relatively small things - tweaking them doesnt get me significantly past the 3 Gbps limit. Any suggestions on how to make this budge (or design criticism of the patch) would be

Re: ipsec impact on performance

On (12/01/15 16:56), David Ahern wrote: > > Using iperf3 and AH with NULL algorithm between 2 peers connected by > a 10G link. > I'm using esp-null, not AH, and iperf2, which I understand is quite different from, and more aggressive than, iperf3 (though I'm not sure that it matters for this

Re: ipsec impact on performance

On 12/1/15 5:09 PM, Sowmini Varadhan wrote: The not-so-great news is that I see that just adding perf tracepoints (not even enabling them!) seems to make a small diff (3 Gbps vs 3.2 Gbps) to my numbers. Is that mere standard-deviation, or something one should be aware of, about perf? existence

Re: ipsec impact on performance

On Tue, Dec 1, 2015 at 9:59 AM, Sowmini Varadhan wrote: > > I instrumented iperf with and without ipsec, just using esp-null, > and 1 thread, to keep things simple. I'm seeing some pretty dismal > performance numbers with ipsec, and trying to think of ways to >

Re: ipsec impact on performance

On 12/01/2015 09:59 AM, Sowmini Varadhan wrote: But these are all still relatively small things - tweaking them doesnt get me significantly past the 3 Gbps limit. Any suggestions on how to make this budge (or design criticism of the patch) would be welcome. What do the perf profiles show?

Re: ipsec impact on performance

On Tue, Dec 01, 2015 at 12:59:53PM -0500, Sowmini Varadhan wrote: > > I instrumented iperf with and without ipsec, just using esp-null, > and 1 thread, to keep things simple. I'm seeing some pretty dismal > performance numbers with ipsec, and trying to think of ways to > improve this. Here are

Re: ipsec impact on performance

On (12/01/15 10:18), Tom Herbert wrote: > > 8.5-9.5 Gbps clear traffic, TSO disabled, so GSO, GRO is in effect > > 3-4 Gbps clear traffic, with both TSO/GSO disabled > > 1.8-2 Gbps for esp-null. > > Are you losing checksum offload also? I tried with both checksum offload on and

Re: ipsec impact on performance

On (12/01/15 10:17), Rick Jones wrote: > > What do the perf profiles show? Presumably, loss of TSO/GSO means > an increase in the per-packet costs, but if the ipsec path > significantly increases the per-byte costs... For ESP-null, there's actually very little work to do - we just need to add

Re: ipsec impact on performance

On 12/01/2015 10:45 AM, Sowmini Varadhan wrote: On (12/01/15 10:17), Rick Jones wrote: What do the perf profiles show? Presumably, loss of TSO/GSO means an increase in the per-packet costs, but if the ipsec path significantly increases the per-byte costs... For ESP-null, there's actually

Re: ipsec impact on performance

On (12/01/15 10:50), Rick Jones wrote: > > Something of a longshot, but are you sure you are still getting > effective CKO/GRO on the receiver? Good question. With ipsec, GRO (like GSO) gets implicitly disabled. But when I explictly disable GRO on receiver, leaving only GSO on sender, I can