Hi Antony, Thank you for your time.
I have been referring to this page (https://libreswan.org/wiki/XFRM_pCPU) and it doesn't say that XFRM is only supported for ikev2. I am setting up a shared VTI for 500 Remote Clients IPSec (xAUTH using PAM, IKEv1) tunnels. I have attached my ipsec.conf at the bottom of this email. *What I understand from your response: Please correct me* 1. Lbreswan experimental versions only support pCPU with IKEv2. (Lod balancing one big IPSec flow over multiple vCPUs.) *Question: *For my use case (500 Clients, xAUTH using PAM, IKEv1 ) the SAs per client will be created per vCPU. - The vCPU will be picked randomly (How will the 500 SAs be distributed?) 500/6 = 82 SAs per CPU. - There shall be no duplicate SAs for a single connection over multiple vCPU because there is no pCPU XFRM. Correct? - Is there a way fro me to check how any SAs got allocated to a vCPU on my system? *My Observation: *When I start pushing traffic across all the 500 SAs - Some times the load isn't distributed evenly and I see some vCPUs geting overutilized and start slowing down the Libreswan packet processing rate. - The Libreswan server itn't able to process packets fast enough and the TAP interface (tx queue) on the KVM virtulization host starts dropping packets. Currently, my ipsec clients are using: ( Any advice?) vCPU is Intel(R) Xeon(R) Gold 6126 CPU @ 2.60GHz passthrough Host VM ike=3des-sha1-modp1024 esp=aes256-md5-modp1024 ########################### ipsec.conf ########################### config setup uniqueids=no conn %default dpdaction=clear dpddelay=30s dpdtimeout=90s ikev2=no rekey=no ikelifetime=24h lifetime=24h authby=secret leftxauthserver=yes rightxauthclient=yes xauthby=pam left=10.11.251.251 leftsubnet=0.0.0.0/0 leftid=@libreswan right=%any vti-interface=vti01 vti-routing=yes vti-shared=yes mark=5/0xffffffff replay-window=0 nic-offload=auto type=tunnel auto=add conn strswan1 rightid=STRSWANAT1 rightsubnet=10.15.0.0/30 ...... 1 through N conn strswan500 rightid=STRSWANAT500 rightsubnet=10.16.0.0/30 ########################### ipsec.conf ########################### On Thu, Sep 17, 2020 at 12:40 PM Antony Antony <[email protected]> wrote: > On Tue, Sep 15, 2020 at 11:11:30AM -0400, Rav Ya wrote: > > Hello Everyone, > > > > Please advice. Any help will be highly appreciated. Thank you in advance. > > > > *Test Setup: *Libreswan Server (Virtual Machine: KVM) > > 500 IPSec Clients (xAuth using PAM-Auth) > > can you share your libreswan config? Where did you get libreswan with > xauth > and pCPU support? > > The libreswan expirimental versions only support pCPU with IKEv2, without > CP(or xauth) payload, INTERNAL_IP options. It is meant for data center > like > envirment without NAT and without xauth. Just one fat IPsec flow, using > AES > GCM, with multiple CPU cores (not hyper threads). > > > I am running a Libreswan server in a virtual environment (VM hosted on > > KVM/oVIRT). The logical network i.e. virtio-net virtual NIC drivers > > supports Multiqueue. I have 6 vCPUs configured with 6 RX/TX Queue (1 > queue > > per vCPU). > > vCPU and XFRM bottlenecks are hard to debug. What is the host CPU? look at > cache miss(using kernel perf) and IRQ distributions using mpstat. > > > The traffic load balancing over XFRM pCPU is flaky. Initially, the load > > gets evenly distributed and after a while, only 1 (at most 2) vCPUs get > > utilized with soft IRQs and the rest go underutilized. > > > I read an article that said XFRM pCPU only supports RSS NIC and > > recently support for vmxnet3 (VMWare) got added. The KVM and virtio > > Multiqueue was listed under future ideas and worklist. > > you are likely mixing up too many things. xauth and RSS can work on its > own. > As far as I see you don"t need pCPU for with 500 clients. The idea behind > pCPU work is traffic for one SA, or a few SAs, split across multiple > CPUs. > and pCPU only work upto number CPUs. We were focused on on Intel CPUs > only, > with AESNI acceleration. As I see it, it will not work very well vCPU 6 > CPUs > and 500 clients use case. > > > Is there a way to work around this limitation? Is this support available > > on the Latest version of Kernel and Libreswan? please advise. > > Which crypto cypher is used? One tip, if it is AES GCM with AESNI > accelartion aovid hyper threading. > CPU threads may share AESNI engine, hence lowering performance on vCPU. > Check your specific CPU model. > > >
_______________________________________________ Swan mailing list [email protected] https://lists.libreswan.org/mailman/listinfo/swan
