Hello Brian,
If you believe that some process interferes with your worker threads,
try and isolates some CPUs from the kernel scheduler (isolcpus on linux
kernel), and set the CPU affinity from /etc/vpp/startup.conf
(corelist-workers & main-core).
Korian
On 12/12/18 4:17 AM, Brian Dickson wrote:
Greetings, VPP folks.
I am continuing to work on my vpp + router-plugin (+FRRouting) set-up.
I have things mostly working with very large routing tables (source
from multiple BGP peers), but am having some challenges when trying to
use multi-threaded (additional worker threads) for increasing overall
VPP forwarding performance.
When using just a single thread, the BGP peers take a long time to
sync up but it is relatively stable. Forwarding performance on a 10G
NIC (i40e driver and vfio-pci selected), is pretty decent, but I am
interested in finding ways to improve performance (and getting things
to the point where I can use a 40G card also in the system). The limit
seems to be packets per second, and maxes out at about 11Mpps.
The problem is, when I try to use worker threads, I start running into
issues with rtnetlink buffers, and BGP, ICMP, ARP, etc, all become
"flaky".
My suspicion is that it has something to do with which thread(s)
handle the netlink traffic, and which thread(s) handle the TCP port
179 (BGP) traffic, which needs to go via the tap-inject path to the
kernel, and then to the BGP speaking application (FRR sub-unit "bgpd").
Is there anyone who can provide information or advice on this issue?
NB: the flakiness is in a COMPLETELY unloaded environment - no other
traffic is being handled, nothing else is consuming CPU cycles. It is
just the BGP traffic itself plus related stuff (ARP) and any
diagnostic traffic I use (ping).
Is this a case where I need to adjust the RSS to direct incoming
packets to the right subset of cores, and do I also need to direct
particular traffic (TCP 179) to the main core? Do I need to ensure
anything else, like using a separate core (and set the core afinity
with taskset -c ) for my BGP speaker?
Any suggestions or advice would be greatly appreciated.
Also, any updates on bringing netlink and router plugins into the main
vppsb tree? Building them on anything other than 18.07 just doesn't
work for me, and even on 18.07 is rather brittle, and I'm not 100%
sure about the build steps, which actually involve passing CFLAGS in
to make, which suggests something isn't quite right...
Thanks in advance,
Brian
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#134): https://lists.fd.io/g/vppsb-dev/message/134
Mute This Topic: https://lists.fd.io/mt/28734872/675702
Group Owner: vppsb-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vppsb-dev/unsub [korian.edel...@ulg.ac.be]
-=-=-=-=-=-=-=-=-=-=-=-
--
Korian Edeline
Université de Liège (ULg)
Bât. B28 Algorithmique des Grands Systèmes
Quartier Polytech 1
Allée de la Découverte 10
4000 Liège
Phone: +32 4 366 56 05
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#11585): https://lists.fd.io/g/vpp-dev/message/11585
Mute This Topic: https://lists.fd.io/mt/28735004/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
