[vpp-dev] vpp performance numbers with 10Gbps interface.
Hi All, I was trying to measure maximum PPS handled by vpp.I have installed ubuntu 16.04 on my server.I have followed vpp recommended bios settings. Hardware specs: root@kujo:~# lscpu Architecture: x86_64 CPU op-mode(s):32-bit, 64-bit Byte Order:Little Endian CPU(s):8 On-line CPU(s) list: 0-7 Thread(s) per core:1 Core(s) per socket:8 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family:6 Model: 45 Model name:Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz Stepping: 7 CPU MHz: 1200.000 CPU max MHz: 2900. CPU min MHz: 1200. BogoMIPS: 5786.39 Virtualization:VT-x L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 20480K NUMA node0 CPU(s): 0-7 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm Vpp version: vpp# show version vpp v17.10-rc0~301-gb2d2fc7 built by root on kujo at Mon Sep 11 16:39:34 IST 2017 My vpp model has 1 main thread and 1 worker thread.I was not able to get more than 6Mpps .After 6 Mpps i can see the rx_miss counters in vpp stats. vpp# show interface Name Idx State Counter Count TenGigabitEtherneta/0/0 1down rx-error 2 TenGigabitEtherneta/0/1 2 up rx packets 52647168 rx bytes 3369416188 tx packets 52638150 tx bytes 4842700014 drops 9024 ip4 52645519 tx-error 1 local00down vpp# show interface Name Idx State Counter Count TenGigabitEtherneta/0/0 1down rx-error 2 TenGigabitEtherneta/0/1 2 up rx packets 54696192 rx bytes 3500553704 tx packets 54687170 tx bytes 5031209822 drops 9028 ip4 54694538 tx-error 1 local00down vpp# show interface Name Idx State Counter Count TenGigabitEtherneta/0/0 1down rx-error 2 TenGigabitEtherneta/0/1 2 up rx packets 56743168 rx bytes 3631560168 tx packets 56734146 tx bytes 5219531614 drops 9028 ip4 56741514 rx-miss 23152160 tx-error 1 local00down vpp# show interface Name Idx State Counter Count TenGigabitEtherneta/0/0 1down rx-error 2 TenGigabitEtherneta/0/1 2 up rx packets 58714624 rx bytes 3757733348 tx packets 58705601 tx bytes 5400905474 drops 9029 ip4 58712969 rx-miss 23152160 tx-error 1 Also output for show run is given below: vpp# show run Thread 0 vpp_main (lcore 0) Time 1049.7, average vectors/node 1.00, last 128 main loops 0.00 per node 0.00 vector rates in 0.e0, out 0.e0, drop 9.5264e-4, punt 0.e0 Name State Calls VectorsSuspends Clocks Vectors/Call TenGigabitEtherneta/0/1-output active 1 1 0 9.04e31.00 acl-plugin-fa-cleaner-process event wait0 0 1 1.07e40.00 admin-up-down-process
Re: [vpp-dev] vpp performnace measurement in sriov configuration.
Hi Folks, Gentle reminder.Can you please confirm the expected numbers in sriov configuration and please point out if something is happening wrong here. Thanks, Rahul On Wed, Aug 16, 2017 at 3:04 PM, Rahul Negi <rahulnegi...@gmail.com> wrote: > sorry for the typo in the previous mail.Complete output of the show run > command is mentioned below:- > > vpp# show run > Thread 0 vpp_main (lcore 0) > Time 393.8, average vectors/node 1.00, last 128 main loops 0.00 per node > 0.00 > vector rates in 0.e0, out 1.0158e-2, drop 2.5394e-3, punt 0.e0 > Name State Calls Vectors >Suspends Clocks Vectors/Call > VirtualFunctionEthernet0/a/0-o active 4 4 > 0 2.69e31.00 > VirtualFunctionEthernet0/a/0-t active 4 4 > 0 4.72e31.00 > acl-plugin-fa-cleaner-process event wait0 0 > 1 1.64e40.00 > admin-up-down-process event wait0 0 > 1 7.39e30.00 > api-rx-from-ringany wait 0 0 > 29 2.64e50.00 > bfd-processevent wait0 0 > 1 6.42e30.00 > cdp-process any wait 0 0 > 41 1.28e50.00 > dhcp-client-process any wait 0 0 > 4 5.04e30.00 > dpdk-ipsec-processdone 1 0 > 0 2.89e50.00 > dpdk-processany wait 0 0 > 132 6.89e40.00 > error-drop active 1 1 > 0 2.35e31.00 > fib-walkany wait 0 0 > 197 8.02e30.00 > flow-report-process any wait 0 0 > 1 7.02e20.00 > flowprobe-timer-process any wait 0 0 > 1 4.88e30.00 > ikev2-manager-process any wait 0 0 > 394 4.81e30.00 > ioam-export-process any wait 0 0 > 1 1.18e30.00 > ip4-gleanactive 1 1 > 0 1.56e41.00 > ip4-lookup active 4 4 > 0 6.53e31.00 > ip4-rewrite active 3 3 > 0 3.02e31.00 > ip6-icmp-neighbor-discovery-ev any wait 0 0 > 394 3.78e30.00 > l2fib-mac-age-scanner-process event wait0 0 > 1 1.22e30.00 > lisp-retry-service any wait 0 0 > 197 8.37e30.00 > lldp-process event wait0 0 > 1 4.15e60.00 > memif-process event wait0 0 > 1 1.74e40.00 > nat64-expire-walk done 1 0 > 0 1.01e40.00 > send-garp-na-process event wait0 0 > 1 1.29e30.00 > snat-det-expire-walk done 1 0 > 0 1.05e30.00 > startup-config-processdone 1 0 > 1 5.97e30.00 > udp-ping-processany wait 0 0 > 1 1.05e40.00 > unix-cli-sockaddr family 1 active 0 0 > 298 7.72e60.00 > unix-epoll-input polling655279 0 > 0 1.56e60.00 > vhost-user-process any wait 0 0 > 1 1.33e30.00 > vhost-user-send-interrupt-
Re: [vpp-dev] vpp performnace measurement in sriov configuration.
(lcore 1) Time 393.8, average vectors/node 17.54, last 128 main loops 0.00 per node 0.00 vector rates in 4.5190e5, out 4.5190e5, drop 3.3698e0, punt 0.e0 Name State Calls Vectors Suspends Clocks Vectors/Call VirtualFunctionEthernet0/a/0-o active 10146115 177952676 0 1.70e1 17.54 VirtualFunctionEthernet0/a/0-t active 10146115 177952676 0 7.86e1 17.54 arp-inputactive 13371340 0 2.58e31.00 dpdk-input polling6678250972 177954003 0 3.26e3 .03 error-drop active 13241327 0 8.68e21.00 ethernet-input active 13371340 0 1.87e31.00 interface-output active 16 16 0 8.06e21.00 ip4-icmp-echo-reply active 3 3 0 3.01e41.00 ip4-icmp-echo-requestactive 2 2 0 3.80e31.00 ip4-icmp-error active 10146097 177952658 0 1.28e2 17.54 ip4-icmp-input active 5 5 0 2.59e31.00 ip4-input-no-checksumactive 10146102 177952663 0 4.05e1 17.54 ip4-load-balance active 2 2 0 3.67e31.00 ip4-localactive 10146102 177952663 0 5.54e1 17.54 ip4-lookup active 20292199 355905321 0 4.37e1 17.54 ip4-rewrite active 10146099 177952660 0 3.23e1 17.54 ip4-udp-lookup active 10146097 177952658 0 3.95e1 17.54 --- Thread 2 vpp_wk_1 (lcore 2) Time 393.8, average vectors/node 0.00, last 128 main loops 0.00 per node 0.00 vector rates in 0.e0, out 0.e0, drop 0.e0, punt 0.e0 Name State Calls Vectors Suspends Clocks Vectors/Call --- Thread 3 vpp_wk_2 (lcore 3) Time 393.8, average vectors/node 0.00, last 128 main loops 0.00 per node 0.00 vector rates in 0.e0, out 0.e0, drop 0.e0, punt 0.e0 Name State Calls Vectors Suspends Clocks Vectors/Call Thanks, rahul On Wed, Aug 16, 2017 at 12:01 PM, Rahul Negi <rahulnegi...@gmail.com> wrote: > HI All, > After configuring hugepages of size 2048KiB at the host,I am not seeing > any improvement in the performance.Still after pumping more than 4Mpps i am > seeing this counter getting incremented > rx_no_dma_resources: 93356 at the host. > > output of following commands are mentioned below-: > > vpp# show interface > Name Idx State Counter > Count > VirtualFunctionEthernet0/a/0 1 up rx packets > 177953727 > rx bytes > 11389034442 > tx packets > 177952680 > tx bytes > 16371646028 > drops > 1052 > ip4 > 177952663 > local00down > > > vpp# show run > Thread 0 vpp_main (lcore 0) > Time 393.8, average vectors/node 1.00, last 128 main loops 0.00 per node > 0.00 > vector rates in 0.e0, out 1.0158e-2, drop 2.5394e-3, punt 0.e0 > Name State Calls Vectors >Suspends Clocks Vectors/Call > VirtualFunctionEthernet0/a/0-o active 4 4 > 0 2.69e31.00 > VirtualFunctionEthernet0/a/0-t active 4 4 > 0 4.72e31.00 > acl-plugin-fa-cleaner-process event wait0 0 > 1 1.64e40.00 > admin-up-down-process event wait0 0 > 1 7.39e30.00 > api-rx-from-ring
Re: [vpp-dev] vpp performnace measurement in sriov configuration.
-user-process any wait 0 0 1 1.33e30.00 vhost-user-send-interrupt-proc any wait 0 0 1 1.06e30.00 vpe-link-state-process event wait0 0 2 5.78e30.00 vpe-oam-process any wait 0 0 193 5.68e30.00 vpe-route-resolver-process any wait 0 0 4 7.33e30.00 vxlan-gpe-ioam-export-process any wait 0 0 1 2.31e30.00 --- Thread 1 vpp_wk_0 (lcore 1) Time 393.8, average vectors/node 17.54, last 128 main loops 0.00 per node 0.00 vector rates in 4.5190e5, out 4.5190e5, drop 3.3698e0, punt 0.e0 Name State Calls Vectors Suspends Clocks Vectors/Call VirtualFunctionEthernet0/a/0-o active 10146115 177952676 0 1.70e1 17.54 VirtualFunctionEthernet0/a/0-t active 10146115 177952676 0 7.86e1 17.54 arp-inputactive 13371340 0 2.58e31.00 dpdk-input polling6678250972 177954003 0 3.26e3 .03 error-drop active 13241327 0 8.68e21.00 ethernet-input active 13371340 0 1.87e31.00 interface-output active 16 16 0 8.06e21.00 ip4-icmp-echo-reply active 3 3 0 3.01e41.00 ip4-icmp-echo-requestactive 2 2 0 3.80e31.00 ip4-icmp-error active 10146097 177952658 0 1.28e2 17.54 ip4-icmp-input active 5 5 0 2.59e31.00 ip4-input-no-checksumactive 10146102 177952663 0 4.05e1 17.54 ip4-load-balance active 2 2 0 3.67e31.00 ip4-localactive 10146102 177952663 0 5.54e1 17.54 ip4-lookup active 20292199 355905321 0 4.37e1 17.54 ip4-rewrite active 10146099 177952660 0 3.23e1 17.54 ip4-udp-lookup active 10146097 177952658 0 3.95e1 17.54 Thanks, Rahul On Fri, Aug 11, 2017 at 1:45 AM, Kinsella, Ray <ray.kinse...@intel.com> wrote: > Hi Rahul, > > So there a few additional hoops you have to jump through to get good > performance. The biggest of which is making sure that > > 1. The host is setup to make hugepages available. > 2. QEMU is pointed at the hugepages. > 3. The guest is setup to make hugepages available. > > The following documented from DPDK covers it in some detail. > > http://dpdk.readthedocs.io/en/v16.04/sample_app_ug/vhost.html > > Ray K > > > On 09/08/2017 23:33, Rahul Negi wrote: > >> Hi All, >> I am working on an use case where i want to measure the Maximum PPS >> handled by vpp in sriov configuration.I have created a virtual machine >> on a host having specifications as follows:- >> 1.RHEL 7.3 installed >> 2.Intel X540 10 gig NIC attached >> >> I have created a virtual function from one of the interface of 10 gig >> NIC(for e.g ens3f0) and attached it to my Virtual Machine.I have >> installed ubuntu 16.04 on my Vm. My vpp version running on vm is 17.10. >> >> vpp# show version verbose >> Version: v17.10-rc0~86-g7d4a22c >> Compiled by: root >> Compile host: ubuntu >> Compile date: Wed Jul 26 18:56:51 EDT 2017 >> Compile location: /root/vpp/vpp >> Compiler: GCC 5.4.0 20160609 >> Current PID: 5006 >> >> Currently My Vm is with 4vcpu and vpp cpu model is as follow:- >> 1.one main thread >> 2.Three worker threads >> >> I am not able to get more than 4Mpps with this configuration of vpp.When >> i am pumping more than 4Mpps traffic to my VM. I can see this
[vpp-dev] vpp performnace measurement in sriov configuration.
Hi All, I am working on an use case where i want to measure the Maximum PPS handled by vpp in sriov configuration.I have created a virtual machine on a host having specifications as follows:- 1.RHEL 7.3 installed 2.Intel X540 10 gig NIC attached I have created a virtual function from one of the interface of 10 gig NIC(for e.g ens3f0) and attached it to my Virtual Machine.I have installed ubuntu 16.04 on my Vm. My vpp version running on vm is 17.10. vpp# show version verbose Version: v17.10-rc0~86-g7d4a22c Compiled by: root Compile host: ubuntu Compile date: Wed Jul 26 18:56:51 EDT 2017 Compile location: /root/vpp/vpp Compiler: GCC 5.4.0 20160609 Current PID: 5006 Currently My Vm is with 4vcpu and vpp cpu model is as follow:- 1.one main thread 2.Three worker threads I am not able to get more than 4Mpps with this configuration of vpp.When i am pumping more than 4Mpps traffic to my VM. I can see this counter rx_no_dma_resources: 5628104 getting incremented at the host.As there is no eth stats available in sriov configuration for virtual functions attached to VM. Guest vcpus are pin to host physical cpus. So 4Mpps is the expected number that we can get in this configuration? Thanks, Rahul ___ vpp-dev mailing list vpp-dev@lists.fd.io https://lists.fd.io/mailman/listinfo/vpp-dev
