I’ll let others jump in on the tx [and rx] drops. At some level, it looks the vfio device simply isn’t disposing of traffic rapidly enough. Perhaps too few buffers have been configured, but I don’t think that’s the most likely explanation.
The “show run” stats are not helpful because they include over 2000 seconds’s worth of [mostly] dead airtime: please start traffic, “clear run”, wait a few seconds, then “show run”. You didn’t mention the packet size that you’re using. It wouldn’t shock me to learn that a “10xGE capable instance” comes with a few stars in the margin: perhaps “10xGE capable at or above 256 byte packets” or some such. HTH... Dave From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Santosh Kumar Dornal Sent: Thursday, October 4, 2018 9:40 PM To: vpp-dev@lists.fd.io Subject: [vpp-dev] Tx packet drops - dpdk tx failure Hi I am using VPP 18.04 on AWS (c5.xlarge instance with ena interfaces) and running into Tx packet drops when pushing traffic over 2.5Gbps (bidirectional). The instance is capable of doing 10gig. Can somebody help? vpp# show version vpp v18.04-16~gca7a68e0 built by vagrant on debian9.localdomain at Wed Oct 3 11:57:44 PDT 2018 vpp# show errors Count Node Reason 68 arp-input ARP replies sent 82693102 VirtualFunctionEthernet0/6/0-tx Tx packet drops (dpdk tx failure) 83200033 VirtualFunctionEthernet0/7/0-tx Tx packet drops (dpdk tx failure) vpp# vpp# show hardware-interfaces detail Name Idx Link Hardware VirtualFunctionEthernet0/6/0 1 up VirtualFunctionEthernet0/6/0 Ethernet address 02:d2:6c:6e:b0:1c AWS ENA VF carrier up full duplex speed 10000 mtu 9216 pci id: device 1d0f:ec20 subsystem 0000:0000 pci address: 0000:00:06.00 max rx packet len: 9216 max num of queues: rx 8 tx 8 promiscuous: unicast off all-multicast off vlan offload: strip off filter off qinq off rx offload caps: tx offload caps: ipv4-cksum udp-cksum tcp-cksum rss active: ipv4-sctp ipv4-other ipv6-udp-ex ipv6-frag ipv6-tcp l2-payload rss supported: none rx queues 1, rx desc 1024, tx queues 1, tx desc 1024 cpu socket 0 tx frames ok 379327687 tx bytes ok 573877720248 rx frames ok 481696392 rx bytes ok 729288200634 rx missed 576271 extended stats: rx good packets 481696392 tx good packets 379327687 rx good bytes 729288200634 tx good bytes 573877720248 rx missed errors 576271 rx errors 0 tx errors 0 rx mbuf allocation errors 0 rx q0packets 0 rx q0bytes 0 rx q0errors 0 tx q0packets 0 tx q0bytes 0 VirtualFunctionEthernet0/7/0 2 up VirtualFunctionEthernet0/7/0 Ethernet address 02:cb:fc:e1:b2:e6 AWS ENA VF carrier up full duplex speed 10000 mtu 9216 pci id: device 1d0f:ec20 subsystem 0000:0000 pci address: 0000:00:07.00 max rx packet len: 9216 max num of queues: rx 8 tx 8 promiscuous: unicast off all-multicast off vlan offload: strip off filter off qinq off rx offload caps: tx offload caps: ipv4-cksum udp-cksum tcp-cksum rss active: ipv4-sctp ipv4-other ipv6-udp-ex ipv6-frag ipv6-tcp l2-payload rss supported: none rx queues 1, rx desc 1024, tx queues 1, tx desc 1024 cpu socket 0 tx frames ok 378158827 tx bytes ok 572532333080 rx frames ok 483669396 rx bytes ok 731851064762 rx missed 576803 extended stats: rx good packets 483669396 tx good packets 378158827 rx good bytes 731851064762 tx good bytes 572532333080 rx missed errors 576803 rx errors 0 tx errors 0 rx mbuf allocation errors 0 rx q0packets 0 rx q0bytes 0 rx q0errors 0 tx q0packets 0 tx q0bytes 0 local0 0 down local0 local vpp# vpp# show interface Name Idx State Counter Count VirtualFunctionEthernet0/6/0 1 up rx packets 481696393 rx bytes 729288200676 tx packets 483669396 tx bytes 731851064730 drops 4 ip4 481696346 rx-miss 576271 tx-error 104341708 VirtualFunctionEthernet0/7/0 2 up rx packets 483669397 rx bytes 731851064804 tx packets 481696392 tx bytes 729288200602 drops 4 ip4 483669350 rx-miss 576803 tx-error 103537564 local0 0 down vpp# show runtime Time 2390.3, average vectors/node 2.46, last 128 main loops 0.00 per node 0.00 vector rates in 3.9979e5, out 3.1282e5, drop 2.5101e-3, punt 0.0000e0 Name State Calls Vectors Suspends Clocks Vectors/Call VirtualFunctionEthernet0/6/0-o active 203354869 478794413 0 7.81e1 2.35 VirtualFunctionEthernet0/6/0-t active 203354869 374452705 0 4.01e2 1.84 VirtualFunctionEthernet0/7/0-o active 203863473 476819589 0 7.60e1 2.34 VirtualFunctionEthernet0/7/0-t active 203863473 373282025 0 3.89e2 1.83 api-rx-from-ring any wait 0 0 120 1.19e4 0.00 arp-input active 61 88 0 2.75e4 1.44 dhcp-client-process any wait 0 0 24 5.04e3 0.00 dns-resolver-process any wait 0 0 2 4.74e3 0.00 dpdk-input polling 16754698175 955614004 0 4.47e3 .06 dpdk-process any wait 0 0 797 1.01e5 0.00 error-drop active 6 6 0 3.91e3 1.00 ethernet-input active 61 88 0 3.33e3 1.44 fib-walk any wait 0 0 1195 4.96e3 0.00 ikev2-manager-process any wait 0 0 2390 2.49e3 0.00 interface-output active 57 84 0 2.79e3 1.47 ip-route-resolver-process any wait 0 0 24 3.33e3 0.00 ip4-glean active 2 2 0 2.06e4 1.00 ip4-input active 382393615 955613916 0 1.37e2 2.49 ip4-lookup active 382393615 955613916 0 9.37e1 2.49 ip4-reassembly-expire-walk any wait 0 0 239 5.99e3 0.00 ip4-rewrite active 382393613 955613914 0 9.20e1 2.49 ip6-icmp-neighbor-discovery-ev any wait 0 0 2390 2.26e3 0.00 ip6-reassembly-expire-walk any wait 0 0 239 5.69e3 0.00 lisp-retry-service any wait 0 0 1195 5.34e3 0.00 rd-cp-process any wait 0 0 118658595 3.15e2 0.00 unix-cli-local:2 active 2 0 1363 3.00e9 0.00 unix-epoll-input polling 16346047 0 0 1.07e3 0.00 vpe-oam-process any wait 0 0 1171 3.69e3 0.00 vpp# -- Santosh
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#10759): https://lists.fd.io/g/vpp-dev/message/10759 Mute This Topic: https://lists.fd.io/mt/26799215/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-