> On 07.12.2020., at 22:55, Christian Hopps <cho...@chopps.org> wrote: > > > >> On Dec 7, 2020, at 3:02 PM, Damjan Marion <damjan.mar...@gmail.com> wrote: >> >> >> >>> On 07.12.2020., at 20:41, Christian Hopps <cho...@chopps.org> wrote: >>> >>> >>> >>>> On Dec 7, 2020, at 1:44 PM, Damjan Marion <damjan.mar...@gmail.com> wrote: >>>> >>>>> >>>>> On 07.12.2020., at 17:02, Christian Hopps <cho...@chopps.org> wrote: >>>>> >>>> >>>> please send me output of: extras/scripts/lsnet script and exact “create >>>> interface avf” commands you use…. >>> >>> PCI Address MAC address Device Name Driver State Speed >>> Port Type >>> ============ ================= ============== ========== ======== >>> ========== ==================== >>> 0000:65:00.0 40:a6:b7:4b:62:08 enp101s0f0 i40e down 10000Mb/s >>> Direct Attach Copper >>> 0000:65:00.1 40:a6:b7:4b:62:09 enp101s0f1 i40e down 10000Mb/s >>> Direct Attach Copper >>> 0000:65:00.2 40:a6:b7:4b:62:0a enp101s0f2 i40e down 10000Mb/s >>> Direct Attach Copper >>> 0000:65:00.3 40:a6:b7:4b:62:0b enp101s0f3 i40e down 10000Mb/s >>> Direct Attach Copper >>> 0000:b3:00.0 00:e0:8d:7e:1f:36 enp179s0f0 ixgbe down Unknown! >>> Direct Attach Copper >>> 0000:b3:00.1 00:e0:8d:7e:1f:37 enp179s0f1 ixgbe down Unknown! >>> Direct Attach Copper >>> 0000:01:00.0 a0:42:3f:3c:f8:ee enp1s0f0 ixgbe up 10000Mb/s >>> Twisted Pair >>> 0000:01:00.1 a0:42:3f:3c:f8:ef enp1s0f1 ixgbe down Unknown! >>> Twisted Pair >>> 0000:17:00.0 f8:f2:1e:3c:15:ec enp23s0f0 ixgbe down Unknown! >>> Direct Attach Copper >>> 0000:17:00.1 f8:f2:1e:3c:15:ed enp23s0f1 ixgbe down Unknown! >>> Direct Attach Copper >>> 0000:01:10.0 52:bf:27:59:df:50 eth0 ixgbevf down Unknown! >>> Other >>> 0000:01:10.2 ee:24:0b:0c:93:3f eth1 ixgbevf down Unknown! >>> Other >>> 0000:01:10.4 9e:8e:ce:da:38:f5 eth2 ixgbevf down Unknown! >>> Other >>> 0000:01:10.6 2a:f4:a2:ea:4c:5d eth3 ixgbevf down Unknown! >>> Other >>> >> >> I dont see your VFs on the list?Do you have them created? >> Do you see them with “lspci | grep Ether”? > > [ The VFs are not in the list b/c they are created as part of the automation > that also launches vpp, I ran this before running that b/c once vpp is up the > VFs also don't show up in that list b/c they have been rebound. :) ] > > First, things work with 2.12.; however, 2.12 does not load on reboot, I must > rmmod and modprobe after rebooting to get the 2.12 driver. I do have another > problem though (mentioned at end)...
I just bumped it to 2.13.10 and added REMAKE_INITRD=“yes” into debian/dkms.in so that should be fixed: $ lsinitramfs /boot/initrd.img | grep i40e usr/lib/modules/5.8.0-31-generic/updates/dkms/i40e.ko > >> also, I asked for your crete interface config…. > > create interface avf 0000:65:0a.0 > > As mentioned above, when I rmmod/modprobe the new driver I don't hit the lut > error anymore. > > Now I need to figure out why I see such bad performance (tons of rx discards) > using these interfaces (when using 2 of them), but not when using 1 avf VF > and the other interface is a 10G i520 nic (dpdk driver). > > Bad Side: > > $ docker-compose exec p1 vppctl show hard > Name Idx Link Hardware > avf-0/65/2/0 2 up avf-0/65/2/0 > Link speed: 10 Gbps > Ethernet address 02:41:0d:0d:0d:0b > flags: initialized admin-up vaddr-dma link-up rx-interrupts > offload features: l2 adv-link-speed vlan rx-polling rss-pf > num-queue-pairs 6 max-vectors 5 max-mtu 0 rss-key-size 52 rss-lut-size 64 > speed > stats: > rx bytes 1520728906 > rx unicast 184910147 > rx discards 129083610 > tx bytes 4226915088 > tx unicast 184349288 > avf-0/65/a/0 1 up avf-0/65/a/0 > Link speed: 10 Gbps > Ethernet address 02:41:0b:0b:0b:0b > flags: initialized admin-up vaddr-dma link-up rx-interrupts > offload features: l2 adv-link-speed vlan rx-polling rss-pf > num-queue-pairs 6 max-vectors 5 max-mtu 0 rss-key-size 52 rss-lut-size 64 > speed > stats: > rx bytes 3781507000 > rx unicast 93533516 > rx broadcast 3 > rx discards 73223358 > tx bytes 2212324998 > tx unicast 13714424 > ipsec0 3 up ipsec0 > Link speed: unknown > IPSec > local0 0 down local0 > Link speed: unknown > local > loop0 4 up loop0 > Link speed: unknown > Ethernet address de:ad:00:00:00:00 > > Good Side: > > $ docker-compose exec p2 vppctl show hard > Name Idx Link Hardware > TenGigabitEthernet17/0/0 1 up TenGigabitEthernet17/0/0 > Link speed: 10 Gbps > Ethernet address f8:f2:1e:3c:15:ec > Intel 82599 > carrier up full duplex mtu 9206 > flags: admin-up pmd maybe-multiseg tx-offload intel-phdr-cksum > rx-ip4-cksum > Devargs: > rx: queues 1 (max 128), desc 1024 (min 32 max 4096 align 8) > tx: queues 6 (max 64), desc 1024 (min 32 max 4096 align 8) > pci: device 8086:10fb subsystem 8086:0003 address 0000:17:00.00 numa 0 > max rx packet len: 15872 > promiscuous: unicast off all-multicast on > vlan offload: strip off filter off qinq off > rx offload avail: vlan-strip ipv4-cksum udp-cksum tcp-cksum tcp-lro > macsec-strip vlan-filter vlan-extend jumbo-frame > scatter > security keep-crc rss-hash > rx offload active: ipv4-cksum jumbo-frame scatter > tx offload avail: vlan-insert ipv4-cksum udp-cksum tcp-cksum sctp-cksum > tcp-tso macsec-insert multi-segs security > tx offload active: udp-cksum tcp-cksum multi-segs > rss avail: ipv4-tcp ipv4-udp ipv4 ipv6-tcp-ex ipv6-udp-ex ipv6-tcp > ipv6-udp ipv6-ex ipv6 > rss active: none > tx burst function: ixgbe_xmit_pkts > rx burst function: ixgbe_recv_scattered_pkts_vec > > tx frames ok 183386223 > tx bytes ok 277646741622 > rx frames ok 182834391 > rx bytes ok 276811244995 > extended stats: > rx_good_packets 182834488 > tx_good_packets 183386223 > rx_good_bytes 276811391853 > tx_good_bytes 277646741622 > rx_q0packets 182834488 > rx_q0bytes 276811391853 > tx_q0packets 183386223 > tx_q0bytes 277646741622 > rx_size_65_to_127_packets 15 > rx_size_1024_to_max_packets 182834425 > rx_multicast_packets 15 > rx_total_packets 182834439 > rx_total_bytes 276811319181 > tx_total_packets 183386223 > tx_size_1024_to_max_packets 183386223 > out_pkts_untagged 183386223 > avf-0/65/e/0 2 up avf-0/65/e/0 > Link speed: 10 Gbps > Ethernet address 02:42:0c:0c:0c:0c > flags: initialized admin-up vaddr-dma link-up rx-interrupts > offload features: l2 adv-link-speed vlan rx-polling rss-pf > num-queue-pairs 6 max-vectors 5 max-mtu 0 rss-key-size 52 rss-lut-size 64 > speed > stats: > rx bytes 2520527812 > rx unicast 92644250 > rx broadcast 3 > tx bytes 2677106258 > tx unicast 20118042 > ipsec0 3 up ipsec0 > Link speed: unknown > IPSec > local0 0 down local0 > Link speed: unknown > local > loop0 4 up loop0 > Link speed: unknown > Ethernet address de:ad:00:00:00:00 > Weird, no idea based on this outputs… What show run says? > Thanks, > Chris. > >> >>> >>>> >>>>> >>>>>> What kernel and driver version do you use? >>>>> >>>>> Host Config: >>>>> >>>>> $ cat /etc/lsb-release >>>>> DISTRIB_ID=Ubuntu >>>>> DISTRIB_RELEASE=20.10 >>>>> DISTRIB_CODENAME=groovy >>>>> DISTRIB_DESCRIPTION="Ubuntu 20.10" >>>>> $ uname -a >>>>> Linux labnh 5.8.0-31-generic #33-Ubuntu SMP Mon Nov 23 18:44:54 UTC 2020 >>>>> x86_64 x86_64 x86_64 GNU/Linux >>>>> >>>>> Docker Config (compiled and run inside): >>>>> >>>>> root@p1:/# cat /etc/lsb-release >>>>> DISTRIB_ID=Ubuntu >>>>> DISTRIB_RELEASE=18.04 >>>>> DISTRIB_CODENAME=bionic >>>>> DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS" >>>>> >>>>>> >>>>>> Have you tried latest PF driver from intel? >>>>> >>>>> No; however, I am running such a new ubuntu (5.8 kernel) so I was hoping >>>>> that was sufficient. >>>> >>>> Agree, still, please, do me a favour and try with latest, so I know i’m >>>> looking at the same thing. >>> >>> I did the build and installed the deb, I also updated the firmware on the >>> NIC; however, >>> >>> $ dpkg -l | grep i40 >>> ii i40e-dkms 2.12.6 >>> all Intel i40e adapter driver >>> >>> $ sudo ethtool -i enp101s0f0 >>> driver: i40e >>> version: 2.8.20-k >>> firmware-version: 8.15 0x80009621 1.2829.0 >>> expansion-rom-version: >>> bus-info: 0000:65:00.0 >>> supports-statistics: yes >>> supports-test: yes >>> supports-eeprom-access: yes >>> supports-register-dump: yes >>> supports-priv-flags: yes >>> >>> It doesn't seem to have used it. Is there something else I need to do to >>> have it use the dkms driver? >> >> rmmod i40e; modprobe i40e >> or reboot… >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18335): https://lists.fd.io/g/vpp-dev/message/18335 Mute This Topic: https://lists.fd.io/mt/78779141/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
