I try to run OVS in VMware VM with two X710 VFs (only one is relevant for
simplified OVS config posted below). Ports are configured as SR-IOV
interfaces
in port group using virtual guest tagging.
With simplified config posted below I should be able to ping
interface vl1075 from other hosts on vlan 1075 but I'm not. Ovs-ofctl
dump-flows shows no incomming packets. Ovs-tcpdump doesn't show incomming
packets either.
I don't know what exactly the problem is but interestingly, there seems to
be a workaround.
When I run the DPDK's testpmd and then OVS (the same configuration that
fails
without workaraound) OVS works fine till next ESXi reboot.
OVS:
Interfaces:
root@test2k:~# dpdk-devbind.py -s
Network devices using DPDK-compatible driver
============================================
0000:0b:00.0 'Ethernet Virtual Function 700 Series 154c' drv=vfio-pci
unused=iavf
0000:13:00.0 'Ethernet Virtual Function 700 Series 154c' drv=vfio-pci
unused=iavf
[ ... ]
config:
ovs-vsctl emer-reset
ovs-vsctl set Open_vSwitch . "other_config:dpdk-init=true"
ovs-vsctl set Open_vSwitch . "other_config:hw-offload=true"
ovs-vsctl set Open_vSwitch . "other_config:per-port-memory=true"
ovs-vsctl set Open_vSwitch . "other_config:dpdk-alloc-mem=2048"
ovs-vsctl set Open_vSwitch . other_config:smc-enable=true
ovs-vsctl set Open_vSwitch . other_config:emc-insert-inv-prob=1
ovs-vsctl --if-exists del-br ovsbr0
ovs-vsctl add-br ovsbr0 -- set bridge ovsbr0 datapath_type=netdev
ovs-vsctl set bridge ovsbr0
protocols=OpenFlow10,OpenFlow11,OpenFlow12,OpenFlow13,OpenFlow14,OpenFlow15
ovs-vsctl add-port ovsbr0 dp_13000 -- set Interface dp_13000 type=dpdk
"options:dpdk-devargs=13:00.0" -- set interface dp_13000 ofport_request=4
ovs-vsctl set port dp_13000 trunks=1075
ovs-vsctl add-br vl1075 ovsbr0 1075
sysctl -w net.ipv6.conf.vl1075.disable_ipv6=1 > /dev/null
ip a add 192.168.77.3/24 dev vl1075
ip l set up vl1075
ovs-ofctl del-flows ovsbr0
ovs-appctl dpctl/del-flows
ovs-ofctl -O OpenFlow15 add-flow ovsbr0
"table=0,priority=20,in_port=vl1075,actions=push_vlan:0x8100,mod_vlan_vid:1075,output:dp_13000"
ovs-ofctl -O OpenFlow15 add-flow ovsbr0
"table=0,priority=200,in_port=dp_13000,dl_vlan=1075,actions=strip_vlan,output:vl1075"
workaround testpmd config
dpdk-testpmd -n 2 --socket-mem 1024 -l 0,1,2 -a 0000:0b:00.0 -a
0000:13:00.0 -- --port-topology=paired --forward-mode=io --nb-cores=2
--txd=4096 --rxd=4096 --cmdline-file=cmd.list << END
END
cmd.list contents:
set fwd mac
set eth-peer 0 00:0c:29:29:7d:8a
set eth-peer 1 00:0c:29:29:7d:8b
vlan set filter on 0
vlan set filter on 1
vlan set strip on 0
vlan set strip on 1
rx_vlan add 1075 1
rx_vlan add 1076 0
port stop 0
port stop 1
tx_vlan set 1 1075
tx_vlan set 0 1076
port start 0
port start 1
quit
Versions being used:
ESXi: 7.0U3, i40en Version: 2.2.7.0, NIC Firmware Version: 8.40 0x8000af82
20.5.13
guest OS: Ubuntu Kinetic
Linux version 5.15.0-27-generic (buildd@ubuntu) (gcc (Ubuntu
11.2.0-19ubuntu1) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #28-Ubuntu
SMP Thu Apr 14 04:55:28 UTC 2022
OVS 3.0.0 (from Kinetic apt repos)
DPDK: 21.11.1 (from Kinetic apt repos)
Other possibly usefull outputs:
root@345fd4e20203:~# ovs-dpctl show
root@345fd4e20203:~# ovs-ofctl dump-flows ovsbr0
cookie=0x0, duration=464.397s, table=0, n_packets=0, n_bytes=0,
priority=200,in_port="dp_13000",dl_vlan=1075
actions=strip_vlan,output:vl1075
cookie=0x0, duration=464.411s, table=0, n_packets=0, n_bytes=0,
priority=20,in_port=vl1075 actions=mod_vlan_vid:1075,output:"dp_13000"
root@345fd4e20203:~# ovs-appctl dpctl/dump-flows
flow-dump from pmd on cpu core: 3
recirc_id(0),in_port(2),packet_type(ns=0,id=0),eth_type(0/0xffff),
packets:239, bytes:15264, used:1.339s, actions:drop
root@345fd4e20203:~# ovs-appctl dpctl/dump-flows
flow-dump from pmd on cpu core: 3
recirc_id(0),in_port(2),packet_type(ns=0,id=0),eth_type(0/0xffff),
packets:244, bytes:15564, used:1.691s, actions:drop
recirc_id(0),in_port(2),packet_type(ns=0,id=0),eth_type(0x88cc), packets:0,
bytes:0, used:never, actions:drop
root@345fd4e20203:~#
root@345fd4e20203:~# ovs-appctl dpif/show
netdev@ovs-netdev: hit:455 missed:41
ovsbr0:
dp_13000 4/2: (dpdk: configured_rx_queues=1,
configured_rxq_descriptors=2048, configured_tx_queues=2,
configured_txq_descriptors=2048, lsc_interrupt_mode=false, mtu=1500,
requested_rx_queues=1, requested_rxq_descriptors=2048,
requested_tx_queues=2, requested_txq_descriptors=2048,
rx_csum_offload=true, tx_tso_offload=false)
ovsbr0 65534/1: (tap)
vl1075 1/3: (tap)
root@345fd4e20203:~# ovs-vsctl show
4cbe326a-fda7-4102-bf47-f7c16c49e2e5
Bridge ovsbr0
datapath_type: netdev
Port ovsbr0
Interface ovsbr0
type: internal
Port vl1075
tag: 1075
Interface vl1075
type: internal
Port dp_13000
trunks: [1075]
Interface dp_13000
type: dpdk
options: {dpdk-devargs="13:00.0"}
root@345fd4e20203:~#
Thanks in advance for any help.
Paweł
--
-------------------------------
This document contains material that is
confidential in CodiLime Sp. z o.o. DO NOT PRINT. DO NOT COPY. DO NOT
DISTRIBUTE. If you are not the intended recipient of this document, be
aware that any use, review, retransmission, distribution, reproduction or
any action taken in reliance upon this message is strictly prohibited. If
you received this in error, please contact the sender and [email protected]
<mailto:[email protected]>. Return the paper copy, delete the material from
all computers and storage media.
_______________________________________________
discuss mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
