Hi
I have two VMs (Ubuntu 14.04, BIGIP Virtual Edition) running over the
ovs-dpdk vhostuserport, everything works for Ubuntu 14.04 VM, but no
network connectivity for BIGIP Virtual Edition, I used DPDK pdump to
capture packet, I see arp reply egress over the dpdk port, but the arp
reply not seen on tcpdump at the other end. it appears to me there is some
interaction between BIGIP virtual edition and OVS 2.7.1 + DPDK 16.11.2 that
does not work together, I am not sure if this is issue at OVS 2.7.1 + DPDK
16.11.2 side or BIGIP virtual edition side, any clue would be appreciated!
Lab info:
Two Dell servers, Dell R710 and R210. both with Intel 82599ES dual 10G
port, used fiber cable directly connect the two 10G port
Dell R710: Ubuntu 16.04 + OVS 2.7.1 + DPDK 16.11.2
Dell R210: Ubuntu 14.04
/external vlan (10.1.72.1)|<----fiber cable--------->|em1 <--->iperf
client (10.1.72.62) \
| Dell R710 OVS+DPDK(BIGIP virtual edition)
Dell R210 |
\Internal vlan|(10.2.72.1)<----fiber cable--------->|em2 <--->iperf
server (10.2.72.86)/
I followed through Documentation/intro/install/dpdk.rst,
Documentation/howto/dpdk.rst, Documentation/topics/dpdk/vhost-user.rst to
setup the OVS and DPDK vhostuser interface, here is vs-vsctl show:
root@dell710:~# ovs-vsctl show
7344de6e-8717-4a32-a883-7f6743a730fd
Bridge "ovs-br2"
Port "ovs-br2"
Interface "ovs-br2"
type: internal
Port "dpdkvhostuser1"
Interface "dpdkvhostuser1"
type: dpdkvhostuser
Port "dpdkvhostuser3"
Interface "dpdkvhostuser3"<======assigned to VM
bigip-ovsdpdk-vhostuser (BIGIP virtual edition)
type: dpdkvhostuser
Port "dpdkvhostuser5"
Interface "dpdkvhostuser5" <======assigned to VM demovm-ipfwd
(Ubuntu 14.04)
type: dpdkvhostuser
Port "dpdk-port1"
Interface "dpdk-port1"
type: dpdk
options: {dpdk-devargs="0000:04:00.1"} <=====port 1
Bridge "ovs-br1"
Port "ovs-br1"
Interface "ovs-br1"
type: internal
Port "dpdkvhostuser4"
Interface "dpdkvhostuser4"
type: dpdkvhostuser
Port "dpdkvhostuser2"
Interface "dpdkvhostuser2"
type: dpdkvhostuser
Port "dpdk-port0"
Interface "dpdk-port0"
type: dpdk
options: {dpdk-devargs="0000:04:00.0"} <====port 0
Port "dpdkvhostuser0"
Interface "dpdkvhostuser0"
type: dpdkvhostuser
I have two VMs (Ubuntu 14.04, BIGIP Virtual Edition) running over the
ovs-dpdk vhostuserport
root@dell710:~# virsh list --all
Id Name State
----------------------------------------------------
27 bigip-ovsdpdk-vhostuser running
28 demovm-ipfwd running
the vhostuser assignment like below:
root@dell710:~# virsh dumpxml bigip-ovsdpdk-vhostuser
.................SKIP......................
<interface type='vhostuser'>
<mac address='00:00:00:00:00:04'/>
<source type='unix'
path='/usr/local/var/run/openvswitch/dpdkvhostuser3' mode='client'/>
<model type='virtio'/>
<driver queues='2'>
<host mrg_rxbuf='on'/>
</driver>
<alias name='net2'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07'
function='0x0'/>
</interface>
...............SKIP..........................
root@dell710:~# virsh dumpxml demovm-ipfwd
........................SKIP................
<interface type='vhostuser'>
<mac address='00:00:00:00:00:05'/>
<source type='unix'
path='/usr/local/var/run/openvswitch/dpdkvhostuser5' mode='client'/>
<model type='virtio'/>
<driver queues='2'>
<host mrg_rxbuf='off'/>
</driver>
<alias name='net2'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04'
function='0x0'/>
</interface>
..............SKIP.................
DPDK pdump on port 1:
root@dell710:/home/dpdk/dpdk-stable-16.11.2/x86_64-native-linuxapp-gcc/build/app/pdump#
./dpdk-pdump -- --pdump
'port=1,queue=*,rx-dev=/tmp/pkts.pcap,tx-dev=/tmp/pkts.pcap'
--server-socket-path=/usr/local/var/run/openvswitch
root@dell710:/home/dpdk/dpdk-stable-16.11.2/x86_64-native-linuxapp-gcc/build/app/pdump#
tcpdump -nr /tmp/pkts.pcap
reading from file /tmp/pkts.pcap, link-type EN10MB (Ethernet)
12:58:23.110027 IP6 fe80::200:ff:fe00:4 > ff02::1: ICMP6, neighbor
advertisement, tgt is fe80::200:ff:fe00:4, length 32
12:58:23.150007 IP6 fe80::200:ff:fe00:4 > ff02::1: ICMP6, neighbor
advertisement, tgt is fe80::200:ff:fe00:4, length 32
12:58:23.150009 IP6 fe80::200:ff:fe00:4 > ff02::1: ICMP6, neighbor
advertisement, tgt is fe80::200:ff:fe00:4, length 32
12:58:23.190007 IP6 fe80::200:ff:fe00:4 > ff02::1: ICMP6, neighbor
advertisement, tgt is fe80::200:ff:fe00:4, length 32
12:58:23.226163 IP6 fe80::200:ff:fe00:4 > ff02::1: ICMP6, neighbor
advertisement, tgt is fe80::200:ff:fe00:4, length 32
12:58:55.626367 ARP, Request who-has 10.2.72.1 (ff:ff:ff:ff:ff:ff) tell
10.2.72.1, length 28
12:58:55.626384 ARP, Request who-has 10.2.72.1 (ff:ff:ff:ff:ff:ff) tell
10.2.72.1, length 28
12:58:55.654094 ARP, Request who-has 10.2.72.1 (ff:ff:ff:ff:ff:ff) tell
10.2.72.1, length 28
12:58:55.654100 ARP, Request who-has 10.2.72.1 (ff:ff:ff:ff:ff:ff) tell
10.2.72.1, length 28
12:58:55.654101 ARP, Request who-has 10.2.72.1 (ff:ff:ff:ff:ff:ff) tell
10.2.72.1, length 28
12:59:35.078030 ARP, Request who-has 10.2.72.1 tell 10.2.72.86, length 46
12:59:35.102052 ARP, Reply 10.2.72.1 is-at 00:00:00:00:00:04, length 1504
<=====shows correct ARP reply
12:59:36.078044 ARP, Request who-has 10.2.72.1 tell 10.2.72.86, length 46
12:59:36.102050 ARP, Reply 10.2.72.1 is-at 00:00:00:00:00:04, length 1504
12:59:37.077999 ARP, Request who-has 10.2.72.1 tell 10.2.72.86, length 46
12:59:37.102017 ARP, Reply 10.2.72.1 is-at 00:00:00:00:00:04, length 1504
12:59:38.090002 ARP, Request who-has 10.2.72.1 tell 10.2.72.86, length 46
12:59:38.114060 ARP, Reply 10.2.72.1 is-at 00:00:00:00:00:04, length 1504
12:59:39.085945 ARP, Request who-has 10.2.72.1 tell 10.2.72.86, length 46
12:59:39.113939 ARP, Reply 10.2.72.1 is-at 00:00:00:00:00:04, length 1504
12:59:40.094024 ARP, Request who-has 10.2.72.1 tell 10.2.72.86, length 46
12:59:40.118035 ARP, Reply 10.2.72.1 is-at 00:00:00:00:00:04, length 1504
TCPDUMP on the Dell R210: (NO ARP REPLY SEEN HERE)
#tcpdump -nn -i em2 -s0 icmp or arp &
root@r210:~# telnet 10.2.72.1 443
Trying 10.2.72.1...
telnet: Unable to connect to remote host: No route to host
12:39:39.735757 ARP, Request who-has 10.2.72.1 (ff:ff:ff:ff:ff:ff) tell
10.2.72.1, length 46
12:39:39.742311 ARP, Request who-has 10.2.72.1 (ff:ff:ff:ff:ff:ff) tell
10.2.72.1, length 46
12:39:39.761819 ARP, Request who-has 10.2.72.1 (ff:ff:ff:ff:ff:ff) tell
10.2.72.1, length 46
12:39:39.762703 ARP, Request who-has 10.2.72.1 (ff:ff:ff:ff:ff:ff) tell
10.2.72.1, length 46
12:39:39.772618 ARP, Request who-has 10.2.72.1 (ff:ff:ff:ff:ff:ff) tell
10.2.72.1, length 46
12:40:19.193757 ARP, Request who-has 10.2.72.1 tell 10.2.72.86, length 28
12:40:20.191565 ARP, Request who-has 10.2.72.1 tell 10.2.72.86, length 28
12:40:21.191568 ARP, Request who-has 10.2.72.1 tell 10.2.72.86, length 28
12:40:22.191623 ARP, Request who-has 10.2.72.1 tell 10.2.72.86, length 28
12:40:23.191564 ARP, Request who-has 10.2.72.1 tell 10.2.72.86, length 28
12:40:24.191565 ARP, Request who-has 10.2.72.1 tell 10.2.72.86, length 28
I can provide the capture file or any other information if needed.
Regards,
Vincent
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
