On Tue, Nov 6, 2018 at 5:42 PM Gregory Rose <gvrose8...@gmail.com> wrote:
> > On 11/6/2018 8:51 AM, Siva Teja ARETI wrote: > > Hi Greg, > > Thanks for looking into this. > > I have two VMs in my setup each with two interfaces. Trying to setup the > VXLAN tunnels across these interfaces which are in different subnets. A > docker container is attached to ovs bridge using ovs-docker utility on each > VM and doing a ping from one container to another. > > > Hi Siva, > > In reading through the documentation and looking at your configuration I > noticed that when using the > local_ip option the remote_ip is not set to flow. If the local_ip option > is specified then remote_ip must > equal flow. > > From the documentation (man ovs-vswitchd.conf.db): > > options : local_ip: optional string > Optional. The tunnel destination IP that received packets > must > match. Default is to match all addresses. If specified, > may be > one of: > > · An IPv4/IPv6 address (not a DNS name), e.g. > 192.168.12.3. > > · The word flow. The tunnel accepts packets sent to > any of > the local IP addresses of the system running > OVS. To > process only packets sent to a specific IP address, > the > flow entries may match on the tun_dst or > tun_ipv6_dst > field. When sending packets to a local_ip=flow > tunnel, > the flow actions may explicitly set the > tun_src or > tun_ipv6_src field to the desired IP address, e.g. > with a > set_field action. However, while routing the > tunneled > packet out, the local system may override the > specified > address with the local IP address configured for the > out‐ > going system interface. > > This option is valid only for tunnels also > configured > with the remote_ip=flow option. > As I understand this documentation, option local_ip=flow can only be specified when remote_ip=flow is also specified. Otherwise, ovs-vsctl throws this error I guess? [root@vm1 ~]# ovs-vsctl add-port testbr0 vxlan1 -- set interface vxlan1 type=vxlan options:local_ip=flow options:remote_ip=30.30.0.193 options:dst_port=4789 ovs-vsctl: Error detected while setting up 'vxlan1': vxlan1: vxlan type requires 'remote_ip=flow' with 'local_ip=flow'. See ovs-vswitchd log for details. ovs-vsctl: The default log directory is "/var/log/openvswitch". Please try using the remote_ip=flow option and then configuring the proper > flow and action. > Anyways, I also tried by specifying remote_ip=flow option when creating the tunnel. But I still the same issue [root@vm1 ~]# ovs-vsctl add-port testbr0 vxlan0 -- set interface vxlan0 type=vxlan options:local_ip=20.20.0.183 options:remote_ip=flow options:dst_port=4789 [root@vm1 ~]# ovs-appctl dpif/show system@ovs-system: hit:69275 missed:1050 testbr0: 2cfb62a9b0f04_l 2/3: (system) testbr0 65534/1: (internal) vxlan0 3/2: (vxlan: local_ip=20.20.0.183, remote_ip=flow) [root@vm1 ~]# ovs-ofctl add-flow testbr0 'table=0 priority=100 in_port=2 actions=set_tunnel:3,set_field:30.30.0.193->tun_dst,output:3' > Thanks, > > - Greg > > > *VM1 details:* > > [root@vm1 ~]# ip a > ....... > 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state > UP qlen 1000 > link/ether 52:54:00:b8:05:be brd ff:ff:ff:ff:ff:ff > inet 30.30.0.59/24 brd 30.30.0.255 scope global dynamic eth1 > valid_lft 3002sec preferred_lft 3002sec > inet6 fe80::5054:ff:feb8:5be/64 scope link > valid_lft forever preferred_lft forever > 4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state > UP qlen 1000 > link/ether 52:54:00:f0:64:37 brd ff:ff:ff:ff:ff:ff > inet 20.20.0.183/24 brd 20.20.0.255 scope global dynamic eth2 > valid_lft 3248sec preferred_lft 3248sec > inet6 fe80::5054:ff:fef0:6437/64 scope link > valid_lft forever preferred_lft forever > ....... > [root@vm1 ~]# ovs-vsctl show > ff70c814-d1b0-4018-aee8-8b635187afee > Bridge "testbr0" > Port "gre0" > Interface "gre0" > type: gre > options: {local_ip="20.20.0.183", remote_ip="30.30.0.193"} > Port "testbr0" > Interface "testbr0" > type: internal > Port "2cfb62a9b0f04_l" > Interface "2cfb62a9b0f04_l" > ovs_version: "2.9.2" > [root@vm1 ~]# ip rule list > 0: from all lookup local > 32765: from 20.20.0.183 lookup siva > 32766: from all lookup main > 32767: from all lookup default > [root@vm1 ~]# ip route show table siva > default dev eth2 scope link src 20.20.0.183 > [root@vm1 ~]# ######################### A docker container is attached to > ovs bridge using ovs-docker utility > [root@vm1 ~]# docker ps > CONTAINER ID IMAGE COMMAND CREATED > STATUS PORTS NAMES > be4ab434db99 busybox "sh" 5 days ago > Up 5 days admiring_euclid > [root@vm1 ~]# nsenter -n -t `docker inspect be4 --format={{.State.Pid}}` > -- ip a > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 > inet 127.0.0.1/8 scope host lo > valid_lft forever preferred_lft forever > inet6 ::1/128 scope host > valid_lft forever preferred_lft forever > 2: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN qlen 1 > link/gre 0.0.0.0 brd 0.0.0.0 > 3: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN > qlen 1000 > link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff > 9: eth0@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue > state UP qlen 1000 > link/ether 22:98:41:0f:e8:50 brd ff:ff:ff:ff:ff:ff link-netnsid 0 > inet 70.70.0.10/24 scope global eth0 > valid_lft forever preferred_lft forever > inet6 fe80::2098:41ff:fe0f:e850/64 scope link > valid_lft forever preferred_lft forever > > > *VM2 details:* > > [root@vm2 ~]# ip a > ........ > 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state > UP qlen 1000 > link/ether 52:54:00:79:ef:92 brd ff:ff:ff:ff:ff:ff > inet 30.30.0.193/24 brd 30.30.0.255 scope global dynamic eth1 > valid_lft 2406sec preferred_lft 2406sec > inet6 fe80::5054:ff:fe79:ef92/64 scope link > valid_lft forever preferred_lft forever > 4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state > UP qlen 1000 > link/ether 52:54:00:05:93:7c brd ff:ff:ff:ff:ff:ff > inet 20.20.0.64/24 brd 20.20.0.255 scope global dynamic eth2 > valid_lft 2775sec preferred_lft 2775sec > inet6 fe80::5054:ff:fe05:937c/64 scope link > valid_lft forever preferred_lft forever > ....... > [root@vm2 ~]# ovs-vsctl show > b85514db-3f29-4f7a-9001-37d70adfca34 > Bridge "testbr0" > Port "gre0" > Interface "gre0" > type: gre > options: {local_ip="30.30.0.193", remote_ip="20.20.0.183"} > Port "a0769422cfc04_l" > Interface "a0769422cfc04_l" > Port "testbr0" > Interface "testbr0" > type: internal > ovs_version: "2.9.2" > [root@vm2 ~]# ip rule list > 0: from all lookup local > 32766: from all lookup main > 32767: from all lookup default > [root@vm2 ~]# ######################### A docker container is attached to > ovs bridge using ovs-docker utility > [root@vm2 ~]# docker ps > CONTAINER ID IMAGE COMMAND CREATED > STATUS PORTS NAMES > 86214f0d99e8 busybox:latest "sh" 5 days ago > Up 5 days peaceful_snyder > [root@vm2 ~]# nsenter -n -t `docker inspect 862 --format={{.State.Pid}}` > -- ip a > 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 > inet 127.0.0.1/8 scope host lo > valid_lft forever preferred_lft forever > inet6 ::1/128 scope host > valid_lft forever preferred_lft forever > 2: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN qlen 1 > link/gre 0.0.0.0 brd 0.0.0.0 > 3: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN > qlen 1000 > link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff > 9: eth0@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue > state UP qlen 1000 > link/ether ae:ac:14:7a:40:5f brd ff:ff:ff:ff:ff:ff link-netnsid 0 > inet 70.70.0.20/24 scope global eth0 > valid_lft forever preferred_lft forever > inet6 fe80::acac:14ff:fe7a:405f/64 scope link > valid_lft forever preferred_lft forever > > With this configuration, if I do a ping from docker container on VM1 to > docker container on VM2 it works. > > [root@vm1 ~]# nsenter -n -t `docker inspect be4 --format={{.State.Pid}}` > -- ping 70.70.0.20 > PING 70.70.0.20 (70.70.0.20) 56(84) bytes of data. > 64 bytes from 70.70.0.20: icmp_seq=1 ttl=64 time=0.831 ms > 64 bytes from 70.70.0.20: icmp_seq=2 ttl=64 time=0.933 ms > 64 bytes from 70.70.0.20: icmp_seq=3 ttl=64 time=0.564 ms > ^C > --- 70.70.0.20 ping statistics --- > 3 packets transmitted, 3 received, 0% packet loss, time 2001ms > rtt min/avg/max/mdev = 0.564/0.776/0.933/0.155 ms > > And the traffic is as expected on VM2. > > [root@vm2 ~]# tcpdump -n -i any host 20.20.0.183 > > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 > bytes > 16:37:32.262553 IP 20.20.0.183 > 30.30.0.193: GREv0, length 102: IP > 70.70.0.10 > 70.70.0.20: ICMP echo request, id 28158, seq 1, length 64 > 16:37:32.262835 IP 30.30.0.193 > 20.20.0.183: GREv0, length 102: IP > 70.70.0.20 > 70.70.0.10: ICMP echo reply, id 28158, seq 1, length 64 > 16:37:33.263211 IP 20.20.0.183 > 30.30.0.193: GREv0, length 102: IP > 70.70.0.10 > 70.70.0.20: ICMP echo request, id 28158, seq 2, length 64 > 16:37:33.263374 IP 30.30.0.193 > 20.20.0.183: GREv0, length 102: IP > 70.70.0.20 > 70.70.0.10: ICMP echo reply, id 28158, seq 2, length 64 > 16:37:34.264159 IP 20.20.0.183 > 30.30.0.193: GREv0, length 102: IP > 70.70.0.10 > 70.70.0.20: ICMP echo request, id 28158, seq 3, length 64 > 16:37:34.264252 IP 30.30.0.193 > 20.20.0.183: GREv0, length 102: IP > 70.70.0.20 > 70.70.0.10: ICMP echo reply, id 28158, seq 3, length 64 > > But when I change the tunnel type to vxlan, ping fails. > > [root@vm1 ~]# ovs-vsctl del-port testbr0 gre0 > [root@vm1 ~]# ovs-vsctl add-port testbr0 vxlan0 -- set interface vxlan0 > type=vxlan options:local_ip=20.20.0.183 options:remote_ip=30.30.0.193 > options:dst_port=4789 > [root@vm1 ~]# ovs-vsctl show > ff70c814-d1b0-4018-aee8-8b635187afee > Bridge "testbr0" > Port "testbr0" > Interface "testbr0" > type: internal > Port "vxlan0" > Interface "vxlan0" > type: vxlan > options: {dst_port="4789", local_ip="20.20.0.183", > remote_ip="30.30.0.193"} > Port "2cfb62a9b0f04_l" > Interface "2cfb62a9b0f04_l" > ovs_version: "2.9.2" > > [root@vm2 ~]# ovs-vsctl del-port testbr0 gre0 > [root@vm2 ~]# ovs-vsctl add-port testbr0 vxlan0 -- set interface vxlan0 > type=vxlan options:local_ip=30.30.0.193 options:remote_ip=20.20.0.183 > options:dst_port=4789 > [root@vm2 ~]# ovs-vsctl show > b85514db-3f29-4f7a-9001-37d70adfca34 > Bridge "testbr0" > Port "a0769422cfc04_l" > Interface "a0769422cfc04_l" > Port "vxlan0" > Interface "vxlan0" > type: vxlan > options: {dst_port="4789", local_ip="30.30.0.193", > remote_ip="20.20.0.183"} > Port "testbr0" > Interface "testbr0" > type: internal > ovs_version: "2.9.2" > > Ping fails with this setup > > [root@vm1 ~]# nsenter -n -t `docker inspect be4 --format={{.State.Pid}}` > -- ping 70.70.0.20 > PING 70.70.0.20 (70.70.0.20) 56(84) bytes of data. > ^C > --- 70.70.0.20 ping statistics --- > 6 packets transmitted, 0 received, 100% packet loss, time 4999ms > > Expected traffic is not seen on VM2 > > [root@vm2 ~]# tcpdump -n -i any host 20.20.0.183 > > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 > bytes > ^C > 0 packets captured > 0 packets received by filter > 0 packets dropped by kernel > > Kindly let me know if you need more information. > > Siva Teja. > > On Tue, Nov 6, 2018 at 10:49 AM Gregory Rose <gvrose8...@gmail.com> wrote: > >> >> On 11/5/2018 6:10 PM, Siva Teja ARETI wrote: >> >> Hi, >> >> I am trying to use local_ip option for a VXLAN tunnel using ovs but it >> does not seem to work. The same works when I use GRE tunnel. I also found a >> previous discussion from another user who tried the exact same approach. >> Here is the link to the discussion >> >> *https://www.mail-archive.com/ovs-discuss@openvswitch.org/msg03643.html >> <https://www.mail-archive.com/ovs-discuss@openvswitch.org/msg03643.html>* >> >> I am unable to find any working resolution at the end of this discussion. >> Could you please help? >> >> >> I looked into that but was never able to set up a configuration like the >> one in that discussion and could >> not repro the bug. >> >> Please provide some details on your usage, configuration and steps to >> repro and I can look into it. >> >> Thanks, >> >> - Greg >> >> >> I am using ovs 2.9.2 >> >> [root@localhost ~]# ovs-vsctl --version >> ovs-vsctl (Open vSwitch) 2.9.2 >> DB Schema 7.15.1 >> >> Thanks, >> Siva Teja. >> >> >> _______________________________________________ >> discuss mailing >> listdiscuss@openvswitch.orghttps://mail.openvswitch.org/mailman/listinfo/ovs-discuss >> >> >> >
_______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss