Re: [ovs-discuss] [OVN] Packets flooded when using VLAN backed networks

2020-09-17 Thread Ankur Sharma 
Hi Daniel,

Glad to see your reply.
Yes, we need to do something about it. I will try to come back with some 
patches (or IRC discussions) and we will discuss the same further.

Thanks

Regards,
Ankur

From: Daniel Alvarez Sanchez 
Sent: Thursday, September 17, 2020 3:11 AM
To: Ankur Sharma 
Cc: ovs-discuss 
Subject: Re: [OVN] Packets flooded when using VLAN backed networks

Thanks Ankur for your reply!

On Sat, Sep 12, 2020 at 4:27 AM Ankur Sharma 
mailto:ankur.sha...@nutanix.com>> wrote:
Hi Daniel,

Thanks a lot for starting the thread.
Yes, you have a valid observation.

"The reason is that, as we translate the eth.src to that of the 
"ovn-chassis-mac-mappings", the ToR will never see a packet whose eth.src is 
either vm1 or vm3 so it'll never learn their addresses and flood the traffic to 
all ports."

The main reason is not chassis-mac-mappings, but the way OVN LR works.

Yes, that's right! Sorry for the confusion. Even with the 
'reside-on-redirect-chassis' option we'd be observing this as well.

TOPOLOGY:
LS1  LR LS2

VM1--LS1---CHASSIS1
VM2--LS2---CHASSIS2


Let us say we are pinging from VM1 --> VM2.

a. For a routed traffic , we will never have source mac as VM mac, i.e once 
traffic is routed then, source mac will be replaced.
b. The reason flooding is observed is because of following:
i. A typical router will send ARP for the destination endpoint (VM2 in the 
example above) and the corresponding reply will cause initial learning of VM 
MAC (VM2 mac in the example above).
   ii. Similarly, after initial ARP resolution, it will do periodic ARP refresh 
by generating an ARP request (could be a unicast ARP) and again the reply will 
ensure that MAC entry in TOR is refreshed.
c. Now, OVN LR may not send out ARP request on the wire (because of ARP 
suppression) and even if it does (let us say port just has mac and not the IP), 
then ARP response is not sent on the wire.
d. Similarly, since there is no periodic ARP refresh, hence even if somehow 
initial learning happens, the MAC will eventually age out.

We can do either of following:
a. Assign a chassis to do periodic ARP refreshes.
i. I started some work around this, but looked tricky, i.e besides sending 
the ARP refresh, then we should also remove the ARP cache entry for which 
certain threshhold of ARP refresh is failing.
b. Periodic RARP advertisement of endpoints on localnet logical switches.


And like you said, we can use this thread to converge.

Thanks Ankur!  I don't have a strong opinion of either of the above, but it 
looks like we want to do something about this (maybe configurable) to avoid 
situations where we flood large amounts of traffic (for example in long lived 
connections or bulk transfers or ...)

Thanks

Regards,
Ankur


From: Daniel Alvarez Sanchez mailto:dalva...@redhat.com>>
Sent: Friday, September 11, 2020 8:57 AM
To: ovs-discuss 
mailto:ovs-discuss@openvswitch.org>>
Cc: Ankur Sharma mailto:ankur.sha...@nutanix.com>>
Subject: [OVN] Packets flooded when using VLAN backed networks

Hi folks,

This is probably not a bug and not sure if much can be done about it but 
thought of raising it here for discussion.

I have deployed a simple topology with two logical switches (VLAN backed 
network), a logical router and a couple of VMs. When pinging between the 
logical switches, all the traffic is flooded in the upstream switch.

Example using this logical [0] and physical [1] topologies, when pinging from 
vm3 (worker2) to vm1 (worker1) and capturing traffic on host1:

15:50:18.790323 1e:02:ad:bb:aa:dd > 40:44:00:00:00:01, ethertype 802.1Q 
(0x8100), length 102: vlan 190, p 0, ethertype IPv4, (tos 0x0, ttl 63, id 
47366, offset 0, flags [DF], proto ICMP (1), length 84)
192.168.1.13 > 192.168.0.11 
[192.168.0.11]:
 ICMP echo request, id 1671, seq 11, length 64
15:50:18.790428 1e:02:ad:bb:aa:77 > 40:44:33:00:00:03, ethertype 802.1Q 
(0x8100), length 102: vlan 170, p 0, ethertype IPv4, (tos 0x0, ttl 63, id 
44948, offset 0, flags [none], proto ICMP (1), length 84)
192.168.0.11 > 192.168.1.13 
[192.168.1.13]:
 ICMP echo reply, id 1671, seq 11, length 64

The reason is that, as we translate the eth.src to that of the 
"ovn-chassis-mac-mappings", the ToR will never see a packet whose eth.src is 
either vm1 or vm3 so it'll never learn their addresses and flood the traffic to 
all ports.

In the example above:

[root@worker1 ~]# ovs-vsctl get open . external_ids:ovn-chassis-mac-mappings

Re: [ovs-discuss] [OVN] Packets flooded when using VLAN backed networks

2020-09-17 Thread Daniel Alvarez Sanchez 
Thanks Ankur for your reply!

On Sat, Sep 12, 2020 at 4:27 AM Ankur Sharma 
wrote:

> Hi Daniel,
>
> Thanks a lot for starting the thread.
> Yes, you have a valid observation.
>
> "The reason is that, as we translate the eth.src to that of the
> "ovn-chassis-mac-mappings", the ToR will never see a packet whose eth.src
> is either vm1 or vm3 so it'll never learn their addresses and flood the
> traffic to all ports."
>
> The main reason is not chassis-mac-mappings, but the way OVN LR works.
>

Yes, that's right! Sorry for the confusion. Even with the
'reside-on-redirect-chassis' option we'd be observing this as well.

>
> TOPOLOGY:
> LS1  LR LS2
>
> VM1--LS1---CHASSIS1
> VM2--LS2---CHASSIS2
>
>
> Let us say we are pinging from VM1 --> VM2.
>
> a. For a routed traffic , we will never have source mac as VM mac, i.e
> once traffic is routed then, source mac will be replaced.
> b. The reason flooding is observed is because of following:
> i. A typical router will send ARP for the destination endpoint (VM2 in
> the example above) and the corresponding reply will cause initial learning
> of VM MAC (VM2 mac in the example above).
>ii. Similarly, after initial ARP resolution, it will do periodic ARP
> refresh by generating an ARP request (could be a unicast ARP) and again the
> reply will ensure that MAC entry in TOR is refreshed.
> c. Now, OVN LR may not send out ARP request on the wire (because of ARP
> suppression) and even if it does (let us say port just has mac and not the
> IP), then ARP response is not sent on the wire.
> d. Similarly, since there is no periodic ARP refresh, hence even if
> somehow initial learning happens, the MAC will eventually age out.
>
> We can do either of following:
> a. Assign a chassis to do periodic ARP refreshes.
> i. I started some work around this, but looked tricky, i.e besides
> sending the ARP refresh, then we should also remove the ARP cache entry for
> which certain threshhold of ARP refresh is failing.
> b. Periodic RARP advertisement of endpoints on localnet logical switches.
>

>
> And like you said, we can use this thread to converge.
>
> Thanks Ankur!  I don't have a strong opinion of either of the above, but
it looks like we want to do something about this (maybe configurable) to
avoid situations where we flood large amounts of traffic (for example in
long lived connections or bulk transfers or ...)

>
> Thanks
>
> Regards,
> Ankur
>
> --
> *From:* Daniel Alvarez Sanchez 
> *Sent:* Friday, September 11, 2020 8:57 AM
> *To:* ovs-discuss 
> *Cc:* Ankur Sharma 
> *Subject:* [OVN] Packets flooded when using VLAN backed networks
>
> Hi folks,
>
> This is probably not a bug and not sure if much can be done about it but
> thought of raising it here for discussion.
>
> I have deployed a simple topology with two logical switches (VLAN backed
> network), a logical router and a couple of VMs. When pinging between the
> logical switches, all the traffic is flooded in the upstream switch.
>
> Example using this logical [0] and physical [1] topologies, when pinging
> from vm3 (worker2) to vm1 (worker1) and capturing traffic on host1:
>
> 15:50:18.790323 1e:02:ad:bb:aa:dd > 40:44:00:00:00:01, ethertype 802.1Q
> (0x8100), length 102: vlan 190, p 0, ethertype IPv4, (tos 0x0, ttl 63, id
> 47366, offset 0, flags [DF], proto ICMP (1), length 84)
> 192.168.1.13 > 192.168.0.11 [192.168.0.11]
> :
> ICMP echo request, id 1671, seq 11, length 64
> 15:50:18.790428 1e:02:ad:bb:aa:77 > 40:44:33:00:00:03, ethertype 802.1Q
> (0x8100), length 102: vlan 170, p 0, ethertype IPv4, (tos 0x0, ttl 63, id
> 44948, offset 0, flags [none], proto ICMP (1), length 84)
> 192.168.0.11 > 192.168.1.13 [192.168.1.13]
> :
> ICMP echo reply, id 1671, seq 11, length 64
>
> The reason is that, as we translate the eth.src to that of the
> "ovn-chassis-mac-mappings", the ToR will never see a packet whose eth.src
> is either vm1 or vm3 so it'll never learn their addresses and flood the
> traffic to all ports.
>
> In the example above:
>
> [root@worker1 ~]# ovs-vsctl get open .
> external_ids:ovn-chassis-mac-mappings
> "tenant:1e:02:ad:bb:aa:77"
>
> [root@worker2 vagrant]# ovs-vsctl get open .
> external_ids:ovn-chassis-mac-mappings
> "tenant:1e:02:ad:bb:aa:dd"
>
> I understand that the benefit of using the ovn-chassis-mac-mappings is the
> distributed routing capabilities but I wonder if we could come up with a
> way of avoiding the flood.
>
> In case somebody's interested in replicating this scenario, you can find