On Tue, May 5, 2020 at 11:57 AM Han Zhou <hz...@ovn.org> wrote: > > > > On Fri, May 1, 2020 at 2:14 PM Dan Winship <danwins...@redhat.com> wrote: > > > > On 5/1/20 12:37 PM, Girish Moodalbail wrote: > > > If we now look at table=12 (lr_in_arp_resolve) in the ingress pipeline > > > of Gateway Router-1, then you will see that there will be 2000 logical > > > flow entries... > > > > > In the topology above, the only intended path is North-South between > > > each gateway router and the logical router. There is no east-west > > > traffic between the gateway routers > > > > > Is there an another way to solve the above problem with just keeping the > > > single join logical switch? > > > > Two thoughts: > > > > 1. In openshift-sdn, the bridge doesn't try to handle ARP itself. It > > just lets ARP requests pass through normally, and lets ARP replies pass > > through normally as long as they are correct (ie, it doesn't let > > spoofing through). This means fewer flows but more traffic. Maybe that's > > the right tradeoff? > > > The 2M entries here is not for ARP responder, but more equivalent to the neighbour table (or ARP cache), on each LR. The ARP responder resides in the LS (join logical switch), which is O(n) instead of O(n^2), so it is not a problem here. > > However, a similar idea may works here to avoid the O(n^2) scale issue. For the neighbour table, actually OVN has two parts, one is statically build, which is the 2M entires mentioned in this case, and the other is the dynamic ARP resolve - the mac_binding table, which is dynamically populated by handling ARP messages. To solve the problem here, it is possible to change OVN to support configuring a LR to avoid static neighbour table, and relies only on dynamic ARP resolving. In this case, all the gateway routers can be configured as not using static ARP resolving, and eventually there will be only 2 entries (one for IPv4 and one for IPv6) for each gateway router in mac_binding table for the north-south traffic to the join router. (of source there will be still same amount of mac_bindings in each router for the external traffic on the other side of the gateway routers). > > This change seems straightforward, but I am not sure if there is any corner cases.
Hi Girish, I've sent a RFC patch here for the above proposal: https://patchwork.ozlabs.org/project/openvswitch/patch/1589614395-99499-1-git-send-email-hz...@ovn.org/ For this use case, just set options:dynamic_neigh_routes=true for all the Gateway Routers. Could you try it in your scale environment and see if it solves the problem? Thanks, Han > > > 2. In most places in ovn-kubernetes, our MAC addresses are > > programmatically related to the corresponding IP addresses, and in > > places where that's not currently true, we could try to make it true, > > and then perhaps the thousands of rules could just be replaced by a > > single rule? > > > This may be a good idea, but I am not sure how to implement in OVN to make it generic, since most OVN users can't make such assumption. > > On the other hand, why wouldn't splitting the join logical switch to 1000 LSes solve the problem? I understand that there will be 1000 more datapaths, and 1000 more LRPs, but these are all O(n), which is much more efficient than the O(n^2) exploding. What's the other scale issues created by this? > > In addition, Girish, for the external LS, I am not sure why can't it be shared, if all the nodes are connected to a single L2 network. (If they are connected to separate L2 networks, different external LSes should be created, at least according to current OVN model). > > Thanks, > Han
_______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
