On Sat, May 16, 2020 at 12:36 AM Han Zhou <[email protected]> wrote: > > > On Tue, May 5, 2020 at 11:57 AM Han Zhou <[email protected]> wrote: > > > > > > > > On Fri, May 1, 2020 at 2:14 PM Dan Winship <[email protected]> > wrote: > > > > > > On 5/1/20 12:37 PM, Girish Moodalbail wrote: > > > > If we now look at table=12 (lr_in_arp_resolve) in the ingress > pipeline > > > > of Gateway Router-1, then you will see that there will be 2000 > logical > > > > flow entries... > > > > > > > In the topology above, the only intended path is North-South between > > > > each gateway router and the logical router. There is no east-west > > > > traffic between the gateway routers > > > > > > > Is there an another way to solve the above problem with just keeping > the > > > > single join logical switch? > > > > > > Two thoughts: > > > > > > 1. In openshift-sdn, the bridge doesn't try to handle ARP itself. It > > > just lets ARP requests pass through normally, and lets ARP replies pass > > > through normally as long as they are correct (ie, it doesn't let > > > spoofing through). This means fewer flows but more traffic. Maybe > that's > > > the right tradeoff? > > > > > The 2M entries here is not for ARP responder, but more equivalent to the > neighbour table (or ARP cache), on each LR. The ARP responder resides in > the LS (join logical switch), which is O(n) instead of O(n^2), so it is not > a problem here. > > > > However, a similar idea may works here to avoid the O(n^2) scale issue. > For the neighbour table, actually OVN has two parts, one is statically > build, which is the 2M entires mentioned in this case, and the other is the > dynamic ARP resolve - the mac_binding table, which is dynamically populated > by handling ARP messages. To solve the problem here, it is possible to > change OVN to support configuring a LR to avoid static neighbour table, and > relies only on dynamic ARP resolving. In this case, all the gateway routers > can be configured as not using static ARP resolving, and eventually there > will be only 2 entries (one for IPv4 and one for IPv6) for each gateway > router in mac_binding table for the north-south traffic to the join router. > (of source there will be still same amount of mac_bindings in each router > for the external traffic on the other side of the gateway routers). > > > > This change seems straightforward, but I am not sure if there is any > corner cases. > > Hi Girish, > > I've sent a RFC patch here for the above proposal: > https://patchwork.ozlabs.org/project/openvswitch/patch/[email protected]/ > For this use case, just set options:dynamic_neigh_routes=true for all the > Gateway Routers. Could you try it in your scale environment and see if it > solves the problem? > > Thanks, > Han > > > > > > 2. In most places in ovn-kubernetes, our MAC addresses are > > > programmatically related to the corresponding IP addresses, and in > > > places where that's not currently true, we could try to make it true, > > > and then perhaps the thousands of rules could just be replaced by a > > > single rule? > > > > > This may be a good idea, but I am not sure how to implement in OVN to > make it generic, since most OVN users can't make such assumption. > > > > On the other hand, why wouldn't splitting the join logical switch to > 1000 LSes solve the problem? I understand that there will be 1000 more > datapaths, and 1000 more LRPs, but these are all O(n), which is much more > efficient than the O(n^2) exploding. What's the other scale issues created > by this? > > > > In addition, Girish, for the external LS, I am not sure why can't it be > shared, if all the nodes are connected to a single L2 network. (If they are > connected to separate L2 networks, different external LSes should be > created, at least according to current OVN model). >
Thanks Han for the patch. Will give it a try and let you know. Regards, ~Girish > > > > Thanks, > > Han >
_______________________________________________ discuss mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
