On Tue, Mar 11, 2025 at 12:47 PM Dumitru Ceara <dce...@redhat.com> wrote: > > On 2/28/25 10:41 PM, num...@ovn.org wrote: > > From: Numan Siddique <num...@ovn.org> > > > > Consider the below logical topology > > > > sw0-p1 - > > | > > sw0-p2 - -> sw0 -> lr0 ---- > > ... | | > > sw0-pn - | > > | > > sw1-p1 - | > > | | > > sw1p-2 - -> sw1 -> lr1 ---- --- public (provider switch) > > ... | | > > sw1-pn- | > > | > > swn-p1 - | > > | | > > swn-p2- -> swn -> lrn ---- > > ... | > > swn-pn - > > > > All the routers are connected to the provider switch via > > a ditributed gateway port. > > > > If sw0-p1 is resident on the chassis C1, then since there is a path > > to all the switches and the routers, ovn-controller will add all > > these datapaths to its 'local_datapaths' map. This in turn results > > in processing all the logical flows and installing all the openflows > > and in turn wasting the CPU time. This can be very costly in > > a highly scaled deployment. > > > > Previous commit sets a flag "only_dgp_peer_ports" in the SB Datapath > > binding for a provider switch (with only dgp peer ports). > > > > In this commit, ovn-controller makes use of this flag and stops > > adding other datapaths connected to the public provider switch > > to the 'local_datapaths'. > > > > For example, when it claims sw0-p1, it adds sw0, lr0 and public > > to the local_datapaths and stops there. If it later claims > > sw1-p1, it will add sw1 and lr1. > > > > This reduces the recompute time and the number of openflow rules > > added to ovs-vswitchd significantly. > > > > I tested this patch with a deployment of below logical resources: > > > > No of logical switches - 778 > > No of logical routers - 871 > > No of logical flows - 85626 > > No of 'ovn-sbctl dump-flows' - 208631 > > > > Without this patch, afte claiming sw0-p1, ovn-controller adds > > 269098 openflow rules and it takes approx 2500 milli seconds > > for a recompute. > > > > With this patch, after claiming sw0-p1, ovn-controller adds > > 21350 openflow rules and it takes approx 280 milli seconds > > for a recompute. > > > > There is approx 90% reduction in the openflow rules and > > 88% reduction in recompute time when a comoute node has > > VIFs from one logical switch. > > > > Signed-off-by: Numan Siddique <num...@ovn.org> > > --- > > Hi Numan, > > This is a very interesting optimization! I didn't review the tests for > now and I only looked at the code so I didn't do much testing in general > yet. > > I do have some comments, please see below. > > I think what would make it easier would be (as mentioned inline) to > split this rather large change into smaller, easier to review patches, e.g.: > - a patch to add the debug/dump-local-datapaths command > - a patch to add the helper to get the cr-lrp port binding (and use it > in other places in the code base too) > - a patch to handle addition of datapaths with the new > "only_dgp_peer_ports" flag > - a patch to handle deletion of "not-anymore-relevant-locally" datapaths > > What do you think?
Hi Dumitru, Thanks for the reviews. Sounds good to me. I'll reply to the comments below in a separate email. I've submitted a small series here - https://patchwork.ozlabs.org/project/ovn/list/?series=448321 which adds the debug/dump-local-datapaths command and adds the helper function to get the cr-lrp. Can you please take a look at them? I think they can be considered independently of this patch series. I'll work on addressing your review comments and if they are ready soon, I'll merge the refactor series in the v3. Thanks Numan > > > controller/binding.c | 242 ++++++++-- > > controller/binding.h | 2 + > > controller/local_data.c | 84 +++- > > controller/local_data.h | 6 + > > controller/lport.c | 12 + > > controller/lport.h | 4 + > > controller/ovn-controller.c | 38 ++ > > tests/multinode.at | 185 +++++++- > > tests/ovn-performance.at | 6 +- > > tests/ovn.at | 853 ++++++++++++++++++++++++++++++++++++ > > 10 files changed, 1392 insertions(+), 40 deletions(-) > > > > diff --git a/controller/binding.c b/controller/binding.c > > index c76a0c06c5..9521240bb9 100644 > > --- a/controller/binding.c > > +++ b/controller/binding.c > > @@ -824,6 +824,16 @@ static bool binding_lport_update_port_sec( > > static bool ovs_iface_matches_lport_iface_id_ver( > > const struct ovsrec_interface *, > > const struct sbrec_port_binding *); > > +static bool cleanup_patch_port_local_dps( > > + const struct sbrec_port_binding *, const struct sbrec_port_binding > > *cr_pb, > > + const struct sbrec_port_binding *peer, struct local_datapath *ld, > > + struct binding_ctx_in *b_ctx_in, > > + struct binding_ctx_out *b_ctx_out, > > + bool *cleanup); > > +static bool local_datapath_is_relevant( > > + struct local_datapath *, struct local_datapath *ignore_peer_ld, > > + struct hmap *local_datapaths, int *depth, const struct sbrec_chassis *, > > + struct ovsdb_idl_index *); > > > > void > > related_lports_init(struct related_lports *rp) > > @@ -1128,6 +1138,19 @@ binding_dump_local_bindings(struct > > local_binding_data *lbinding_data, > > free(nodes); > > } > > > > +void > > +binding_dump_local_datapaths(struct hmap *local_datapaths, > > Nit: const struct hmap * > > > + struct ds *out_data) > > +{ > > + ds_put_cstr(out_data, "Local datapaths:\n"); > > + struct local_datapath *ld; > > + HMAP_FOR_EACH (ld, hmap_node, local_datapaths) { > > + ds_put_format(out_data, "Datapath: %s, type: %s\n", > > + smap_get(&ld->datapath->external_ids, "name"), > > + ld->is_switch ? "switch" : "router"); > > + } > > +} > > + > > void > > set_pb_chassis_in_sbrec(const struct sbrec_port_binding *pb, > > const struct sbrec_chassis *chassis_rec, > > @@ -2144,7 +2167,9 @@ build_local_bindings(struct binding_ctx_in *b_ctx_in, > > > > static bool consider_patch_port_for_local_datapaths( > > const struct sbrec_port_binding *, > > - struct binding_ctx_in *, struct binding_ctx_out *); > > + const struct sbrec_port_binding *cr_pb, > > + struct binding_ctx_in *, struct binding_ctx_out *, > > + bool check_and_remove_localdps); > > > > void > > binding_run(struct binding_ctx_in *b_ctx_in, struct binding_ctx_out > > *b_ctx_out) > > @@ -2189,7 +2214,8 @@ binding_run(struct binding_ctx_in *b_ctx_in, struct > > binding_ctx_out *b_ctx_out) > > switch (lport_type) { > > case LP_PATCH: > > update_related_lport(pb, b_ctx_out); > > - consider_patch_port_for_local_datapaths(pb, b_ctx_in, > > b_ctx_out); > > + consider_patch_port_for_local_datapaths(pb, NULL, b_ctx_in, > > + b_ctx_out, false); > > break; > > > > case LP_VTEP: > > @@ -2245,6 +2271,9 @@ binding_run(struct binding_ctx_in *b_ctx_in, struct > > binding_ctx_out *b_ctx_out) > > struct lport *lnet_lport = xmalloc(sizeof *lnet_lport); > > lnet_lport->pb = pb; > > ovs_list_push_back(&localnet_lports, &lnet_lport->list_node); > > + if (pb->chassis == b_ctx_in->chassis_rec) { > > + sbrec_port_binding_set_chassis(pb, NULL); > > + } > > break; > > } > > > > @@ -2574,7 +2603,6 @@ consider_iface_release(const struct ovsrec_interface > > *iface_rec, > > if_status_mgr_remove_ovn_installed(b_ctx_out->if_mgr, > > lbinding->iface); > > } > > - > > } else if (b_lport && b_lport->type == LP_LOCALPORT) { > > /* lbinding is associated with a localport. Remove it from the > > * related lports. */ > > @@ -2964,12 +2992,27 @@ handle_updated_vif_lport(const struct > > sbrec_port_binding *pb, > > > > static bool > > consider_patch_port_for_local_datapaths(const struct sbrec_port_binding > > *pb, > > + const struct sbrec_port_binding > > *cr_pb, > > struct binding_ctx_in *b_ctx_in, > > - struct binding_ctx_out *b_ctx_out) > > + struct binding_ctx_out *b_ctx_out, > > + bool check_and_remove_localdps) > > { > > - struct local_datapath *ld = > > - get_local_datapath(b_ctx_out->local_datapaths, > > - pb->datapath->tunnel_key); > > + const struct sbrec_port_binding *peer; > > + struct local_datapath *peer_ld = NULL; > > + struct local_datapath *ld = NULL; > > + > > + ld = get_local_datapath(b_ctx_out->local_datapaths, > > + pb->datapath->tunnel_key); > > + if (ld && ld->has_only_dgp_peer_ports) { > > + /* Nothing much to do. */ > > + return true; > > + } > > + > > + peer = lport_get_peer(pb, b_ctx_in->sbrec_port_binding_by_name); > > + if (peer) { > > + peer_ld = get_local_datapath(b_ctx_out->local_datapaths, > > + peer->datapath->tunnel_key); > > + } > > > > if (!ld) { > > /* If 'ld' for this lport is not present, then check if > > @@ -2977,17 +3020,9 @@ consider_patch_port_for_local_datapaths(const struct > > sbrec_port_binding *pb, > > * and peer's datapath is already in the local datapaths, > > * then add this lport's datapath to the local_datapaths. > > * */ > > - const struct sbrec_port_binding *peer; > > - struct local_datapath *peer_ld = NULL; > > - peer = lport_get_peer(pb, b_ctx_in->sbrec_port_binding_by_name); > > - if (peer) { > > - peer_ld = > > - get_local_datapath(b_ctx_out->local_datapaths, > > - peer->datapath->tunnel_key); > > - } > > - if (peer_ld && need_add_peer_to_local( > > - b_ctx_in->sbrec_port_binding_by_name, peer, > > - b_ctx_in->chassis_rec)) { > > + if (peer_ld && !peer_ld->has_only_dgp_peer_ports && > > + need_add_peer_to_local(b_ctx_in->sbrec_port_binding_by_name, > > peer, > > + b_ctx_in->chassis_rec)) { > > ld = add_local_datapath( > > b_ctx_in->sbrec_datapath_binding_by_key, > > b_ctx_in->sbrec_port_binding_by_datapath, > > @@ -3000,7 +3035,7 @@ consider_patch_port_for_local_datapaths(const struct > > sbrec_port_binding *pb, > > /* Add the peer datapath to the local datapaths if it's > > * not present yet. > > */ > > - if (need_add_peer_to_local( > > + if (peer && need_add_peer_to_local( > > b_ctx_in->sbrec_port_binding_by_name, pb, > > b_ctx_in->chassis_rec)) { > > add_local_datapath_peer_port( > > @@ -3011,6 +3046,18 @@ consider_patch_port_for_local_datapaths(const struct > > sbrec_port_binding *pb, > > ld, b_ctx_out->local_datapaths, > > b_ctx_out->tracked_dp_bindings); > > } > > + > > + if (check_and_remove_localdps) { > > + bool cleanedup = false; > > + if (!cleanup_patch_port_local_dps(pb, cr_pb, peer, ld, > > b_ctx_in, > > + b_ctx_out, &cleanedup)) { > > + return false; > > + } > > + > > + if (cleanedup) { > > + ld = NULL; > > Can we actually continue and claim ports at this point? The code just > below will try to claim a port that's requested for this datapath that > might have just been marked as non-local (we call this during > incremental processing too when LP_PATCH ports change and we could be > handling multiple port changes at once). > > If we set ld to NULL here we'll still call claim_lport() if > > /* If this chassis is requested - try to claim. */ > if (pb->requested_chassis == b_ctx_in->chassis_rec) { > return claim_lport(pb, ld, NULL, b_ctx_in->chassis_rec, NULL, > .... > } > > but we didn't add the pb->datapath to the set of local datapaths. That > sounds wrong. > > > + } > > + } > > } > > > > /* If this chassis is requested - try to claim. */ > > @@ -3029,12 +3076,10 @@ consider_patch_port_for_local_datapaths(const > > struct sbrec_port_binding *pb, > > || if_status_is_port_claimed(b_ctx_out->if_mgr, pb->logical_port)) > > { > > > > remove_local_lports(pb->logical_port, b_ctx_out); > > - if (!release_lport(pb, ld, b_ctx_in->chassis_rec, > > - !b_ctx_in->ovnsb_idl_txn, > > - b_ctx_out->tracked_dp_bindings, > > - b_ctx_out->if_mgr)) { > > - return false; > > - } > > + return release_lport(pb, ld, b_ctx_in->chassis_rec, > > + !b_ctx_in->ovnsb_idl_txn, > > + b_ctx_out->tracked_dp_bindings, > > + b_ctx_out->if_mgr); > > } > > return true; > > } > > @@ -3094,8 +3139,8 @@ handle_updated_port(struct binding_ctx_in *b_ctx_in, > > > > case LP_PATCH: > > update_related_lport(pb, b_ctx_out); > > - handled = consider_patch_port_for_local_datapaths(pb, b_ctx_in, > > - b_ctx_out); > > + handled = consider_patch_port_for_local_datapaths(pb, NULL, > > b_ctx_in, > > + b_ctx_out, true); > > break; > > > > case LP_VTEP: > > @@ -3138,8 +3183,8 @@ handle_updated_port(struct binding_ctx_in *b_ctx_in, > > break; > > } > > handled = consider_patch_port_for_local_datapaths(distributed_pb, > > - b_ctx_in, > > - b_ctx_out); > > + pb, b_ctx_in, > > + b_ctx_out, true); > > break; > > > > case LP_EXTERNAL: > > @@ -3912,3 +3957,142 @@ binding_destroy(void) > > shash_destroy_free_data(&_qos_ports); > > sset_clear(&_postponed_ports); > > } > > + > > +static bool > > +is_patch_pb_chassis_relevant( > > + const struct sbrec_port_binding *pb, > > + const struct sbrec_chassis *chassis, > > + struct ovsdb_idl_index *sbrec_port_binding_by_name) > > +{ > > + if (ha_chassis_group_contains(pb->ha_chassis_group, chassis)) { > > + return true; > > + } > > + > > + const struct sbrec_port_binding *pb_crp = > > + lport_get_cr_port(sbrec_port_binding_by_name, pb); > > + if (pb_crp) { > > + return ha_chassis_group_contains(pb_crp->ha_chassis_group, > > chassis); > > + } > > + > > + return false; > > +} > > + > > +static bool > > +cleanup_patch_port_local_dps(const struct sbrec_port_binding *pb, > > + const struct sbrec_port_binding *cr_pb, > > + const struct sbrec_port_binding *peer, > > + struct local_datapath *ld, > > + struct binding_ctx_in *b_ctx_in, > > + struct binding_ctx_out *b_ctx_out, > > + bool *cleanedup) > > Shouldn't we add the cleanup of local datapath as a separate patch? > > The current patch is about optimizing how we add local datapaths. In > general, even without your series, when all local ports of a datapath > are unbound we don't incrementally remove datapaths that have become > non-local. > > It's a tricky area and I'd prefer if we could review it in smaller chunks. > > > +{ > > + *cleanedup = false; > > + if (!peer) { > > + /* Remove 'pb' from the ld's peer ports as it has no peer. */ > > + remove_local_datapath_peer_port(pb, ld, > > + b_ctx_out->local_datapaths); > > + } > > + > > + /* We can consider removing the 'ld' of the patch port 'pb' from the > > + * local datapaths, if all the below conditions are met > > + * - 'pb' doesn't have a peer or ld' is a router datapath > > + * - if 'pb' is a distributed gateway port (dgp), then > > + * its chassisredirect port's ha chassis group doesn't > > + * contain our 'chassis rec' > > + * - and finally 'ld' is not relevant any more. See > > + * local_datapath_is_relevant() for more details. > > + * > > + * Note: If 'ld' can be removed, then all its connected local datapaths > > + * can also be removed. > > + * > > + * For example, if we had sw1-port1 -> sw1 -> lr1 -> sw2 and if > > + * sw1-port1 resides on this chassis, and if the link between sw1 and > > + * lr1 is broken, then we can remove lr1 and sw2 from the > > + * local_datapaths. > > + * */ > > + > > + bool consider_ld_for_removal = !peer || !ld->is_switch; > > + if (consider_ld_for_removal && cr_pb) { > > + consider_ld_for_removal = !ha_chassis_group_contains( > > + cr_pb->ha_chassis_group, b_ctx_in->chassis_rec); > > + } > > + > > + if (!consider_ld_for_removal) { > > + return true; > > + } > > + > > + int depth = 0; > > + > > + bool is_relevant = local_datapath_is_relevant( > > + ld, NULL, b_ctx_out->local_datapaths, > > + &depth, b_ctx_in->chassis_rec, > > + b_ctx_in->sbrec_port_binding_by_name); > > + > > + if (depth >= 100) { > > + /* datapaths are too deeply nested. Fall back to recompute. */ > > + return false; > > + } > > + > > + if (!is_relevant) { > > + /* This 'ld' can be removed from the local datapaths as > > + * - its a router datapath and > > + * - it has no peers locally. */ > > + local_datapath_remove_and_destroy(ld, b_ctx_out->local_datapaths, > > + b_ctx_out->tracked_dp_bindings); > > + *cleanedup = true; > > + } > > + > > + return true; > > +} > > + > > +static bool > > +local_datapath_is_relevant(struct local_datapath *ld, > > + struct local_datapath *ignore_peer_ld, > > + struct hmap *local_datapaths, int *depth, > > + const struct sbrec_chassis *chassis, > > + struct ovsdb_idl_index *sbrec_pb_by_name) > > +{ > > + if (!sset_is_empty(&ld->claimed_lports) || > > + !shash_is_empty(&ld->external_ports) || > > + !shash_is_empty(&ld->multichassis_ports) || > > + ld->vtep_port) { > > + return true; > > + } > > + > > + bool relevant = false; > > + > > + if (*depth >= 100) { > > + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); > > + VLOG_WARN_RL(&rl, "datapaths nested too deep"); > > + return true; > > + } > > + > > + for (size_t i = 0; i < ld->n_peer_ports && !relevant; i++) { > > + const struct sbrec_port_binding *remote = ld->peer_ports[i].remote; > > + const struct sbrec_port_binding *local = ld->peer_ports[i].local; > > + > > + if (is_patch_pb_chassis_relevant(local, chassis, > > + sbrec_pb_by_name)) { > > + return true; > > + } > > + > > + if (is_patch_pb_chassis_relevant(remote, chassis, > > + sbrec_pb_by_name)) { > > + return true; > > + } > > + > > + struct local_datapath *peer_ld; > > + uint32_t remote_peer_ld_key; > > + remote_peer_ld_key = > > ld->peer_ports[i].remote->datapath->tunnel_key; > > + peer_ld = get_local_datapath(local_datapaths, remote_peer_ld_key); > > + if (peer_ld && !peer_ld->has_only_dgp_peer_ports && > > + peer_ld != ignore_peer_ld) { > > + *depth = *depth + 1; > > + relevant = local_datapath_is_relevant(peer_ld, ld, > > + local_datapaths, depth, > > + chassis, > > sbrec_pb_by_name); > > We don't really need the 'relevant' variable if we change this to: > > if (local_datapath_is_relevant(....)) { > return true; > } > > or am I missing something? > > > + } > > + } > > + > > + return relevant; > > +} > > diff --git a/controller/binding.h b/controller/binding.h > > index d13ae36c79..a4346c3e10 100644 > > --- a/controller/binding.h > > +++ b/controller/binding.h > > @@ -201,6 +201,8 @@ bool binding_handle_port_binding_changes(struct > > binding_ctx_in *, > > void binding_tracked_dp_destroy(struct hmap *tracked_datapaths); > > > > void binding_dump_local_bindings(struct local_binding_data *, struct ds *); > > +void binding_dump_local_datapaths(struct hmap *local_datapaths, > > + struct ds *out_data); > > > > void binding_dump_related_lports(struct related_lports *related_lports, > > struct ds *); > > diff --git a/controller/local_data.c b/controller/local_data.c > > index 24e871f639..2d493c4de4 100644 > > --- a/controller/local_data.c > > +++ b/controller/local_data.c > > @@ -53,6 +53,13 @@ static struct tracked_datapath *tracked_datapath_create( > > > > static bool datapath_is_switch(const struct sbrec_datapath_binding *); > > static bool datapath_is_transit_switch(const struct sbrec_datapath_binding > > *); > > +static bool datapath_has_only_dgp_peer_ports( > > + const struct sbrec_datapath_binding *); > > +static void local_datapath_remove_and_destroy__( > > + struct local_datapath *ld, > > + const struct sbrec_port_binding *ignore_peer_port, > > + struct hmap *local_datapaths, > > + struct hmap *tracked_datapaths); > > > > static uint64_t local_datapath_usage; > > > > @@ -86,6 +93,7 @@ local_datapath_alloc(const struct sbrec_datapath_binding > > *dp) > > ld->datapath = dp; > > ld->is_switch = datapath_is_switch(dp); > > ld->is_transit_switch = datapath_is_transit_switch(dp); > > + ld->has_only_dgp_peer_ports = datapath_has_only_dgp_peer_ports(dp); > > shash_init(&ld->external_ports); > > shash_init(&ld->multichassis_ports); > > sset_init(&ld->claimed_lports); > > @@ -132,6 +140,14 @@ local_datapath_destroy(struct local_datapath *ld) > > free(ld); > > } > > > > +void local_datapath_remove_and_destroy(struct local_datapath *ld, > > + struct hmap *local_datapaths, > > + struct hmap *tracked_datapaths) > > +{ > > + local_datapath_remove_and_destroy__(ld, NULL, local_datapaths, > > + tracked_datapaths); > > +} > > + > > /* Checks if pb is running on local gw router or pb is a patch port > > * and the peer datapath should be added to local datapaths. */ > > bool > > @@ -226,12 +242,12 @@ add_local_datapath_peer_port( > > get_local_datapath(local_datapaths, > > peer->datapath->tunnel_key); > > if (!peer_ld) { > > - add_local_datapath__(sbrec_datapath_binding_by_key, > > - sbrec_port_binding_by_datapath, > > - sbrec_port_binding_by_name, 1, > > - peer->datapath, chassis, local_datapaths, > > - tracked_datapaths); > > - return; > > + peer_ld = add_local_datapath__(sbrec_datapath_binding_by_key, > > + sbrec_port_binding_by_datapath, > > + sbrec_port_binding_by_name, 1, > > + peer->datapath, chassis, > > + local_datapaths, > > + tracked_datapaths); > > } > > > > local_datapath_peer_port_add(peer_ld, peer, pb); > > @@ -618,6 +634,17 @@ add_local_datapath__(struct ovsdb_idl_index > > *sbrec_datapath_binding_by_key, > > tracked_datapaths); > > } > > > > + if (ld->has_only_dgp_peer_ports) { > > + /* If this flag is set, it means this 'switch' datapath has > > + * - one ore many localnet ports. > > + * - all the router ports it is connected to are > > + * distributed gateway ports (DGPs). > > + * There is no need to add the routers of the dgps to > > + * the local datapaths. > > + * */ > > + return ld; > > + } > > + > > if (depth >= 100) { > > static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); > > VLOG_WARN_RL(&rl, "datapaths nested too deep"); > > @@ -710,6 +737,13 @@ datapath_is_transit_switch(const struct > > sbrec_datapath_binding *ldp) > > return smap_get(&ldp->external_ids, "interconn-ts") != NULL; > > } > > > > +static bool > > +datapath_has_only_dgp_peer_ports(const struct sbrec_datapath_binding *ldp) > > +{ > > + return datapath_is_switch(ldp) && > > + smap_get_bool(&ldp->external_ids, "only_dgp_peer_ports", false); > > +} > > + > > bool > > lb_is_local(const struct sbrec_load_balancer *sbrec_lb, > > const struct hmap *local_datapaths) > > @@ -750,3 +784,41 @@ lb_is_local(const struct sbrec_load_balancer *sbrec_lb, > > > > return false; > > } > > + > > +static void > > +local_datapath_remove_and_destroy__(struct local_datapath *ld, > > + const struct sbrec_port_binding > > *ignore_pb, > > + struct hmap *local_datapaths, > > + struct hmap *tracked_datapaths) > > +{ > > + for (size_t i = 0; i < ld->n_peer_ports; i++) { > > + const struct sbrec_port_binding *remote = ld->peer_ports[i].remote; > > + const struct sbrec_port_binding *local = ld->peer_ports[i].local; > > + > > + if (local == ignore_pb) { > > + continue; > > + } > > + > > + struct local_datapath *peer_ld; > > + uint32_t remote_peer_ld_key; > > + > > + remote_peer_ld_key = > > ld->peer_ports[i].remote->datapath->tunnel_key; > > + peer_ld = get_local_datapath(local_datapaths, remote_peer_ld_key); > > + if (peer_ld && !peer_ld->has_only_dgp_peer_ports) { > > + local_datapath_remove_and_destroy__(peer_ld, remote, > > + local_datapaths, > > + tracked_datapaths); > > + } else if (peer_ld && peer_ld->has_only_dgp_peer_ports) { > > + remove_local_datapath_peer_port(ld->peer_ports[i].remote, > > + peer_ld, local_datapaths); > > + } > > + } > > + > > + hmap_remove(local_datapaths, &ld->hmap_node); > > + if (tracked_datapaths) { > > + tracked_datapath_add(ld->datapath, TRACKED_RESOURCE_REMOVED, > > + tracked_datapaths); > > + } > > + > > + local_datapath_destroy(ld); > > +} > > diff --git a/controller/local_data.h b/controller/local_data.h > > index d2eb33b1eb..576ca8d56d 100644 > > --- a/controller/local_data.h > > +++ b/controller/local_data.h > > @@ -46,6 +46,8 @@ struct local_datapath { > > const struct sbrec_datapath_binding *datapath; > > bool is_switch; > > bool is_transit_switch; > > + /* Valid only for 'is_switch' local datapath. */ > > Nit: If we'd use "pure_provider_switch" for the option name, this would > be clear without the need of a comment. :) > > > + bool has_only_dgp_peer_ports; > > > > /* The localnet port in this datapath, if any (at most one is > > allowed). */ > > const struct sbrec_port_binding *localnet_port; > > @@ -91,6 +93,10 @@ struct local_datapath * add_local_datapath( > > > > void local_datapaths_destroy(struct hmap *local_datapaths); > > void local_datapath_destroy(struct local_datapath *ld); > > +void local_datapath_remove_and_destroy(struct local_datapath *, > > + struct hmap *local_datapaths, > > + struct hmap *tracked_datapaths); > > This is the counterpart of add_local_datapath() right? Should we call > it remove_local_datapath() instead? > > > + > > void add_local_datapath_peer_port( > > const struct sbrec_port_binding *, > > const struct sbrec_chassis *, > > diff --git a/controller/lport.c b/controller/lport.c > > index f522b654b4..03f839302e 100644 > > --- a/controller/lport.c > > +++ b/controller/lport.c > > @@ -132,6 +132,18 @@ lport_get_l3gw_peer(const struct sbrec_port_binding > > *pb, > > return get_peer_lport(pb, sbrec_port_binding_by_name); > > } > > > > +const struct sbrec_port_binding * > > +lport_get_cr_port(struct ovsdb_idl_index *sbrec_port_binding_by_name, > > + const struct sbrec_port_binding *pb) > > +{ > > + const char *crp = smap_get(&pb->options, "chassis-redirect-port"); > > + if (crp) { > > + return lport_lookup_by_name(sbrec_port_binding_by_name, crp); > > + } > > + > > + return NULL; > > +} > > Thanks for adding this helper! However, let's use it in all the places > where we manually do this kind of lookup already. I see the same thing > happening at least in: > > - need_add_peer_to_local() > - lport_is_local() - partially > - run_buffered_binding() > - route_exchange_find_port() - partially > > I think I'd move this helper and changing the existing callers to use it > to a separate patch too. > > > + > > enum can_bind > > lport_can_bind_on_this_chassis(const struct sbrec_chassis *chassis_rec, > > const struct sbrec_port_binding *pb) > > diff --git a/controller/lport.h b/controller/lport.h > > index c410454e4c..ab6647b81b 100644 > > --- a/controller/lport.h > > +++ b/controller/lport.h > > @@ -77,4 +77,8 @@ const struct sbrec_port_binding *lport_get_l3gw_peer( > > bool > > lport_is_activated_by_activation_strategy(const struct sbrec_port_binding > > *pb, > > const struct sbrec_chassis > > *chassis); > > +const struct sbrec_port_binding *lport_get_cr_port( > > + struct ovsdb_idl_index *sbrec_port_binding_by_name, > > + const struct sbrec_port_binding *); > > + > > #endif /* controller/lport.h */ > > diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c > > index 081411cba8..a1bf209b5e 100644 > > --- a/controller/ovn-controller.c > > +++ b/controller/ovn-controller.c > > @@ -104,6 +104,7 @@ static unixctl_cb_func debug_pause_execution; > > static unixctl_cb_func debug_resume_execution; > > static unixctl_cb_func debug_status_execution; > > static unixctl_cb_func debug_dump_local_bindings; > > +static unixctl_cb_func debug_dump_local_datapaths; > > static unixctl_cb_func debug_dump_related_lports; > > static unixctl_cb_func debug_dump_local_template_vars; > > static unixctl_cb_func debug_dump_local_mac_bindings; > > @@ -1722,6 +1723,22 @@ runtime_data_sb_datapath_binding_handler(struct > > engine_node *node OVS_UNUSED, > > return false; > > } > > } > > + > > + if (sbrec_datapath_binding_is_updated( > > + dp, SBREC_DATAPATH_BINDING_COL_EXTERNAL_IDS) && > > + !sbrec_datapath_binding_is_new(dp)) { > > + struct local_datapath *ld = > > + get_local_datapath(&rt_data->local_datapaths, > > + dp->tunnel_key); > > + if (ld && ld->is_switch) { > > + bool only_dgp_peer_ports = > > + smap_get_bool(&dp->external_ids, > > "only_dgp_peer_ports", > > + false); > > + if (ld->has_only_dgp_peer_ports != > > only_dgp_peer_ports) { > > + return false; > > + } > > + } > > + } > > } > > > > return true; > > @@ -4391,6 +4408,12 @@ lflow_output_runtime_data_handler(struct engine_node > > *node, > > init_lflow_ctx(node, fo, &l_ctx_in, &l_ctx_out); > > > > struct tracked_datapath *tdp; > > + HMAP_FOR_EACH (tdp, node, tracked_dp_bindings) { > > + if (tdp->tracked_type == TRACKED_RESOURCE_REMOVED) { > > + return false; > > + } > > + } > > + > > HMAP_FOR_EACH (tdp, node, tracked_dp_bindings) { > > if (tdp->tracked_type == TRACKED_RESOURCE_NEW) { > > if (!lflow_add_flows_for_datapath(tdp->dp, &l_ctx_in, > > @@ -5955,6 +5978,10 @@ main(int argc, char *argv[]) > > debug_dump_local_bindings, > > &runtime_data->lbinding_data); > > > > + unixctl_command_register("debug/dump-local-datapaths", "", 0, 0, > > + debug_dump_local_datapaths, > > + &runtime_data->local_datapaths); > > This is a very useful thing to have in general but I think it would make > review easier the code that adds the debug command is a separate commit. > > > + > > unixctl_command_register("debug/dump-related-ports", "", 0, 0, > > debug_dump_related_lports, > > &runtime_data->related_lports); > > @@ -6928,6 +6955,17 @@ debug_dump_local_bindings(struct unixctl_conn *conn, > > int argc OVS_UNUSED, > > ds_destroy(&binding_data); > > } > > > > +static void > > +debug_dump_local_datapaths(struct unixctl_conn *conn, int argc OVS_UNUSED, > > + const char *argv[] OVS_UNUSED, > > + void *local_datapaths) > > +{ > > + struct ds local_dps_data = DS_EMPTY_INITIALIZER; > > + binding_dump_local_datapaths(local_datapaths, &local_dps_data); > > + unixctl_command_reply(conn, ds_cstr(&local_dps_data)); > > + ds_destroy(&local_dps_data); > > +} > > + > > static void > > debug_dump_related_lports(struct unixctl_conn *conn, int argc OVS_UNUSED, > > const char *argv[] OVS_UNUSED, void > > *related_lports) > > diff --git a/tests/multinode.at b/tests/multinode.at > > index 68c9eba222..0cfe7bd35a 100644 > > --- a/tests/multinode.at > > +++ b/tests/multinode.at > > @@ -2576,6 +2576,189 @@ fi > > > > AT_CLEANUP > > > > +AT_SETUP([ovn multinode - only_dgp_peer_ports provider switch > > functionality]) > > + > > +# Check that ovn-fake-multinode setup is up and running > > +check_fake_multinode_setup > > + > > +# Delete the multinode NB and OVS resources before starting the test. > > +cleanup_multinode_resources > > + > > +check multinode_nbctl ls-add sw0 > > +check multinode_nbctl lsp-add sw0 sw0-port1 > > +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 > > 10.0.0.3 1000::3" > > +check multinode_nbctl lsp-add sw0 sw0-port2 > > +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 > > 10.0.0.4 1000::4" > > + > > +check multinode_nbctl lr-add lr0 > > +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 > > 1000::1/64 > > +check multinode_nbctl lsp-add sw0 sw0-lr0 > > +check multinode_nbctl lsp-set-type sw0-lr0 router > > +check multinode_nbctl lsp-set-addresses sw0-lr0 router > > +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0 > > + > > +check multinode_nbctl ls-add public > > +check multinode_nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 > > 172.16.1.100/24 2000::1/64 > > +check multinode_nbctl lsp-add public public-lr0 > > +check multinode_nbctl lsp-set-type public-lr0 router > > +check multinode_nbctl lsp-set-addresses public-lr0 router > > +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public > > + > > +# localnet port > > +check multinode_nbctl lsp-add public ln-public > > +check multinode_nbctl lsp-set-type ln-public localnet > > +check multinode_nbctl lsp-set-addresses ln-public unknown > > +check multinode_nbctl lsp-set-options ln-public network_name=public > > + > > +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 20 > > +check multinode_nbctl lr-nat-add lr0 snat 172.16.1.100 10.0.0.0/24 > > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.16.1.110 10.0.0.3 > > sw0-port1 50:54:00:00:00:03 > > +check multinode_nbctl lr-nat-add lr0 snat 2000::1 1000::/64 > > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 2000::2 1000::3 > > sw0-port1 50:54:00:00:00:03 > > + > > +check multinode_nbctl --wait=hv sync > > + > > +check multinode_nbctl ls-add sw1 > > +check multinode_nbctl lsp-add sw1 sw1-port1 > > +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 > > 20.0.0.3 2000::3" > > + > > +check multinode_nbctl lr-add lr1 > > +check multinode_nbctl lrp-add lr1 lr1-sw1 00:00:01:00:ef:01 20.0.0.1/24 > > 2000::a/64 > > +check multinode_nbctl lsp-add sw1 sw1-lr1 > > +check multinode_nbctl lsp-set-type sw1-lr1 router > > +check multinode_nbctl lsp-set-addresses sw1-lr1 router > > +check multinode_nbctl lsp-set-options sw1-lr1 router-port=lr1-sw1 > > + > > +check multinode_nbctl lrp-add lr1 lr1-public 00:00:20:30:22:13 > > 172.16.1.101/24 > > +check multinode_nbctl lsp-add public public-lr1 > > +check multinode_nbctl lsp-set-type public-lr1 router > > +check multinode_nbctl lsp-set-addresses public-lr1 router > > +check multinode_nbctl lsp-set-options public-lr1 router-port=lr1-public > > + > > +check multinode_nbctl lr-nat-add lr1 snat 172.16.1.101 20.0.0.0/24 > > +check multinode_nbctl lr-nat-add lr1 dnat_and_snat 172.16.1.120 20.0.0.3 > > +check multinode_nbctl lrp-set-gateway-chassis lr1-public ovn-gw-1 20 > > + > > +check multinode_nbctl --wait=hv sync > > + > > +# Delete already used ovs-ports (if any) > > +m_as ovn-chassis-1 ip link del sw0p1-p || : > > +m_as ovn-chassis-2 ip link del sw1p1-p || : > > +m_as ovn-chassis-1 ip link del sw0p2-p || : > > + > > +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 > > 50:54:00:00:00:03 1342 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a > > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 > > 40:54:00:00:00:03 1342 20.0.0.3 24 20.0.0.1 2000::4/64 1000::a > > + > > +m_wait_for_ports_up sw0-port1 > > +m_wait_for_ports_up sw1-port1 > > + > > +m_as ovn-central-az1-1 ovn-sbctl show > > + > > +m_as ovn-chassis-1 ovn-appctl debug/dump-local-datapaths | sort > > +m_as ovn-chassis-2 ovn-appctl debug/dump-local-datapaths | sort > > + > > +AT_CHECK([m_as ovn-chassis-1 ovn-appctl debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([m_as ovn-chassis-2 ovn-appctl debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr1, type: router > > +Datapath: sw1, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([m_as ovn-gw-1 ovn-appctl debug/dump-local-datapaths | sort], > > [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: lr1, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Datapath: sw1, type: switch > > +Local datapaths: > > +]) > > + > > +# ping lr0-public IP - 172.168.0.100 > > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 > > 172.16.1.100 | FORMAT_PING], \ > > +[0], [dnl > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > > +]) > > + > > +# ping lr1-public IP - 172.168.0.101 from sw0p1 > > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 > > 172.16.1.101 | FORMAT_PING], \ > > +[0], [dnl > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > > +]) > > + > > +# ping public ip of sw1-port1 - 172.16.1.120 from sw0p1 > > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 > > 172.16.1.120 | FORMAT_PING], \ > > +[0], [dnl > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > > +]) > > + > > +# ping public ip of sw0-port1 - 172.16.1.110 from sw1p1 > > +M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 > > 172.16.1.110 | FORMAT_PING], \ > > +[0], [dnl > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > > +]) > > + > > +# Bind sw0-port2 on chassis-2 > > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 > > 50:54:00:00:00:04 1342 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a > > +m_wait_for_ports_up sw0-port2 > > + > > +AT_CHECK([m_as ovn-chassis-2 ovn-appctl debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: lr1, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Datapath: sw1, type: switch > > +Local datapaths: > > +]) > > + > > +# ping public ip of sw0-port1 - 172.16.1.110 from sw0p2 > > +M_NS_CHECK_EXEC([ovn-chassis-2], [sw0p2], [ping -q -c 3 -i 0.3 -w 2 > > 172.16.1.110 | FORMAT_PING], \ > > +[0], [dnl > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > > +]) > > + > > +# ping public ip of sw1-port1 - 172.16.1.120 from sw0p2 > > +M_NS_CHECK_EXEC([ovn-chassis-2], [sw0p2], [ping -q -c 3 -i 0.3 -w 2 > > 172.16.1.120 | FORMAT_PING], \ > > +[0], [dnl > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms > > +]) > > + > > +# Create a normal router port in public with its peer as a normal > > distributed router port. > > +check multinode_nbctl lsp-add public public-lr2 > > +check multinode_nbctl lsp-set-type public-lr2 router > > +check multinode_nbctl lsp-set-addresses public-lr2 router > > +check multinode_nbctl lsp-set-options public-lr2 router-port=lr2-public > > +check multinode_nbctl lr-add lr2 > > +check multinode_nbctl lrp-add lr2 lr2-public 00:00:41:00:1f:61 > > 172.16.1.102/24 3000::a/64 > > + > > +check multinode_nbctl --wait=hv sync > > +AT_CHECK([m_as ovn-chassis-1 ovn-appctl debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: lr1, type: router > > +Datapath: lr2, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Datapath: sw1, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([m_as ovn-chassis-2 ovn-appctl debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: lr1, type: router > > +Datapath: lr2, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Datapath: sw1, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CLEANUP > > + > > AT_SETUP([ovn multinode - Transit Router basic functionality]) > > > > # Check that ovn-fake-multinode setup is up and running > > @@ -3029,5 +3212,3 @@ m_as ovn-chassis-2 killall tcpdump > > m_as ovn-chassis-3 killall tcpdump > > > > AT_CLEANUP > > - > > - > > diff --git a/tests/ovn-performance.at b/tests/ovn-performance.at > > index 7d480c20c8..a003fc36bc 100644 > > --- a/tests/ovn-performance.at > > +++ b/tests/ovn-performance.at > > @@ -479,7 +479,7 @@ OVN_CONTROLLER_EXPECT_NO_HIT( > > ) > > > > OVN_CONTROLLER_EXPECT_HIT_COND( > > - [hv1 hv2 hv3 hv4 hv5], [lflow_run], [=0 =0 >0 =0 =0], > > + [hv1 hv2 hv3 hv4 hv5], [lflow_run], [>0 >0 >0 =0 =0], > > [ovn-nbctl --wait=hv lrp-set-gateway-chassis lr1-public hv3 30 && > > ovn-nbctl --wait=hv sync] > > ) > > > > @@ -552,8 +552,8 @@ hv5_ch=$(ovn-sbctl --bare --columns _uuid list chassis > > hv5) > > OVS_WAIT_UNTIL([ovn-sbctl find port_binding logical_port=cr-lr1-public > > chassis=$hv5_ch]) > > check ovn-nbctl --wait=hv sync > > # Delete hv5 from gateway chassis. There should be no lflow_run. > > -OVN_CONTROLLER_EXPECT_NO_HIT( > > - [hv1 hv2 hv3 hv4 hv5], [lflow_run], > > +OVN_CONTROLLER_EXPECT_HIT_COND( > > + [hv1 hv2 hv3 hv4 hv5], [lflow_run], [=0 =0 =0 =0 =0] > > [ovn-nbctl --wait=hv lrp-del-gateway-chassis lr1-public hv5] > > ) > > > > diff --git a/tests/ovn.at b/tests/ovn.at > > index ec8ee8de77..50721f9c94 100644 > > --- a/tests/ovn.at > > +++ b/tests/ovn.at > > @@ -41846,6 +41846,859 @@ OVN_CLEANUP([hv1]) > > AT_CLEANUP > > ]) > > > > +OVN_FOR_EACH_NORTHD_NO_HV([ > > +AT_SETUP([ovn-controller -- only_dgp_peer_ports flag in SB > > datapath_binding]) > > +AT_KEYWORDS([multiple-l3dgw-ports]) > > +ovn_start > > +net_add n1 > > +sim_add hv1 > > +as hv1 > > +check ovs-vsctl add-br br-phys > > +ovn_attach n1 br-phys 192.168.0.1 > > + > > +sim_add hv2 > > +as hv2 > > +check ovs-vsctl add-br br-phys > > +ovn_attach n1 br-phys 192.168.0.2 > > + > > +sim_add gw1 > > +as gw1 > > +check ovs-vsctl add-br br-phys > > +ovn_attach n1 br-phys 192.168.0.3 > > + > > +sim_add gw2 > > +as gw2 > > +check ovs-vsctl add-br br-phys > > +ovn_attach n1 br-phys 192.168.0.4 > > + > > +check ovn-nbctl ls-add sw0 > > +check ovn-nbctl lsp-add sw0 sw0-port1 > > +check ovn-nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:01 10.0.0.3 > > 1000::3" > > +check ovn-nbctl lsp-add sw0 sw0-port2 > > +check ovn-nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:02 10.0.0.4 > > 1000::4" > > + > > +check ovn-nbctl lr-add lr0 > > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 > > 1000::1/64 > > +check ovn-nbctl lsp-add sw0 sw0-lr0 > > +check ovn-nbctl lsp-set-type sw0-lr0 router > > +check ovn-nbctl lsp-set-addresses sw0-lr0 router > > +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0 > > + > > +check ovn-nbctl ls-add public > > +check ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24 > > 2000::1/64 > > +check ovn-nbctl lsp-add public public-lr0 > > +check ovn-nbctl lsp-set-type public-lr0 router > > +check ovn-nbctl lsp-set-addresses public-lr0 router > > +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public > > + > > +# localnet port > > +check ovn-nbctl lsp-add public ln-public > > +check ovn-nbctl lsp-set-type ln-public localnet > > +check ovn-nbctl lsp-set-addresses ln-public unknown > > +check ovn-nbctl lsp-set-options ln-public network_name=phys > > + > > +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1 20 > > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24 > > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 > > sw0-port2 f0:00:00:01:02:04 > > +check ovn-nbctl lr-nat-add lr0 snat 2000::1 1000::/64 > > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 2000::2 1000::4 sw0-port2 > > f0:00:00:01:02:04 > > + > > +check ovn-nbctl --wait=hv sync > > + > > +sw0_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key > > external_ids:name=sw0)) > > +lr0_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key > > external_ids:name=lr0)) > > +public_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key > > external_ids:name=public)) > > + > > +check_offlows_for_datapath() { > > + hv=$1 > > + dp_key=$2 > > + should_be_present=$3 > > + > > + if [[ "$should_be_present" == "yes" ]]; then > > + echo "Flows should be present for hv - $hv : datapath - $dp_key" > > + OVS_WAIT_UNTIL( > > + [test $(as $hv ovs-ofctl dump-flows br-int | grep -c > > metadata=0x$dp_key) -gt 0] > > + ) > > + else > > + echo "Flows should NOT be present for hv - $hv : datapath - > > $dp_key" > > + OVS_WAIT_UNTIL( > > + [test $(as $hv ovs-ofctl dump-flows br-int | grep -c > > metadata=0x$dp_key) -eq 0] > > + ) > > + fi > > +} > > + > > +AT_CHECK([ovn-sbctl get datapath_binding public > > external_ids:only_dgp_peer_ports], [0], [dnl > > +"true" > > +]) > > + > > +check_offlows_for_datapath hv1 $sw0_dp_key no > > +check_offlows_for_datapath hv1 $lr0_dp_key no > > +check_offlows_for_datapath hv1 $public_dp_key no > > + > > +check_offlows_for_datapath hv2 $sw0_dp_key no > > +check_offlows_for_datapath hv2 $lr0_dp_key no > > +check_offlows_for_datapath hv2 $public_dp_key no > > + > > +check_offlows_for_datapath gw1 $sw0_dp_key yes > > +check_offlows_for_datapath gw1 $lr0_dp_key yes > > +check_offlows_for_datapath gw1 $public_dp_key yes > > + > > +check_offlows_for_datapath gw2 $sw0_dp_key no > > +check_offlows_for_datapath gw2 $lr0_dp_key no > > +check_offlows_for_datapath gw2 $public_dp_key no > > + > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths], > > [0], [dnl > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], > > [0], [dnl > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], > > [0], [dnl > > +Local datapaths: > > +]) > > + > > +# Create a VIF on hv1 for sw0-port1 > > +AS_BOX([create a VIF on hv1 for sw0-port1]) > > + > > +as hv1 > > +ovs-vsctl -- add-port br-int hv1-vif1 -- \ > > + set interface hv1-vif1 external-ids:iface-id=sw0-port1 \ > > + options:tx_pcap=hv1/vif1-tx.pcap \ > > + options:rxq_pcap=hv1/vif1-rx.pcap \ > > + ofport-request=1 > > + > > +wait_for_ports_up sw0-port1 > > + > > +AS_BOX([Create a VIF on hv1 for sw0-port1 - hv1 should have flows for sw0, > > lr0 and public]) > > + > > +check_offlows_for_datapath hv1 $sw0_dp_key yes > > +check_offlows_for_datapath hv1 $lr0_dp_key yes > > +check_offlows_for_datapath hv1 $public_dp_key yes > > + > > +AS_BOX([hv2 should NOT have flows for sw0, lr0 and public]) > > +check_offlows_for_datapath hv2 $sw0_dp_key no > > +check_offlows_for_datapath hv2 $lr0_dp_key no > > +check_offlows_for_datapath hv2 $public_dp_key no > > + > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], > > [0], [dnl > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], > > [0], [dnl > > +Local datapaths: > > +]) > > + > > +AS_BOX([create a switch sw1 and router lr1, attach both and attach lr1 to > > public]) > > + > > +check ovn-nbctl ls-add sw1 > > +check ovn-nbctl lsp-add sw1 sw1-port1 > > +check ovn-nbctl lsp-set-addresses sw1-port1 "60:54:00:00:00:01 20.0.0.3" > > + > > +check ovn-nbctl lr-add lr1 > > +check ovn-nbctl lrp-add lr1 lr1-sw1 00:00:01:00:ef:01 20.0.0.1/24 > > +check ovn-nbctl lsp-add sw1 sw1-lr1 > > +check ovn-nbctl lsp-set-type sw1-lr1 router > > +check ovn-nbctl lsp-set-addresses sw1-lr1 router > > +check ovn-nbctl lsp-set-options sw1-lr1 router-port=lr1-sw1 > > + > > +check ovn-nbctl lrp-add lr1 lr1-public 00:00:20:30:22:13 172.168.0.101/24 > > +check ovn-nbctl lsp-add public public-lr1 > > +check ovn-nbctl lsp-set-type public-lr1 router > > +check ovn-nbctl lsp-set-addresses public-lr1 router > > +check ovn-nbctl lsp-set-options public-lr1 router-port=lr1-public > > + > > +sw1_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key > > external_ids:name=sw1)) > > +lr1_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key > > external_ids:name=lr1)) > > + > > +check ovn-nbctl lr-nat-add lr1 snat 172.168.0.101 20.0.0.0/24 > > +check ovn-nbctl lr-nat-add lr1 dnat_and_snat 172.168.0.140 20.0.0.3 > > + > > +AS_BOX([create a switch sw2 and router lr2, attach both and attach lr2 to > > public]) > > + > > +check ovn-nbctl ls-add sw2 > > +check ovn-nbctl lsp-add sw2 sw2-port1 > > +check ovn-nbctl lsp-set-addresses sw2-port1 "70:54:00:00:00:01 30.0.0.3" > > + > > +check ovn-nbctl lr-add lr2 > > +check ovn-nbctl lrp-add lr2 lr2-sw2 00:00:02:00:ef:01 30.0.0.1/24 > > +check ovn-nbctl lsp-add sw2 sw2-lr2 > > +check ovn-nbctl lsp-set-type sw2-lr2 router > > +check ovn-nbctl lsp-set-addresses sw2-lr2 router > > +check ovn-nbctl lsp-set-options sw2-lr2 router-port=lr2-sw2 > > + > > +check ovn-nbctl lrp-add lr2 lr2-public 00:00:20:40:22:53 172.168.0.102/24 > > +check ovn-nbctl lsp-add public public-lr2 > > +check ovn-nbctl lsp-set-type public-lr2 router > > +check ovn-nbctl lsp-set-addresses public-lr2 router > > +check ovn-nbctl lsp-set-options public-lr2 router-port=lr2-public > > + > > +sw2_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key > > external_ids:name=sw2)) > > +lr2_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key > > external_ids:name=lr2)) > > + > > +check ovn-nbctl lr-nat-add lr2 snat 172.168.0.102 30.0.0.0/24 > > +check ovn-nbctl lr-nat-add lr2 dnat_and_snat 172.168.0.150 30.0.0.3 > > + > > +check ovn-nbctl --wait=hv sync > > + > > +# Since lr1-public is not a DGP, public is not a "only_dgp_peer_ports". > > +AT_CHECK([ovn-sbctl get datapath_binding public > > external_ids:only_dgp_peer_ports], [1], [ignore], [ignore]) > > + > > +check_offlows_for_datapath hv1 $sw1_dp_key yes > > +check_offlows_for_datapath hv1 $lr1_dp_key yes > > +check_offlows_for_datapath hv1 $sw2_dp_key yes > > +check_offlows_for_datapath hv1 $lr2_dp_key yes > > + > > +check_offlows_for_datapath hv2 $sw1_dp_key no > > +check_offlows_for_datapath hv2 $lr1_dp_key no > > +check_offlows_for_datapath hv2 $sw2_dp_key no > > +check_offlows_for_datapath hv2 $lr2_dp_key no > > + > > +check_offlows_for_datapath gw1 $sw1_dp_key yes > > +check_offlows_for_datapath gw1 $lr1_dp_key yes > > +check_offlows_for_datapath gw1 $sw2_dp_key yes > > +check_offlows_for_datapath gw1 $lr2_dp_key yes > > + > > +check_offlows_for_datapath gw2 $sw1_dp_key no > > +check_offlows_for_datapath gw2 $lr1_dp_key no > > +check_offlows_for_datapath gw2 $sw2_dp_key no > > +check_offlows_for_datapath gw2 $lr2_dp_key no > > + > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: lr1, type: router > > +Datapath: lr2, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Datapath: sw1, type: switch > > +Datapath: sw2, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], > > [0], [dnl > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: lr1, type: router > > +Datapath: lr2, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Datapath: sw1, type: switch > > +Datapath: sw2, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], > > [0], [dnl > > +Local datapaths: > > +]) > > + > > +AS_BOX([Set gw2 as gateway chassis for lr1-public and lr2-public]) > > +check ovn-nbctl --wait=hv lrp-set-gateway-chassis lr1-public gw2 20 > > +check ovn-nbctl --wait=hv lrp-set-gateway-chassis lr2-public gw2 30 > > +wait_row_count Port_Binding 1 logical_port=cr-lr1-public > > +wait_row_count Port_Binding 1 logical_port=cr-lr2-public > > + > > +AT_CHECK([ovn-sbctl get datapath_binding public > > external_ids:only_dgp_peer_ports], [0], [dnl > > +"true" > > +]) > > + > > +check ovn-nbctl --wait=hv sync > > + > > +check_offlows_for_datapath hv1 $sw0_dp_key yes > > +check_offlows_for_datapath hv1 $lr0_dp_key yes > > +check_offlows_for_datapath hv1 $public_dp_key yes > > +check_offlows_for_datapath hv1 $sw1_dp_key no > > +check_offlows_for_datapath hv1 $lr1_dp_key no > > +check_offlows_for_datapath hv1 $sw2_dp_key no > > +check_offlows_for_datapath hv1 $lr2_dp_key no > > + > > +check_offlows_for_datapath hv2 $sw0_dp_key no > > +check_offlows_for_datapath hv2 $lr0_dp_key no > > +check_offlows_for_datapath hv2 $public_dp_key no > > +check_offlows_for_datapath hv2 $sw1_dp_key no > > +check_offlows_for_datapath hv2 $lr1_dp_key no > > +check_offlows_for_datapath hv2 $sw2_dp_key no > > +check_offlows_for_datapath hv2 $lr2_dp_key no > > + > > +check_offlows_for_datapath gw1 $sw0_dp_key yes > > +check_offlows_for_datapath gw1 $lr0_dp_key yes > > +check_offlows_for_datapath gw1 $public_dp_key yes > > +check_offlows_for_datapath gw1 $sw1_dp_key no > > +check_offlows_for_datapath gw1 $lr1_dp_key no > > +check_offlows_for_datapath gw1 $sw2_dp_key no > > +check_offlows_for_datapath gw1 $lr2_dp_key no > > + > > +check_offlows_for_datapath gw2 $sw0_dp_key no > > +check_offlows_for_datapath gw2 $lr0_dp_key no > > +check_offlows_for_datapath hv1 $public_dp_key yes > > +check_offlows_for_datapath gw2 $sw1_dp_key yes > > +check_offlows_for_datapath gw2 $lr1_dp_key yes > > +check_offlows_for_datapath gw2 $sw2_dp_key yes > > +check_offlows_for_datapath gw2 $lr2_dp_key yes > > + > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], > > [0], [dnl > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr1, type: router > > +Datapath: lr2, type: router > > +Datapath: public, type: switch > > +Datapath: sw1, type: switch > > +Datapath: sw2, type: switch > > +Local datapaths: > > +]) > > + > > +AS_BOX([Create a VIF on hv2 for sw1-port1]) > > + > > +as hv2 > > +ovs-vsctl -- add-port br-int hv2-vif1 -- \ > > + set interface hv2-vif1 external-ids:iface-id=sw1-port1 \ > > + options:tx_pcap=hv2/vif1-tx.pcap \ > > + options:rxq_pcap=hv2/vif1-rx.pcap \ > > + ofport-request=1 > > + > > +wait_for_ports_up sw1-port1 > > + > > +check_offlows_for_datapath hv1 $sw0_dp_key yes > > +check_offlows_for_datapath hv1 $lr0_dp_key yes > > +check_offlows_for_datapath hv1 $public_dp_key yes > > +check_offlows_for_datapath hv1 $sw1_dp_key no > > +check_offlows_for_datapath hv1 $lr1_dp_key no > > +check_offlows_for_datapath hv1 $sw2_dp_key no > > +check_offlows_for_datapath hv1 $lr2_dp_key no > > + > > +check_offlows_for_datapath hv2 $sw0_dp_key no > > +check_offlows_for_datapath hv2 $lr0_dp_key no > > + > > +# Since there are no distributed dnat_and_snat entries > > +# in lr1, hv2 will not have "public" in its > > +# local datapaths. > > +check_offlows_for_datapath hv2 $public_dp_key no > > +check_offlows_for_datapath hv2 $sw1_dp_key yes > > +check_offlows_for_datapath hv2 $lr1_dp_key yes > > +check_offlows_for_datapath hv2 $sw2_dp_key no > > +check_offlows_for_datapath hv2 $lr2_dp_key no > > + > > +check_offlows_for_datapath gw1 $sw0_dp_key yes > > +check_offlows_for_datapath gw1 $lr0_dp_key yes > > +check_offlows_for_datapath gw1 $public_dp_key yes > > +check_offlows_for_datapath gw1 $sw1_dp_key no > > +check_offlows_for_datapath gw1 $lr1_dp_key no > > +check_offlows_for_datapath gw1 $sw2_dp_key no > > +check_offlows_for_datapath gw1 $lr2_dp_key no > > + > > +# gw2 should have sw1, lr1, sw2 and lr2 and public in its local datapaths. > > +check_offlows_for_datapath gw2 $sw0_dp_key no > > +check_offlows_for_datapath gw2 $lr0_dp_key no > > +check_offlows_for_datapath gw2 $public_dp_key yes > > +check_offlows_for_datapath gw2 $sw1_dp_key yes > > +check_offlows_for_datapath gw2 $lr1_dp_key yes > > +check_offlows_for_datapath gw2 $sw2_dp_key yes > > +check_offlows_for_datapath gw2 $lr2_dp_key yes > > + > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr1, type: router > > +Datapath: sw1, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr1, type: router > > +Datapath: lr2, type: router > > +Datapath: public, type: switch > > +Datapath: sw1, type: switch > > +Datapath: sw2, type: switch > > +Local datapaths: > > +]) > > + > > +# Add distributed dnat_and_snat in lr1. hv2 should have > > +# public in its local datapaths. > > +AS_BOX([ Add distributed dnat_and_snat in lr1]) > > + > > +check ovn-nbctl lr-nat-del lr1 dnat_and_snat > > +check ovn-nbctl --wait=hv lr-nat-add lr1 dnat_and_snat 172.168.0.140 > > 20.0.0.3 sw1-port1 10:00:00:01:02:14 > > + > > +check_offlows_for_datapath hv2 $sw0_dp_key no > > +check_offlows_for_datapath hv2 $lr0_dp_key no > > +check_offlows_for_datapath hv2 $public_dp_key yes > > +check_offlows_for_datapath hv2 $sw1_dp_key yes > > +check_offlows_for_datapath hv2 $lr1_dp_key yes > > +check_offlows_for_datapath hv2 $sw2_dp_key no > > +check_offlows_for_datapath hv2 $lr2_dp_key no > > + > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr1, type: router > > +Datapath: public, type: switch > > +Datapath: sw1, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr1, type: router > > +Datapath: lr2, type: router > > +Datapath: public, type: switch > > +Datapath: sw1, type: switch > > +Datapath: sw2, type: switch > > +Local datapaths: > > +]) > > + > > +AS_BOX([Create a VIF on hv2 for sw0-port2]) > > + > > +as hv2 > > +ovs-vsctl -- add-port br-int hv2-vif2 -- \ > > + set interface hv2-vif2 external-ids:iface-id=sw0-port2 \ > > + options:tx_pcap=hv2/vif2-tx.pcap \ > > + options:rxq_pcap=hv2/vif2-rx.pcap \ > > + ofport-request=2 > > + > > +wait_for_ports_up sw0-port2 > > + > > +check_offlows_for_datapath hv2 $sw0_dp_key yes > > +check_offlows_for_datapath hv2 $lr0_dp_key yes > > +check_offlows_for_datapath hv2 $public_dp_key yes > > +check_offlows_for_datapath hv2 $sw1_dp_key yes > > +check_offlows_for_datapath hv2 $lr1_dp_key yes > > +check_offlows_for_datapath hv2 $sw2_dp_key no > > +check_offlows_for_datapath hv2 $lr2_dp_key no > > + > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: lr1, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Datapath: sw1, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr1, type: router > > +Datapath: lr2, type: router > > +Datapath: public, type: switch > > +Datapath: sw1, type: switch > > +Datapath: sw2, type: switch > > +Local datapaths: > > +]) > > + > > +AS_BOX([Delete the VIF for sw1-port1 in hv2]) > > + > > +as hv2 ovs-vsctl del-port hv2-vif1 > > +check ovn-nbctl --wait=hv sync > > +check_column "false" Port_Binding up logical_port=sw1-port1 > > + > > +check_offlows_for_datapath hv2 $sw0_dp_key yes > > +check_offlows_for_datapath hv2 $lr0_dp_key yes > > +check_offlows_for_datapath hv2 $public_dp_key yes > > +check_offlows_for_datapath hv2 $sw1_dp_key no > > +check_offlows_for_datapath hv2 $lr1_dp_key no > > +check_offlows_for_datapath hv2 $sw2_dp_key no > > +check_offlows_for_datapath hv2 $lr2_dp_key no > > + > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr1, type: router > > +Datapath: lr2, type: router > > +Datapath: public, type: switch > > +Datapath: sw1, type: switch > > +Datapath: sw2, type: switch > > +Local datapaths: > > +]) > > + > > +AS_BOX([Delete the VIF for sw0-port2 in hv2]) > > + > > +# Presently when a port binding is released we are not > > +# deleting its datapath from the local_datapaths if it > > +# is not relevant anymore. > > + > > +as hv2 ovs-vsctl del-port hv2-vif2 > > +check ovn-nbctl --wait=hv sync > > +check_column "false" Port_Binding up logical_port=sw0-port2 > > + > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +# hv2 would still have public, sw0 and lr0 in its local datapaths. > > +# Next recompute should delete these datapaths. > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr1, type: router > > +Datapath: lr2, type: router > > +Datapath: public, type: switch > > +Datapath: sw1, type: switch > > +Datapath: sw2, type: switch > > +Local datapaths: > > +]) > > + > > +# Trigger a recompute > > +AS_BOX([Trigger a recompute in hv2]) > > +check as hv2 ovn-appctl inc-engine/recompute > > + > > +check_offlows_for_datapath hv2 $sw0_dp_key no > > +check_offlows_for_datapath hv2 $lr0_dp_key no > > +check_offlows_for_datapath hv2 $public_dp_key no > > +check_offlows_for_datapath hv2 $sw1_dp_key no > > +check_offlows_for_datapath hv2 $lr1_dp_key no > > +check_offlows_for_datapath hv2 $sw2_dp_key no > > +check_offlows_for_datapath hv2 $lr2_dp_key no > > + > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr1, type: router > > +Datapath: lr2, type: router > > +Datapath: public, type: switch > > +Datapath: sw1, type: switch > > +Datapath: sw2, type: switch > > +Local datapaths: > > +]) > > + > > +AS_BOX([Disconnect sw2 from lr2]) > > + > > +check ovn-nbctl --wait=hv lsp-set-options sw2-lr2 router-port=lr2-sw2xxx > > +check_offlows_for_datapath hv1 $sw0_dp_key yes > > +check_offlows_for_datapath hv1 $lr0_dp_key yes > > +check_offlows_for_datapath hv1 $public_dp_key yes > > +check_offlows_for_datapath hv1 $sw1_dp_key no > > +check_offlows_for_datapath hv1 $lr1_dp_key no > > +check_offlows_for_datapath hv1 $sw2_dp_key no > > +check_offlows_for_datapath hv1 $lr2_dp_key no > > + > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr1, type: router > > +Datapath: lr2, type: router > > +Datapath: public, type: switch > > +Datapath: sw1, type: switch > > +Local datapaths: > > +]) > > + > > +AS_BOX([Reconnect sw2 to lr2 again]) > > + > > +check ovn-nbctl --wait=hv lsp-set-options sw2-lr2 router-port=lr2-sw2 > > +check_offlows_for_datapath hv1 $sw0_dp_key yes > > +check_offlows_for_datapath hv1 $lr0_dp_key yes > > +check_offlows_for_datapath hv1 $public_dp_key yes > > +check_offlows_for_datapath hv1 $sw1_dp_key no > > +check_offlows_for_datapath hv1 $lr1_dp_key no > > +check_offlows_for_datapath hv1 $sw2_dp_key no > > +check_offlows_for_datapath hv1 $lr2_dp_key no > > + > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr1, type: router > > +Datapath: lr2, type: router > > +Datapath: public, type: switch > > +Datapath: sw1, type: switch > > +Datapath: sw2, type: switch > > +Local datapaths: > > +]) > > + > > +AS_BOX([Create a VIF on gw2 for sw1-port1]) > > + > > +as gw2 > > +ovs-vsctl -- add-port br-int gw2-vif2 -- \ > > + set interface gw2-vif2 external-ids:iface-id=sw1-port1 \ > > + options:tx_pcap=gw2/vif2-tx.pcap \ > > + options:rxq_pcap=gw2/vif2-rx.pcap \ > > + ofport-request=2 > > + > > +wait_for_ports_up sw1-port1 > > + > > +check_offlows_for_datapath gw2 $sw0_dp_key no > > +check_offlows_for_datapath gw2 $lr0_dp_key no > > +check_offlows_for_datapath gw2 $public_dp_key yes > > +check_offlows_for_datapath gw2 $sw1_dp_key yes > > +check_offlows_for_datapath gw2 $lr1_dp_key yes > > +check_offlows_for_datapath gw2 $sw2_dp_key yes > > +check_offlows_for_datapath gw2 $lr2_dp_key yes > > + > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr1, type: router > > +Datapath: lr2, type: router > > +Datapath: public, type: switch > > +Datapath: sw1, type: switch > > +Datapath: sw2, type: switch > > +Local datapaths: > > +]) > > + > > +AS_BOX([Delete the VIF for sw1-port1 in gw2]) > > + > > +as gw2 ovs-vsctl del-port gw2-vif2 > > +check ovn-nbctl --wait=hv sync > > +check_column "false" Port_Binding up logical_port=sw1-port1 > > + > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +# hv2 would still have public in its local datapaths. Next recompute > > should > > +# delete this datapath from the local datapaths. > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr1, type: router > > +Datapath: lr2, type: router > > +Datapath: public, type: switch > > +Datapath: sw1, type: switch > > +Datapath: sw2, type: switch > > +Local datapaths: > > +]) > > + > > +AS_BOX([Create a logical port for public and bind it on hv2]) > > +# hv2 will only have public in its local datapaths. > > +check ovn-nbctl lsp-add public public-p1 > > + > > +as hv2 > > +ovs-vsctl -- add-port br-int hv2-vif3 -- \ > > + set interface hv2-vif3 external-ids:iface-id=public-p1 \ > > + options:tx_pcap=hv2/vif3-tx.pcap \ > > + options:rxq_pcap=hv2/vif3-rx.pcap \ > > + ofport-request=2 > > + > > +wait_for_ports_up public-p1 > > + > > +# as hv2 ovn-appctl -t ovn-controller inc-engine/recompute > > +# check ovn-nbctl --wait=hv sync > > + > > +as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort > > + > > +check_offlows_for_datapath hv2 $sw0_dp_key no > > +check_offlows_for_datapath hv2 $lr0_dp_key no > > +check_offlows_for_datapath hv2 $public_dp_key yes > > +check_offlows_for_datapath hv2 $sw1_dp_key no > > +check_offlows_for_datapath hv2 $lr1_dp_key no > > +check_offlows_for_datapath hv2 $sw2_dp_key no > > +check_offlows_for_datapath hv2 $lr2_dp_key no > > + > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: public, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr0, type: router > > +Datapath: public, type: switch > > +Datapath: sw0, type: switch > > +Local datapaths: > > +]) > > + > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | > > sort], [0], [dnl > > +Datapath: lr1, type: router > > +Datapath: lr2, type: router > > +Datapath: public, type: switch > > +Datapath: sw1, type: switch > > +Datapath: sw2, type: switch > > +Local datapaths: > > +]) > > + > > +OVN_CLEANUP([hv1], [hv2], [gw1], [gw2]) > > +AT_CLEANUP > > +]) > > + > > OVN_FOR_EACH_NORTHD([ > > AT_SETUP([requested-tnl-key-recompute]) > > AT_KEYWORDS([requested-tnl-key-recompute]) > > Regards, > Dumitru > > _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
