On Fri, Feb 14, 2025 at 7:34 AM Martin Kalcok <[email protected]> wrote:
> This change builds on top of the new "dynamic routing" OVN feature > that allows advertising routes to the fabric network. New accepted > values are added to "dynamic-routing-redistribute" option. > > * nat - When included in the option, ovn-controller will advertise > routes for external NAT IPs of the LR, as well as those > of the neighboring routers (directly connected or through > the shared LS). > * lb - When included in the option, ovn-controller will advertise > routes for LB VIPs of the LR, as well as those > of the neighboring routers (directly connected or through > the shared LS). > > Co-authored-by: Frode Nordahl <[email protected]> > Co-authored-by: Dumitru Ceara <[email protected]> > Signed-off-by: Frode Nordahl <[email protected]> > Signed-off-by: Dumitru Ceara <[email protected]> > Signed-off-by: Martin Kalcok <[email protected]> > --- Hi Martin, I have a couple of comments down below. TODO.rst | 10 + > lib/stopwatch-names.h | 1 + > northd/en-advertised-route-sync.c | 198 ++++++-- > northd/en-advertised-route-sync.h | 3 + > northd/en-learned-route-sync.c | 3 +- > northd/inc-proc-northd.c | 5 + > northd/northd.c | 254 +++++++++- > northd/northd.h | 34 +- > ovn-nb.xml | 36 ++ > tests/ovn-northd.at | 628 +++++++++++++++++++++++++ > tests/system-ovn.at | 738 ++++++++++++++++++++++++++++++ > 11 files changed, 1861 insertions(+), 49 deletions(-) > > diff --git a/TODO.rst b/TODO.rst > index d75db3152..593b9a580 100644 > --- a/TODO.rst > +++ b/TODO.rst > @@ -146,6 +146,9 @@ OVN To-do List > * Add incremental processing for northd when the Learned_Route table > changes. > Currently en_lflow is fully recomputed whenever such a change happens. > > + * Add incremental processing of en_dynamic_routes for stateful > configuration > + changes. > + > * The ovn-controller currently loads all Advertised_Route entries on > startup. > This is to prevent deleting our routes on restart. If we defer > updating > routes until we are sure to have loaded all necessary > Advertised_Routes > @@ -156,3 +159,10 @@ OVN To-do List > monitoring conditions to update before we actually try to learn > routes. > Otherwise we could try to add duplicated Learned_Routes and the ovnsb > commit would fail. > + > + * Consider splitting parsed_route structure. When creating parsed routes > + with tracked_port explicitly set, other members of this structure are > + usually unused/default. A new structure dedicated to routes with > + explicitly defined tracked_port would be more efficient. > + More details in > + > https://mail.openvswitch.org/pipermail/ovs-dev/2025-February/420985.html > diff --git a/lib/stopwatch-names.h b/lib/stopwatch-names.h > index c12dd28d0..13aa5e7bf 100644 > --- a/lib/stopwatch-names.h > +++ b/lib/stopwatch-names.h > @@ -36,5 +36,6 @@ > #define LS_STATEFUL_RUN_STOPWATCH_NAME "ls_stateful" > #define ADVERTISED_ROUTE_SYNC_RUN_STOPWATCH_NAME "advertised_route_sync" > #define LEARNED_ROUTE_SYNC_RUN_STOPWATCH_NAME "learned_route_sync" > +#define DYNAMIC_ROUTES_RUN_STOPWATCH_NAME "dynamic_routes" > > #endif > diff --git a/northd/en-advertised-route-sync.c > b/northd/en-advertised-route-sync.c > index cfea7c39b..a984e70a8 100644 > --- a/northd/en-advertised-route-sync.c > +++ b/northd/en-advertised-route-sync.c > @@ -30,7 +30,8 @@ advertised_route_table_sync( > struct ovsdb_idl_txn *ovnsb_txn, > const struct sbrec_advertised_route_table > *sbrec_advertised_route_table, > const struct lr_stateful_table *lr_stateful_table, > - const struct hmap *parsed_routes, > + const struct hmap *routes, > + const struct hmap *dynamic_routes, > struct advertised_route_sync_data *data); > > bool > @@ -141,6 +142,8 @@ en_advertised_route_sync_run(struct engine_node *node, > void *data OVS_UNUSED) > struct advertised_route_sync_data *routes_sync_data = data; > struct routes_data *routes_data > = engine_get_input_data("routes", node); > + struct dynamic_routes_data *dynamic_routes_data > + = engine_get_input_data("dynamic_routes", node); > const struct engine_context *eng_ctx = engine_get_context(); > const struct sbrec_advertised_route_table > *sbrec_advertised_route_table = > EN_OVSDB_GET(engine_get_input("SB_advertised_route", node)); > @@ -153,12 +156,75 @@ en_advertised_route_sync_run(struct engine_node > *node, void *data OVS_UNUSED) > sbrec_advertised_route_table, > &lr_stateful_data->table, > &routes_data->parsed_routes, > + &dynamic_routes_data->parsed_routes, > routes_sync_data); > > stopwatch_stop(ADVERTISED_ROUTE_SYNC_RUN_STOPWATCH_NAME, time_msec()); > engine_set_node_state(node, EN_UPDATED); > } > > +void > +*en_dynamic_routes_init(struct engine_node *node OVS_UNUSED, > + struct engine_arg *arg OVS_UNUSED) > +{ > + struct dynamic_routes_data *data = xmalloc(sizeof *data); > + *data = (struct dynamic_routes_data) { > + .parsed_routes = HMAP_INITIALIZER(&data->parsed_routes), > + }; > + > + return data; > +} > + > +static void > +en_dynamic_routes_clean(struct dynamic_routes_data *data) > +{ > + struct parsed_route *r; > + HMAP_FOR_EACH_POP (r, key_node, &data->parsed_routes) { > + parsed_route_free(r); > + } > +} > +void > +en_dynamic_routes_cleanup(void *data_) > +{ > + struct dynamic_routes_data *data = data_; > + > + en_dynamic_routes_clean(data); > + hmap_destroy(&data->parsed_routes); > +} > + > +void > +en_dynamic_routes_run(struct engine_node *node, void *data) > +{ > + struct dynamic_routes_data *dynamic_routes_data = data; > + struct northd_data *northd_data = engine_get_input_data("northd", > node); > + struct ed_type_lr_stateful *lr_stateful_data = > + engine_get_input_data("lr_stateful", node); > + > + en_dynamic_routes_clean(data); > + const struct lr_stateful_record *lr_stateful_rec; > + HMAP_FOR_EACH (lr_stateful_rec, key_node, > + &lr_stateful_data->table.entries) { > + const struct ovn_datapath *od = > + ovn_datapaths_find_by_index(&northd_data->lr_datapaths, > + lr_stateful_rec->lr_index); > + if (!od->dynamic_routing) { > + continue; > + } > + build_nat_parsed_routes(od, lr_stateful_rec->lrnat_rec, > + &northd_data->ls_ports, > + &dynamic_routes_data->parsed_routes); > + build_nat_connected_parsed_routes(od, &lr_stateful_data->table, > + &northd_data->ls_ports, > + > &dynamic_routes_data->parsed_routes); > + > + build_lb_parsed_routes(od, lr_stateful_rec->lb_ips, > + &dynamic_routes_data->parsed_routes); > + build_lb_connected_parsed_routes(od, &lr_stateful_data->table, > + > &dynamic_routes_data->parsed_routes); > + } > + engine_set_node_state(node, EN_UPDATED); > +} > + > struct ar_entry { > struct hmap_node hmap_node; > > @@ -333,12 +399,89 @@ publish_host_routes(struct hmap *sync_routes, > } > } > > +static void > +advertised_route_table_sync_route_add( > + const struct lr_stateful_table *lr_stateful_table, > + struct advertised_route_sync_data *data, > + struct uuidset *host_route_lrps, > + struct hmap *sync_routes, > + const struct parsed_route *route) > +{ > + if (route->is_discard_route) { > + return; > + } > + if (prefix_is_link_local(&route->prefix, route->plen)) { > + return; > + } > + if (!route->od->dynamic_routing) { > + return; > + } > + > + enum dynamic_routing_redistribute_mode drr = > + route->out_port->dynamic_routing_redistribute; > + if (route->source == ROUTE_SOURCE_CONNECTED) { > + if (!drr_mode_CONNECTED_is_set(drr)) { > + return; > + } > + /* If we advertise host routes, we only need to do so once per > + * LRP. */ > + const struct uuid *lrp_uuid = > &route->out_port->nbrp->header_.uuid; > + if (drr_mode_CONNECTED_AS_HOST_is_set(drr) && > + !uuidset_contains(host_route_lrps, lrp_uuid)) { > + uuidset_insert(host_route_lrps, lrp_uuid); > + publish_host_routes(sync_routes, lr_stateful_table, route, > data); > + return; > + } > + } > + if (route->source == ROUTE_SOURCE_STATIC && > !drr_mode_STATIC_is_set(drr)) { > + return; > + } > + if (route->source == ROUTE_SOURCE_NAT) { > + if (!drr_mode_NAT_is_set(drr)) { > + return; > + } > + /* If NAT route tracks port on a different DP than the one that > + * advertises the route, we need to watch for changes on that DP > as > + * well. */ > + if (route->tracked_port && route->tracked_port->od != route->od) { > + if (route->tracked_port->od->nbr) { > + uuidset_insert(&data->nb_lr, > + > &route->tracked_port->od->nbr->header_.uuid); > + } else if (route->tracked_port->od->nbs) { > + uuidset_insert(&data->nb_ls, > + > &route->tracked_port->od->nbs->header_.uuid); > + } > + } > + } > + if (route->source == ROUTE_SOURCE_LB) { > + if (!drr_mode_LB_is_set(drr)) { > + return; > + } > + /* If LB route tracks port on a different DP than the one that > + * advertises the route, we need to watch for changes on that DP > as > + * well. */ > + if (route->tracked_port && route->tracked_port->od != route->od) { > + uuidset_insert(&data->nb_lr, > + &route->tracked_port->od->nbr->header_.uuid); > + } > + } > + > + char *ip_prefix = normalize_v46_prefix(&route->prefix, route->plen); > + const struct sbrec_port_binding *tracked_port = NULL; > + if (route->tracked_port) { > + tracked_port = route->tracked_port->sb; > + } > + ar_add_entry(sync_routes, route->od->sb, route->out_port->sb, > + ip_prefix, tracked_port); > +} > + > static void > advertised_route_table_sync( > struct ovsdb_idl_txn *ovnsb_txn, > const struct sbrec_advertised_route_table > *sbrec_advertised_route_table, > const struct lr_stateful_table *lr_stateful_table, > - const struct hmap *parsed_routes, > + const struct hmap *routes, > + const struct hmap *dynamic_routes, > struct advertised_route_sync_data *data) > { > struct hmap sync_routes = HMAP_INITIALIZER(&sync_routes); > @@ -346,47 +489,24 @@ advertised_route_table_sync( > const struct parsed_route *route; > > struct ar_entry *route_e; > - const struct sbrec_advertised_route *sb_route; > - HMAP_FOR_EACH (route, key_node, parsed_routes) { > - if (route->is_discard_route) { > - continue; > - } > - if (prefix_is_link_local(&route->prefix, route->plen)) { > - continue; > - } > - if (!route->od->dynamic_routing) { > - continue; > - } > > - enum dynamic_routing_redistribute_mode drr = > - route->out_port->dynamic_routing_redistribute; > - if (route->source == ROUTE_SOURCE_CONNECTED) { > - if (!drr_mode_CONNECTED_is_set(drr)) { > - continue; > - } > - /* If we advertise host routes, we only need to do so once per > - * LRP. */ > - const struct uuid *lrp_uuid = > - &route->out_port->nbrp->header_.uuid; > - if (drr_mode_CONNECTED_AS_HOST_is_set(drr) && > - !uuidset_contains(&host_route_lrps, lrp_uuid)) { > - uuidset_insert(&host_route_lrps, lrp_uuid); > - publish_host_routes(&sync_routes, lr_stateful_table, > - route, data); > - continue; > - } > - } > - if (route->source == ROUTE_SOURCE_STATIC && > - !drr_mode_STATIC_is_set(drr)) { > - continue; > - } > - > - char *ip_prefix = normalize_v46_prefix(&route->prefix, > route->plen); > - route_e = ar_add_entry(&sync_routes, route->od->sb, > - route->out_port->sb, ip_prefix, NULL); > + /* First build the set of non-dynamic routes that need sync-ing. */ > + HMAP_FOR_EACH (route, key_node, routes) { > + advertised_route_table_sync_route_add(lr_stateful_table, > + data, &host_route_lrps, > + &sync_routes, > + route); > + } > + /* Then add the set of dynamic routes that need sync-ing. */ > + HMAP_FOR_EACH (route, key_node, dynamic_routes) { > + advertised_route_table_sync_route_add(lr_stateful_table, > + data, &host_route_lrps, > + &sync_routes, > + route); > } > uuidset_destroy(&host_route_lrps); > > + const struct sbrec_advertised_route *sb_route; > SBREC_ADVERTISED_ROUTE_TABLE_FOR_EACH_SAFE (sb_route, > > sbrec_advertised_route_table) { > route_e = ar_find(&sync_routes, sb_route->datapath, > diff --git a/northd/en-advertised-route-sync.h > b/northd/en-advertised-route-sync.h > index 1f24fd329..0290f08d2 100644 > --- a/northd/en-advertised-route-sync.h > +++ b/northd/en-advertised-route-sync.h > @@ -36,4 +36,7 @@ void *en_advertised_route_sync_init(struct engine_node > *, struct engine_arg *); > void en_advertised_route_sync_cleanup(void *data); > void en_advertised_route_sync_run(struct engine_node *, void *data); > > +void *en_dynamic_routes_init(struct engine_node *, struct engine_arg *); > +void en_dynamic_routes_cleanup(void *data); > +void en_dynamic_routes_run(struct engine_node *, void *data); > #endif /* EN_ADVERTISED_ROUTE_SYNC_H */ > diff --git a/northd/en-learned-route-sync.c > b/northd/en-learned-route-sync.c > index 406f1551f..4e87b3265 100644 > --- a/northd/en-learned-route-sync.c > +++ b/northd/en-learned-route-sync.c > @@ -181,7 +181,8 @@ parse_route_from_sbrec_route(struct hmap > *parsed_routes_out, > > parsed_route_add(od, nexthop, &prefix, plen, false, lrp_addr_s, > out_port, 0, false, false, NULL, > - ROUTE_SOURCE_LEARNED, &route->header_, > parsed_routes_out); > + ROUTE_SOURCE_LEARNED, &route->header_, NULL, > + parsed_routes_out); > } > > static void > diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c > index 438daf1c6..e94e98f78 100644 > --- a/northd/inc-proc-northd.c > +++ b/northd/inc-proc-northd.c > @@ -175,6 +175,7 @@ static ENGINE_NODE(multicast_igmp, "multicast_igmp"); > static ENGINE_NODE(acl_id, "acl_id"); > static ENGINE_NODE(advertised_route_sync, "advertised_route_sync"); > static ENGINE_NODE(learned_route_sync, "learned_route_sync"); > +static ENGINE_NODE(dynamic_routes, "dynamic_routes"); > > void inc_proc_northd_init(struct ovsdb_idl_loop *nb, > struct ovsdb_idl_loop *sb) > @@ -289,7 +290,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, > engine_add_input(&en_ecmp_nexthop, &en_sb_mac_binding, > ecmp_nexthop_mac_binding_handler); > > + engine_add_input(&en_dynamic_routes, &en_lr_stateful, NULL); > + engine_add_input(&en_dynamic_routes, &en_northd, engine_noop_handler); > We usually add a comment explaining why noop is fine. > + > engine_add_input(&en_advertised_route_sync, &en_routes, NULL); > + engine_add_input(&en_advertised_route_sync, &en_dynamic_routes, NULL); > engine_add_input(&en_advertised_route_sync, &en_sb_advertised_route, > NULL); > engine_add_input(&en_advertised_route_sync, &en_lr_stateful, > diff --git a/northd/northd.c b/northd/northd.c > index 1d9f0675e..1558654a0 100644 > --- a/northd/northd.c > +++ b/northd/northd.c > @@ -848,6 +848,14 @@ parse_dynamic_routing_redistribute( > out |= DRRM_STATIC; > continue; > } > + if (!strcmp(token, "nat")) { > + out |= DRRM_NAT; > + continue; > + } > + if (!strcmp(token, "lb")) { > + out |= DRRM_LB; > + continue; > + } > static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); > VLOG_WARN_RL(&rl, > "unkown dynamic-routing-redistribute option '%s' on > %s", > @@ -11000,6 +11008,7 @@ parsed_route_init(const struct ovn_datapath *od, > bool ecmp_symmetric_reply, > const struct sset *ecmp_selection_fields, > enum route_source source, > + const struct ovn_port *tracked_port, > const struct ovsdb_idl_row *source_hint) > { > > @@ -11015,6 +11024,7 @@ parsed_route_init(const struct ovn_datapath *od, > new_pr->is_discard_route = is_discard_route; > new_pr->lrp_addr_s = nullable_xstrdup(lrp_addr_s); > new_pr->out_port = out_port; > + new_pr->tracked_port = tracked_port; > new_pr->source = source; > if (ecmp_selection_fields) { > sset_clone(&new_pr->ecmp_selection_fields, ecmp_selection_fields); > @@ -11040,7 +11050,7 @@ parsed_route_clone(const struct parsed_route *pr) > pr->od, nexthop, pr->prefix, pr->plen, pr->is_discard_route, > pr->lrp_addr_s, pr->out_port, pr->route_table_id, > pr->is_src_route, > pr->ecmp_symmetric_reply, &pr->ecmp_selection_fields, pr->source, > - pr->source_hint); > + pr->tracked_port, pr->source_hint); > > new_pr->hash = pr->hash; > return new_pr; > @@ -11083,13 +11093,14 @@ parsed_route_add(const struct ovn_datapath *od, > const struct sset *ecmp_selection_fields, > enum route_source source, > const struct ovsdb_idl_row *source_hint, > + const struct ovn_port *tracked_port, > struct hmap *routes) > { > > struct parsed_route *new_pr = parsed_route_init( > od, nexthop, *prefix, plen, is_discard_route, lrp_addr_s, > out_port, > route_table_id, is_src_route, ecmp_symmetric_reply, > - ecmp_selection_fields, source, source_hint); > + ecmp_selection_fields, source, tracked_port, source_hint); > > new_pr->hash = route_hash(new_pr); > > @@ -11226,7 +11237,7 @@ parsed_routes_add_static(const struct ovn_datapath > *od, > parsed_route_add(od, nexthop, &prefix, plen, is_discard_route, > lrp_addr_s, > out_port, route_table_id, is_src_route, > ecmp_symmetric_reply, &ecmp_selection_fields, source, > - &route->header_, routes); > + &route->header_, NULL, routes); > sset_destroy(&ecmp_selection_fields); > } > > @@ -11244,7 +11255,7 @@ parsed_routes_add_connected(const struct > ovn_datapath *od, > false, addr->addr_s, op, > 0, false, > false, NULL, ROUTE_SOURCE_CONNECTED, > - &op->nbrp->header_, routes); > + &op->nbrp->header_, NULL, routes); > } > > for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { > @@ -11256,7 +11267,7 @@ parsed_routes_add_connected(const struct > ovn_datapath *od, > false, addr->addr_s, op, > 0, false, > false, NULL, ROUTE_SOURCE_CONNECTED, > - &op->nbrp->header_, routes); > + &op->nbrp->header_, NULL, routes); > } > } > > @@ -11294,6 +11305,237 @@ build_parsed_routes(const struct ovn_datapath > *od, const struct hmap *lr_ports, > } > } > > +/* This function adds new parsed route for each entry in lr_nat record > + * to "routes". Logical port of the route is set to "advertising_op" and > + * tracked port is set to NAT's distributed gw port. If NAT doesn't have > + * DGP (for example if it's set on gateway router), no tracked port will > + * be set.*/ > +static void > +build_nat_parsed_route_for_port(const struct ovn_port *advertising_op, > + const struct lr_nat_record *lr_nat, > + const struct hmap *ls_ports, > + struct hmap *routes) > +{ > + const struct ovn_datapath *advertising_od = advertising_op->od; > + > + for (size_t i = 0; i < lr_nat->n_nat_entries; i++) { > + const struct ovn_nat *nat = &lr_nat->nat_entries[i]; > + int plen = nat_entry_is_v6(nat) ? 128 : 32; > + struct in6_addr prefix; > + ip46_parse(nat->nb->external_ip, &prefix); > + > + const struct ovn_port *tracked_port = > + nat->is_distributed && nat->is_valid > + ? ovn_port_find(ls_ports, nat->nb->logical_port) > + : nat->l3dgw_port; > Why do we still advertise NAT entries if they're not valid? There won't be any flow installed for invalid NAT. IMO we should skip invalid completely. > + parsed_route_add(advertising_od, NULL, &prefix, plen, false, > + nat->nb->external_ip, advertising_op, 0, false, > + false, NULL, ROUTE_SOURCE_NAT, &nat->nb->header_, > + tracked_port, routes); > + } > +} > + > +/* Generate parsed routes for NAT external IPs in lr_nat, for each ovn > port > + * in "od" that has enabled redistribution of NAT adresses.*/ > +void > +build_nat_parsed_routes(const struct ovn_datapath *od, > + const struct lr_nat_record *lr_nat, > + const struct hmap *ls_ports, > + struct hmap *routes) > +{ > + const struct ovn_port *op; > + HMAP_FOR_EACH (op, dp_node, &od->ports) { > + if (!drr_mode_NAT_is_set(op->dynamic_routing_redistribute)) { > + continue; > + } > + > + build_nat_parsed_route_for_port(op, lr_nat, ls_ports, routes); > + } > +} > + > +/* Similar to build_nat_parsed_routes, this function generates parsed > routes > + * for nat records in neighboring routers. For each ovn port in "od" that > has > + * enabled redistribution of NAT adresses, look up their neighbors (either > + * directly routers, or routers connected through common LS) and advertise > + * thier external NAT IPs too.*/ > +void > +build_nat_connected_parsed_routes( > + const struct ovn_datapath *od, > + const struct lr_stateful_table *lr_stateful_table, > + const struct hmap *ls_ports, > + struct hmap *routes) > +{ > + const struct ovn_port *op; > + HMAP_FOR_EACH (op, dp_node, &od->ports) { > + if (!drr_mode_NAT_is_set(op->dynamic_routing_redistribute)) { > + continue; > + } > + > + if (!op->peer) { > + continue; > + } > + > + struct ovn_datapath *peer_od = op->peer->od; > + if (!peer_od->nbs && !peer_od->nbr) { > + continue; > + } > This shouldn't be possible e.g. the datapath will always be either router or switch. > + > + const struct ovn_port *peer_port = NULL; > + /* This is a directly connected LR peer. */ > + if (peer_od->nbr) { > + peer_port = op->peer; > + > + const struct lr_stateful_record *peer_lr_stateful = > + lr_stateful_table_find_by_index(lr_stateful_table, > + peer_od->index); > + if (!peer_lr_stateful) { > + continue; > + } > + > + /* Advertise peer's NAT routes via the local port too. */ > + build_nat_parsed_route_for_port(op, > peer_lr_stateful->lrnat_rec, > + ls_ports, routes); > + return; > + } > + > + /* This peer is LSP, we need to check all connected router ports > + * for NAT.*/ > + for (size_t i = 0; i < peer_od->n_router_ports; i++) { > + peer_port = peer_od->router_ports[i]->peer; > + if (peer_port == op) { > + /* Skip advertising router. */ > + continue; > + } > + > + const struct lr_stateful_record *peer_lr_stateful = > + lr_stateful_table_find_by_index(lr_stateful_table, > + peer_port->od->index); > + if (!peer_lr_stateful) { > + continue; > + } > + > + /* Advertise peer's NAT routes via the local port too. */ > + build_nat_parsed_route_for_port(op, > peer_lr_stateful->lrnat_rec, > + ls_ports, routes); > + } > + } > +} > + > +/* This function adds new parsed route for each IP in lb_ips to > "routes".*/ > +static void > +build_lb_parsed_route_for_port(const struct ovn_port *advertising_op, > + const struct ovn_port *tracked_port, > + const struct ovn_lb_ip_set *lb_ips, > + struct hmap *routes) > +{ > + const struct ovn_datapath *advertising_od = advertising_op->od; > + > + const char *ip_address; > + SSET_FOR_EACH (ip_address, &lb_ips->ips_v4) { > + struct in6_addr prefix; > + ip46_parse(ip_address, &prefix); > + parsed_route_add(advertising_od, NULL, &prefix, 32, false, > + ip_address, advertising_op, 0, false, false, > + NULL, ROUTE_SOURCE_LB, > &advertising_op->nbrp->header_, > + tracked_port, routes); > + } > + SSET_FOR_EACH (ip_address, &lb_ips->ips_v6) { > + struct in6_addr prefix; > + ip46_parse(ip_address, &prefix); > + parsed_route_add(advertising_od, NULL, &prefix, 128, false, > + ip_address, advertising_op, 0, false, false, > + NULL, ROUTE_SOURCE_LB, > &advertising_op->nbrp->header_, > + tracked_port, routes); > + } > +} > + > +/* Similar to build_lb_parsed_routes, this function generates parsed > routes > + * for LB VIPs of neighboring routers. For each ovn port in "od" that has > + * enabled redistribution of LB VIPs, look up their neighbors (either > + * directly routers, or routers connected through common LS) and advertise > + * thier LB VIPs too.*/ > +void > +build_lb_connected_parsed_routes( > + const struct ovn_datapath *od, > + const struct lr_stateful_table *lr_stateful_table, > + struct hmap *routes) > +{ > + const struct ovn_port *op; > + HMAP_FOR_EACH (op, dp_node, &od->ports) { > + if (!drr_mode_LB_is_set(op->dynamic_routing_redistribute)) { > + continue; > + } > + > + if (!op->peer) { > + continue; > + } > + > + struct ovn_datapath *peer_od = op->peer->od; > + if (!peer_od->nbs && !peer_od->nbr) { > + continue; > + } > + > + const struct lr_stateful_record *lr_stateful_rec; > + const struct ovn_port *peer_port = NULL; > + /* This is directly connected LR peer. */ > + if (peer_od->nbr) { > + lr_stateful_rec = lr_stateful_table_find_by_index( > + lr_stateful_table, peer_od->index); > + peer_port = op->peer; > + build_lb_parsed_route_for_port(op, peer_port, > + lr_stateful_rec->lb_ips, > routes); > + return; > + } > + > + /* This peer is LSP, we need to check all connected router ports > for > + * LBs.*/ > + for (size_t i = 0; i < peer_od->n_router_ports; i++) { > + peer_port = peer_od->router_ports[i]->peer; > + if (peer_port == op) { > + /* no need to check for LBs on ovn_port that initiated > this > + * function.*/ > + continue; > + } > + lr_stateful_rec = lr_stateful_table_find_by_index( > + lr_stateful_table, peer_port->od->index); > + > + build_lb_parsed_route_for_port(op, peer_port, > + lr_stateful_rec->lb_ips, > routes); > + } > + } > +} > + > +void > +build_lb_parsed_routes(const struct ovn_datapath *od, > + const struct ovn_lb_ip_set *lb_ips, > + struct hmap *routes) > +{ > + const struct ovn_port *op; > + HMAP_FOR_EACH (op, dp_node, &od->ports) { > + if (!drr_mode_LB_is_set(op->dynamic_routing_redistribute)) { > + continue; > + } > + > + /* Traffic processed by a load balancer is: > + * - handled by the chassis where a gateway router is bound > + * OR > + * - always redirected to a distributed gateway router port > + * > + * Advertise the LB IPs via all 'op' if this is a gateway router > or > + * throuh all DGPs of this distributed router otherwise. */ > + struct ovn_port *op_ = NULL; > + size_t n_tracked_ports = !od->is_gw_router ? od->n_l3dgw_ports : > 1; > + struct ovn_port **tracked_ports = !od->is_gw_router > + ? od->l3dgw_ports > + : &op_; > + > + for (size_t i = 0; i < n_tracked_ports; i++) { > + build_lb_parsed_route_for_port(op, tracked_ports[i], lb_ips, > + routes); > + } > + } > + > +} > struct ecmp_route_list_node { > struct ovs_list list_node; > uint16_t id; /* starts from 1 */ > @@ -11468,6 +11710,8 @@ route_source_to_offset(enum route_source source) > return ROUTE_PRIO_OFFSET_STATIC; > case ROUTE_SOURCE_LEARNED: > return ROUTE_PRIO_OFFSET_LEARNED; > + case ROUTE_SOURCE_NAT: > + case ROUTE_SOURCE_LB: > default: > OVS_NOT_REACHED(); > } > diff --git a/northd/northd.h b/northd/northd.h > index b984e124d..1bb0cbd6f 100644 > --- a/northd/northd.h > +++ b/northd/northd.h > @@ -186,11 +186,15 @@ struct route_policy { > }; > > struct routes_data { > - struct hmap parsed_routes; > + struct hmap parsed_routes; /* Stores struct parsed_route. */ > struct simap route_tables; > struct hmap bfd_active_connections; > }; > > +struct dynamic_routes_data { > + struct hmap parsed_routes; /* Stores struct parsed_route. */ > +}; > + > struct route_policies_data { > struct hmap route_policies; > struct hmap bfd_active_connections; > @@ -308,10 +312,12 @@ struct mcast_port_info { > * (e.g., IGMP join/leave). */ > }; > > -#define DRR_MODES \ > - DRR_MODE(CONNECTED, 0) \ > +#define DRR_MODES \ > + DRR_MODE(CONNECTED, 0) \ > DRR_MODE(CONNECTED_AS_HOST, 1) \ > - DRR_MODE(STATIC, 2) > + DRR_MODE(STATIC, 2) \ > + DRR_MODE(NAT, 3) \ > + DRR_MODE(LB, 4) > > enum dynamic_routing_redistribute_mode_bits { > #define DRR_MODE(PROTOCOL, BIT) DRRM_##PROTOCOL##_BIT = BIT, > @@ -746,6 +752,10 @@ enum route_source { > ROUTE_SOURCE_STATIC, > /* The route is dynamically learned by an ovn-controller. */ > ROUTE_SOURCE_LEARNED, > + /* The route is derived from a NAT's external IP. */ > + ROUTE_SOURCE_NAT, > + /* The route is derived from a LB's VIP. */ > + ROUTE_SOURCE_LB, > }; > > struct parsed_route { > @@ -765,6 +775,7 @@ struct parsed_route { > const struct ovsdb_idl_row *source_hint; > char *lrp_addr_s; > const struct ovn_port *out_port; > + const struct ovn_port *tracked_port; /* May be NULL. */ > }; > > struct parsed_route *parsed_route_clone(const struct parsed_route *); > @@ -784,6 +795,7 @@ void parsed_route_add(const struct ovn_datapath *od, > const struct sset *ecmp_selection_fields, > enum route_source source, > const struct ovsdb_idl_row *source_hint, > + const struct ovn_port *tracked_port, > struct hmap *routes); > > bool > @@ -818,6 +830,20 @@ void route_policies_destroy(struct > route_policies_data *); > void build_parsed_routes(const struct ovn_datapath *, const struct hmap *, > const struct hmap *, struct hmap *, struct simap > *, > struct hmap *); > +void build_nat_parsed_routes(const struct ovn_datapath *, > + const struct lr_nat_record *, > + const struct hmap *ls_ports, > + struct hmap *routes); > +void build_nat_connected_parsed_routes(const struct ovn_datapath *, > + const struct lr_stateful_table *, > + const struct hmap *ls_ports, > + struct hmap *routes); > +void build_lb_parsed_routes(const struct ovn_datapath *, > + const struct ovn_lb_ip_set *, > + struct hmap *); > +void build_lb_connected_parsed_routes(const struct ovn_datapath *, > + const struct lr_stateful_table *, > + struct hmap *routes); > uint32_t get_route_table_id(struct simap *, const char *); > void routes_init(struct routes_data *); > void routes_destroy(struct routes_data *); > diff --git a/ovn-nb.xml b/ovn-nb.xml > index 20d30dd58..f10c1e738 100644 > --- a/ovn-nb.xml > +++ b/ovn-nb.xml > @@ -3101,6 +3101,24 @@ or > <ref table="Advertised_Route" db="OVN_Southbound"/> table. > </p> > > + <p> > + If <code>lb</code> is in the list then northd will create > entries in > + <ref table="Advertised_Route" db="OVN_Southbound"/> table for > each > + Load Balancer VIP on this router and it's neighboring routers. > + Neighboring routers are those that are either directly > connected, > + via Logical Router Port, or those that are connected via shared > + Logical Switch. > + </p> > + > + <p> > + If <code>nat</code> is in the list then northd will create > entries in > + <ref table="Advertised_Route" db="OVN_Southbound"/> table for > each > + NAT's external IP on this router and it's neighboring routers. > + Neighboring routers are those that are either directly > connected, > + via Logical Router Port, or those that are connected via shared > + Logical Switch. > + </p> > + > <p> > This value can be overwritten on a per LRP basis using > <ref column="options" key="dynamic-routing-redistribute" > @@ -3950,6 +3968,24 @@ or > <ref table="Advertised_Route" db="OVN_Southbound"/> table. > </p> > > + <p> > + If <code>lb</code> is in the list then northd will create > entries in > + <ref table="Advertised_Route" db="OVN_Southbound"/> table for > each > + Load Balancer VIP on this port's router, and it's neighboring > + routers. Neighboring routers are those that are either directly > + connected to this Logical Router Port, or those that are > connected > + via shared Logical Switch. > + </p> > + > + <p> > + If <code>nat</code> is in the list then northd will create > entries in > + <ref table="Advertised_Route" db="OVN_Southbound"/> table for > each > + NAT's external IP on this port's router, and it's neighboring > + routers. Neighboring routers are those that are either directly > + connected to this Logical Router Port, or those that are > connected > + via shared Logical Switch. > + </p> > + > <p> > If not set the value from <ref column="options" > key="dynamic-routing-redistribute" table="Logical_Router"/> on > the > diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at > index 64991ff75..a18321818 100644 > --- a/tests/ovn-northd.at > +++ b/tests/ovn-northd.at > @@ -15542,3 +15542,631 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > AT_CLEANUP > ]) > + > +OVN_FOR_EACH_NORTHD_NO_HV([ > +AT_SETUP([dynamic-routing - nat sync to sb IPv4]) > +AT_KEYWORDS([dynamic-routing]) > +ovn_start > + > +# Start with GW router and a single LRP. > +check ovn-nbctl lr-add lr0 > +check ovn-nbctl set Logical_Router lr0 \ > + options:dynamic-routing=true \ > + options:chassis=hv1 > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 > +check ovn-nbctl --wait=sb sync > + > +check_row_count Advertised_Route 0 > + > +datapath=$(fetch_column Datapath_Binding _uuid external_ids:name=lr0) > +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-sw0) > + > +# Adding LRP dynamic-routing-redistribute=nat option and NAT rule > advertises a > +# route entry. > +check ovn-nbctl \ > + -- lrp-set-options lr0-sw0 dynamic-routing-redistribute=nat \ > + -- lr-nat-add lr0 dnat_and_snat 172.16.1.10 192.168.1.10 > +check ovn-nbctl --wait=sb sync > + > +check_row_count Advertised_Route 1 > +check_row_count Advertised_Route 1 \ > + ip_prefix="172.16.1.10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=[[]] > + > +# Add LR with distributed LRP connected to GW router through join LS > +# and switch dynamic-routing-redistribute to lrp-guest-join so that > +# it advertises NAT routes from the neighbors in the join LS. > +check ovn-nbctl \ > + -- remove Logical_Router_Port lr0-sw0 options > dynamic-routing-redistribute \ > + -- lrp-add lr0 lr0-join 00:00:00:00:ff:02 10.42.0.1/24 \ > + -- lrp-set-options lr0-join dynamic-routing-redistribute=nat \ > + -- ls-add ls-join \ > + -- lsp-add ls-join lsp-join-to-lr0 \ > + -- lsp-set-type lsp-join-to-lr0 router \ > + -- lsp-set-options lsp-join-to-lr0 router-port=lr0-join \ > + -- lsp-set-addresses lsp-join-to-lr0 router \ > + -- lr-add lr-guest0 \ > + -- lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 10.51.0.1/24 \ > + -- lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 10.42.0.2/24 \ > + -- lsp-add ls-join lsp-join-to-guest0 \ > + -- lsp-set-type lsp-join-to-guest0 router \ > + -- lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \ > + -- lrp-set-gateway-chassis lrp-guest0-join hv1 > +check ovn-nbctl --wait=sb sync > + > +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-join) > +pb2=$(fetch_column Port_Binding _uuid logical_port=lrp-guest0-join) > + > +check ovn-nbctl \ > + --add-route lr-nat-add lr-guest0 dnat_and_snat 172.16.2.10 > 192.168.2.10 > +check ovn-nbctl --wait=sb sync > + > +check_row_count Advertised_Route 2 > + > +# Routes to local NAT addresses are advertised without tracked port. > +check_row_count Advertised_Route 1 \ > + ip_prefix="172.16.1.10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=[[]] > + > +# Routes to NAT addresses of neighboring routers are advertised with > +# tracked port pointing to the LRP of the neighboring router. > +check_row_count Advertised_Route 1 \ > + ip_prefix="172.16.2.10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=$pb2 > + > +# Add nonlocal LR with distributed LRP connected to GW router through > join LS. > +check ovn-nbctl \ > + -- lr-add lr-guest1 \ > + -- lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 10.51.1.1/24 \ > + -- lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 10.42.0.3/24 \ > + -- lsp-add ls-join lsp-join-to-guest1 \ > + -- lsp-set-type lsp-join-to-guest1 router \ > + -- lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \ > + -- lrp-set-gateway-chassis lrp-guest1-join nonlocalhv > +check ovn-nbctl --wait=sb sync > + > +pb3=$(fetch_column Port_Binding _uuid logical_port=lrp-guest1-join) > + > +check ovn-nbctl \ > + -- --add-route lr-nat-add lr-guest1 dnat_and_snat 172.16.3.10 > 192.168.3.10 > +check ovn-nbctl --wait=sb sync > + > +# Add a distributed router with distributed NAT connected to a different > +# public LS. > +check ovn-nbctl \ > + -- ls-add ls-pub \ > + -- lsp-add ls-pub ls-pub-ln \ > + -- lsp-set-type ls-pub-ln localnet \ > + -- lsp-set-addresses ls-pub-ln unknown \ > + -- lsp-set-options ls-pub-ln network_name=public \ > + -- lr-add dr \ > + -- set logical_router dr options:dynamic-routing=true \ > + -- lrp-add dr lrp-dr-sw1 00:00:00:00:fd:01 10.52.1.1/24 \ > + -- lrp-add dr lrp-dr-pub 00:00:00:00:fd:02 10.42.0.4/24 \ > + -- lrp-set-options lrp-dr-pub dynamic-routing-redistribute=nat \ > + -- lsp-add ls-pub lsp-pub-to-guest2 \ > + -- lsp-set-type lsp-pub-to-guest2 router \ > + -- lsp-set-options lsp-pub-to-guest2 router-port=lrp-dr-pub \ > + -- lrp-set-gateway-chassis lrp-dr-pub nonlocalhv \ > + -- ls-add sw1 \ > + -- lsp-add sw1 sw1-lrp-dr \ > + -- lsp-set-type sw1-lrp-dr router \ > + -- lsp-set-addresses sw1-lrp-dr router \ > + -- lsp-set-options sw1-lrp-dr router-port=lrp-dr-sw1 \ > + -- lsp-add sw1 vm > +check ovn-nbctl --wait=sb sync > + > +datapath_dr=$(fetch_column Datapath_Binding _uuid external_ids:name=dr) > +pb_dr=$(fetch_column Port_Binding _uuid logical_port=lrp-dr-pub) > +pb_vm=$(fetch_column Port_Binding _uuid logical_port=vm) > + > +check ovn-nbctl \ > + -- --add-route lr-nat-add dr dnat_and_snat 172.16.4.10 192.168.3.20 \ > + vm 00:00:00:00:fd:03 > +check ovn-nbctl --wait=sb sync > + > +check_row_count Advertised_Route 4 > + > +# Routes to local NAT addresses are advertised without tracked port. > +check_row_count Advertised_Route 1 \ > + ip_prefix="172.16.1.10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=[[]] > + > +# Routes to NAT addresses of neighboring routers are advertised with > +# tracked port pointing to the LRP of the neighboring router. > +check_row_count Advertised_Route 1 \ > + ip_prefix="172.16.2.10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=$pb2 > +check_row_count Advertised_Route 1 \ > + ip_prefix="172.16.3.10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=$pb3 > + > +# Routes to distributed NAT addresses are advertised with tracked port set > +# to the logical_port of the NAT entry. > +check_row_count Advertised_Route 1 \ > + ip_prefix="172.16.4.10" \ > + datapath=$datapath_dr \ > + logical_port=$pb_dr \ > + tracked_port=$pb_vm > + > +# Removing the option:dynamic-routing removes all routes. > +check ovn-nbctl --wait=sb remove Logical_Router lr0 option dynamic-routing > +check_row_count Advertised_Route 1 > + > +check ovn-nbctl --wait=sb remove Logical_Router dr option dynamic-routing > +check_row_count Advertised_Route 0 > + > +# And setting it again adds them again. > +check ovn-nbctl --wait=sb set Logical_Router lr0 > option:dynamic-routing=true > +check_row_count Advertised_Route 3 > + > +check ovn-nbctl --wait=sb set Logical_Router dr > option:dynamic-routing=true > +check_row_count Advertised_Route 4 > + > +# Removing the lr will remove all routes. > +check ovn-nbctl lr-del lr0 -- lr-del dr > +check ovn-nbctl --wait=sb sync > +check_row_count Advertised_Route 0 > + > +AT_CLEANUP > +]) > + > +OVN_FOR_EACH_NORTHD_NO_HV([ > +AT_SETUP([dynamic-routing - nat sync to sb IPv6]) > +AT_KEYWORDS([dynamic-routing]) > +ovn_start > + > +# Start with GW router and a single LRP. > +check ovn-nbctl lr-add lr0 > +check ovn-nbctl set Logical_Router lr0 \ > + options:dynamic-routing=true \ > + options:chassis=hv1 > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 2001:db0::1/64 > +check ovn-nbctl --wait=sb sync > + > +check_row_count Advertised_Route 0 > + > +datapath=$(fetch_column Datapath_Binding _uuid external_ids:name=lr0) > +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-sw0) > + > +# Adding LRP dynamic-routing-redistribute=nat option and NAT rule > advertises > +# a route entry. > +check ovn-nbctl \ > + -- lrp-set-options lr0-sw0 dynamic-routing-redistribute=nat \ > + -- lr-nat-add lr0 dnat_and_snat 2001:db1::10 2001:db2::10 > +check ovn-nbctl --wait=sb sync > + > +check_row_count Advertised_Route 1 > +check_row_count Advertised_Route 1 \ > + ip_prefix="2001\:db1\:\:10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=[[]] > + > +# Add LR with distributed LRP connected to GW router through join LS > +# and switch dynamic-routing-redistribute to lrp-guest-join so that > +# it advertises NAT routes from the neighbors in the join LS. > +check ovn-nbctl \ > + -- remove Logical_Router_Port lr0-sw0 options > dynamic-routing-redistribute \ > + -- lrp-add lr0 lr0-join 00:00:00:00:ff:02 2001:db42::1/64 \ > + -- lrp-set-options lr0-join dynamic-routing-redistribute=nat \ > + -- ls-add ls-join \ > + -- lsp-add ls-join lsp-join-to-lr0 \ > + -- lsp-set-type lsp-join-to-lr0 router \ > + -- lsp-set-options lsp-join-to-lr0 router-port=lr0-join \ > + -- lsp-set-addresses lsp-join-to-lr0 router \ > + -- lr-add lr-guest0 \ > + -- lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 2001:db51::1/64 > \ > + -- lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 > 2001:db42::2/64 \ > + -- lsp-add ls-join lsp-join-to-guest0 \ > + -- lsp-set-type lsp-join-to-guest0 router \ > + -- lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \ > + -- lrp-set-gateway-chassis lrp-guest0-join hv1 > +check ovn-nbctl --wait=sb sync > + > +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-join) > +pb2=$(fetch_column Port_Binding _uuid logical_port=lrp-guest0-join) > + > +check ovn-nbctl \ > + --add-route lr-nat-add lr-guest0 dnat_and_snat 2001:db3::10 > 2001:db4::10 > +check ovn-nbctl --wait=sb sync > + > +check_row_count Advertised_Route 2 > + > +# Routes to local NAT addresses are advertised without tracked port. > +check_row_count Advertised_Route 1 \ > + ip_prefix="2001\:db1\:\:10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=[[]] > + > +# Routes to NAT addresses of neighboring routers are advertised with > +# tracked port pointing to the LRP of the neighboring router. > +check_row_count Advertised_Route 1 \ > + ip_prefix="2001\:db3\:\:10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=$pb2 > + > +# Add nonlocal LR with distributed LRP connected to GW router through > join LS. > +check ovn-nbctl \ > + -- lr-add lr-guest1 \ > + -- lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 2001:db52::1/64 > \ > + -- lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 > 2001:db42::3/64 \ > + -- lsp-add ls-join lsp-join-to-guest1 \ > + -- lsp-set-type lsp-join-to-guest1 router \ > + -- lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \ > + -- lrp-set-gateway-chassis lrp-guest1-join nonlocalhv > +check ovn-nbctl --wait=sb sync > + > +pb3=$(fetch_column Port_Binding _uuid logical_port=lrp-guest1-join) > + > +check ovn-nbctl \ > + --add-route lr-nat-add lr-guest1 dnat_and_snat 2001:db5::10 > 2001:db6::10 > +check ovn-nbctl --wait=sb sync > + > +# Add a distributed router with distributed NAT connected to a different > +# public LS. > +check ovn-nbctl \ > + -- ls-add ls-pub \ > + -- lsp-add ls-pub ls-pub-ln \ > + -- lsp-set-type ls-pub-ln localnet \ > + -- lsp-set-addresses ls-pub-ln unknown \ > + -- lsp-set-options ls-pub-ln network_name=public \ > + -- lr-add dr \ > + -- set logical_router dr options:dynamic-routing=true \ > + -- lrp-add dr lrp-dr-sw1 00:00:00:00:fd:01 2001:db53::1/64 \ > + -- lrp-add dr lrp-dr-pub 00:00:00:00:fd:02 2001:db42::4/64 \ > + -- lrp-set-options lrp-dr-pub dynamic-routing-redistribute=nat \ > + -- lsp-add ls-pub lsp-pub-to-guest2 \ > + -- lsp-set-type lsp-pub-to-guest2 router \ > + -- lsp-set-options lsp-pub-to-guest2 router-port=lrp-dr-pub \ > + -- lrp-set-gateway-chassis lrp-dr-pub nonlocalhv \ > + -- ls-add sw1 \ > + -- lsp-add sw1 sw1-lrp-dr \ > + -- lsp-set-type sw1-lrp-dr router \ > + -- lsp-set-addresses sw1-lrp-dr router \ > + -- lsp-set-options sw1-lrp-dr router-port=lrp-dr-sw1 \ > + -- lsp-add sw1 vm > +check ovn-nbctl --wait=sb sync > + > +datapath_dr=$(fetch_column Datapath_Binding _uuid external_ids:name=dr) > +pb_dr=$(fetch_column Port_Binding _uuid logical_port=lrp-dr-pub) > +pb_vm=$(fetch_column Port_Binding _uuid logical_port=vm) > + > +check ovn-nbctl \ > + -- --add-route lr-nat-add dr dnat_and_snat 2001:db6::10 2001:db6::20 \ > + vm 00:00:00:00:fd:03 > +check ovn-nbctl --wait=sb sync > + > +check_row_count Advertised_Route 4 > + > +# Routes to local NAT addresses are advertised without tracked port. > +check_row_count Advertised_Route 1 \ > + ip_prefix="2001\:db1\:\:10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=[[]] > + > +# Routes to NAT addresses of neighboring routers are advertised with > +# tracked port pointing to the LRP of the neighboring router. > +check_row_count Advertised_Route 1 \ > + ip_prefix="2001\:db3\:\:10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=$pb2 > +check_row_count Advertised_Route 1 \ > + ip_prefix="2001\:db5\:\:10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=$pb3 > + > +# Routes to distributed NAT addresses are advertised with tracked port set > +# to the logical_port of the NAT entry. > +check_row_count Advertised_Route 1 \ > + ip_prefix="2001\:db6\:\:10" \ > + datapath=$datapath_dr \ > + logical_port=$pb_dr \ > + tracked_port=$pb_vm > + > +# Removing the option:dynamic-routing removes all routes. > +check ovn-nbctl --wait=sb remove Logical_Router lr0 option dynamic-routing > +check_row_count Advertised_Route 1 > + > +check ovn-nbctl --wait=sb remove Logical_Router dr option dynamic-routing > +check_row_count Advertised_Route 0 > + > +# And setting it again adds them again. > +check ovn-nbctl --wait=sb set Logical_Router lr0 > option:dynamic-routing=true > +check_row_count Advertised_Route 3 > + > +check ovn-nbctl --wait=sb set Logical_Router dr > option:dynamic-routing=true > +check_row_count Advertised_Route 4 > + > +# Removing the lr will remove all routes. > +check ovn-nbctl lr-del lr0 -- lr-del dr > +check ovn-nbctl --wait=sb sync > +check_row_count Advertised_Route 0 > + > +AT_CLEANUP > +]) > + > +OVN_FOR_EACH_NORTHD_NO_HV([ > +AT_SETUP([dynamic-routing - LB sync to sb IPv4]) > +AT_KEYWORDS([dynamic-routing]) > +ovn_start > + > +# Start with GW router and a single LRP, > +check ovn-nbctl lr-add lr0 > +check ovn-nbctl set Logical_Router lr0 \ > + options:dynamic-routing=true \ > + options:chassis=hv1 > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 > +check ovn-nbctl --wait=sb sync > + > +check_row_count Advertised_Route 0 > + > +datapath=$(fetch_column Datapath_Binding _uuid external_ids:name=lr0) > +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-sw0) > + > +# Adding LRP dynamic-routing-lb-vips option and LB VIP rule advertises > +# a route entry. > +check ovn-nbctl lrp-set-options lr0-sw0 dynamic-routing-redistribute=lb \ > + -- lb-add lb0 172.16.1.10:80 192.168.1.10:80,192.168.1.11:80 \ > + -- lr-lb-add lr0 lb0 > +check ovn-nbctl --wait=sb sync > + > +check_row_count Advertised_Route 1 > +check_row_count Advertised_Route 1 \ > + ip_prefix="172.16.1.10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=[[]] > + > +# Add LR with distributed LRP connected to GW router through join LS > +# and switch dynamic-routing-redistribute to lrp-guest-join so that > +# it advertises LB VIP routes from the neighbors in the join LS. > +check ovn-nbctl \ > + -- remove Logical_Router_Port lr0-sw0 options > dynamic-routing-redistribute \ > + -- lrp-add lr0 lr0-join 00:00:00:00:ff:02 10.42.0.1/24 \ > + -- lrp-set-options lr0-join dynamic-routing-redistribute=lb \ > + -- ls-add ls-join \ > + -- lsp-add ls-join lsp-join-to-lr0 \ > + -- lsp-set-type lsp-join-to-lr0 router \ > + -- lsp-set-options lsp-join-to-lr0 router-port=lr0-join \ > + -- lsp-set-addresses lsp-join-to-lr0 router \ > + -- lr-add lr-guest0 \ > + -- lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 10.51.0.1/24 \ > + -- lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 10.42.0.2/24 \ > + -- lrp-set-options lrp-guest0-join dynamic-routing-lb-vips=true \ > + -- lsp-add ls-join lsp-join-to-guest0 \ > + -- lsp-set-type lsp-join-to-guest0 router \ > + -- lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \ > + -- lrp-set-gateway-chassis lrp-guest0-join hv1 > +check ovn-nbctl --wait=sb sync > + > +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-join) > +pb2=$(fetch_column Port_Binding _uuid logical_port=lrp-guest0-join) > + > +check ovn-nbctl \ > + --add-route lb-add lb1 172.16.2.10:80 192.168.2.10:80,192.168.2.11:80 > \ > + -- lr-lb-add lr-guest0 lb1 > +check ovn-nbctl --wait=sb sync > + > +check_row_count Advertised_Route 2 > + > +# Routes to local LB VIPs are advertised without tracked port. > +check_row_count Advertised_Route 1 \ > + ip_prefix="172.16.1.10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=[[]] > + > +# Routes to LB VIPs of neighboring routers are advertised with > +# tracked port pointing to the LRP of the neighboring router. > +check_row_count Advertised_Route 1 \ > + ip_prefix="172.16.2.10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=$pb2 > + > +# Add nonlocal LR with distributed LRP connected to GW router through > join LS. > +check ovn-nbctl \ > + -- lr-add lr-guest1 \ > + -- lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 10.51.1.1/24 \ > + -- lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 10.42.0.3/24 \ > + -- lsp-add ls-join lsp-join-to-guest1 \ > + -- lsp-set-type lsp-join-to-guest1 router \ > + -- lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \ > + -- lrp-set-gateway-chassis lrp-guest1-join nonlocalhv > +check ovn-nbctl --wait=sb sync > + > +pb3=$(fetch_column Port_Binding _uuid logical_port=lrp-guest1-join) > + > +check ovn-nbctl \ > + -- --add-route lb-add lb2 172.16.3.10:80 192.168.3.10:80, > 192.168.3.11:80 \ > + -- lr-lb-add lr-guest1 lb2 > +check ovn-nbctl --wait=sb sync > + > +check_row_count Advertised_Route 3 > + > +# Routes to local LB VIPs are advertised without tracked port. > +check_row_count Advertised_Route 1 \ > + ip_prefix="172.16.1.10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=[[]] > + > +# Routes to LB VIPs of neighboring routers are advertised with > +# tracked port pointing to the LRP of the neighboring router. > +check_row_count Advertised_Route 1 \ > + ip_prefix="172.16.2.10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=$pb2 > +check_row_count Advertised_Route 1 \ > + ip_prefix="172.16.3.10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=$pb3 > + > +# Removing the option:dynamic-routing removes all routes. > +check ovn-nbctl --wait=sb remove Logical_Router lr0 option dynamic-routing > +check_row_count Advertised_Route 0 > + > +# And setting it again adds them again. > +check ovn-nbctl --wait=sb set Logical_Router lr0 > option:dynamic-routing=true > +check_row_count Advertised_Route 3 > + > +# Removing the lr will remove all routes. > +check ovn-nbctl --wait=sb lr-del lr0 > +check_row_count Advertised_Route 0 > + > +AT_CLEANUP > +]) > + > +OVN_FOR_EACH_NORTHD_NO_HV([ > +AT_SETUP([dynamic-routing - LB sync to sb IPv6]) > +AT_KEYWORDS([dynamic-routing]) > +ovn_start > + > +# Start with GW router and a single LRP. > +check ovn-nbctl lr-add lr0 > +check ovn-nbctl set Logical_Router lr0 \ > + options:dynamic-routing=true \ > + options:chassis=hv1 > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 2001:db0::1/64 > +check ovn-nbctl --wait=sb sync > + > +check_row_count Advertised_Route 0 > + > +datapath=$(fetch_column Datapath_Binding _uuid external_ids:name=lr0) > +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-sw0) > + > +# Adding LRP dynamic-routing-lb-vips option and LB VIP rule advertises > +# a route entry. > +check ovn-nbctl lrp-set-options lr0-sw0 dynamic-routing-redistribute=lb > \ > + -- lb-add lb0 [[2001:db1::10]]:80 > [[2001:db2::10]]:80,[[2001:db2::11]]:80 \ > + -- lr-lb-add lr0 lb0 > +check ovn-nbctl --wait=sb sync > + > +check_row_count Advertised_Route 1 > +check_row_count Advertised_Route 1 \ > + ip_prefix="2001\:db1\:\:10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=[[]] > + > +# Add LR with distributed LRP connected to GW router through join LS > +# and switch dynamic-routing-redistribute to lrp-guest-join so that > +# it advertises LB VIP routes from the neighbors in the join LS. > +check ovn-nbctl \ > + -- remove Logical_Router_Port lr0-sw0 options > dynamic-routing-redistribute \ > + -- lrp-add lr0 lr0-join 00:00:00:00:ff:02 2001:db42::1/64 \ > + -- lrp-set-options lr0-join dynamic-routing-redistribute=lb \ > + -- ls-add ls-join \ > + -- lsp-add ls-join lsp-join-to-lr0 \ > + -- lsp-set-type lsp-join-to-lr0 router \ > + -- lsp-set-options lsp-join-to-lr0 router-port=lr0-join \ > + -- lsp-set-addresses lsp-join-to-lr0 router \ > + -- lr-add lr-guest0 \ > + -- lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 2001:db52::1/64 > \ > + -- lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 > 2001:db42::2/64 \ > + -- lrp-set-options lrp-guest0-join dynamic-routing-lb-vips=true \ > + -- lsp-add ls-join lsp-join-to-guest0 \ > + -- lsp-set-type lsp-join-to-guest0 router \ > + -- lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \ > + -- lrp-set-gateway-chassis lrp-guest0-join hv1 > +check ovn-nbctl --wait=sb sync > + > +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-join) > +pb2=$(fetch_column Port_Binding _uuid logical_port=lrp-guest0-join) > + > +check ovn-nbctl \ > + -- --add-route lb-add lb1 [[2001:db3::10]]:80 > [[2001:db4::10]]:80,[[2001:db4::11]]:80 \ > + -- lr-lb-add lr-guest0 lb1 > +check ovn-nbctl --wait=sb sync > + > +check_row_count Advertised_Route 2 > + > +# Routes to local LB VIPs are advertised without tracked port. > +check_row_count Advertised_Route 1 \ > + ip_prefix="2001\:db1\:\:10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=[[]] > + > +# Routes to LB VIPs of neighboring routers are advertised with > +# tracked port pointing to the LRP of the neighboring router. > +check_row_count Advertised_Route 1 \ > + ip_prefix="2001\:db3\:\:10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=$pb2 > + > +# Add nonlocal LR with distributed LRP connected to GW router through > join LS. > +check ovn-nbctl \ > + -- lr-add lr-guest1 \ > + -- lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 2001:db2::1/64 \ > + -- lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 > 2001:db42::3/64 \ > + -- lsp-add ls-join lsp-join-to-guest1 \ > + -- lsp-set-type lsp-join-to-guest1 router \ > + -- lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \ > + -- lrp-set-gateway-chassis lrp-guest1-join nonlocalhv > +check ovn-nbctl --wait=sb sync > + > +pb3=$(fetch_column Port_Binding _uuid logical_port=lrp-guest1-join) > + > +check ovn-nbctl \ > + -- --add-route lb-add lb2 [[2001:db5::10]]:80 > [[2001:db4::10]]:80,[[2001:db4::11]]:80 \ > + -- lr-lb-add lr-guest1 lb2 > +check ovn-nbctl --wait=sb sync > + > +check_row_count Advertised_Route 3 > + > +# Routes to local LB VIPs are advertised without tracked port. > +check_row_count Advertised_Route 1 \ > + ip_prefix="2001\:db1\:\:10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=[[]] > + > +# Routes to LB VIPs of neighboring routers are advertised with > +# tracked port pointing to the LRP of the neighboring router. > +check_row_count Advertised_Route 1 \ > + ip_prefix="2001\:db3\:\:10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=$pb2 > +check_row_count Advertised_Route 1 \ > + ip_prefix="2001\:db5\:\:10" \ > + datapath=$datapath \ > + logical_port=$pb \ > + tracked_port=$pb3 > + > +# Removing the option:dynamic-routing removes all routes. > +check ovn-nbctl --wait=sb remove Logical_Router lr0 option dynamic-routing > +check_row_count Advertised_Route 0 > + > +# And setting it again adds them again. > +check ovn-nbctl --wait=sb set Logical_Router lr0 > option:dynamic-routing=true > +check_row_count Advertised_Route 3 > + > +# Removing the lr will remove all routes. > +check ovn-nbctl --wait=sb lr-del lr0 > +check_row_count Advertised_Route 0 > + > +AT_CLEANUP > +]) > diff --git a/tests/system-ovn.at b/tests/system-ovn.at > index d24e50974..d7b16deda 100644 > --- a/tests/system-ovn.at > +++ b/tests/system-ovn.at > @@ -16561,3 +16561,741 @@ OVS_TRAFFIC_VSWITCHD_STOP(["/.*error > receiving.*/d > /.*terminating with signal 15.*/d"]) > AT_CLEANUP > ]) > + > +OVN_FOR_EACH_NORTHD([ > +AT_SETUP([dynamic-routing - LB VIPs IPv4]) > +AT_KEYWORDS([dynamic-routing]) > + > +CHECK_VRF() > +CHECK_CONNTRACK() > +CHECK_CONNTRACK_NAT() > + > +vrf=1000 > +VRF_RESERVE([$vrf]) > +ovn_start > +OVS_TRAFFIC_VSWITCHD_START() > +ADD_BR([br-int]) > +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone]) > + > +# Set external-ids in br-int needed for ovn-controller > +ovs-vsctl \ > + -- set Open_vSwitch . external-ids:system-id=hv1 \ > + -- set Open_vSwitch . > external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ > + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ > + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ > + -- set bridge br-int fail-mode=secure > other-config:disable-in-band=true > + > +# Start ovn-controller > +start_daemon ovn-controller > + > +ovn-appctl vlog/set route_exchange > +check ovn-nbctl -- lr-add R1 \ > + -- set Logical_Router R1 options:requested-tnl-key=$vrf > options:dynamic-routing=true > + > +check ovn-nbctl ls-add sw0 > +check ovn-nbctl ls-add public > + > +check ovn-nbctl --wait=hv sync > Missing in all tests: OVN_POPULATE_ARP wait_for_ports_up > + > +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) > Has this potential to flake? Wouldn't OVS_WAIT_UNTIL be better here? > + > +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 192.168.1.1/24 > +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 172.16.1.1/24 \ > + -- lrp-set-options rp-public \ > + dynamic-routing-maintain-vrf=true \ > + dynamic-routing-redistribute=lb > + > +check ovn-nbctl set logical_router R1 options:chassis=hv1 > + > +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \ > + type=router options:router-port=rp-sw0 \ > + -- lsp-set-addresses sw0-rp router > + > +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port > public-rp \ > + type=router options:router-port=rp-public \ > + -- lsp-set-addresses public-rp router > + > +check ovs-vsctl set Open_vSwitch . > external-ids:ovn-bridge-mappings=phynet:br-ext > + > +check ovn-nbctl lsp-add public public1 \ > + -- lsp-set-addresses public1 unknown \ > + -- lsp-set-type public1 localnet \ > + -- lsp-set-options public1 network_name=phynet > + > +check ovn-nbctl --wait=hv sync > + > +AT_CHECK([test `ip route show table $vrf | wc -l` -eq 1], [1]) > + > + > +# Create a load balancer and associate to R1 > +check ovn-nbctl lb-add lb1 172.16.1.150:80 172.16.1.100:80 > +check ovn-nbctl lr-lb-add R1 lb1 > + > +check ovn-nbctl --wait=hv sync > + > +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) > + > +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > [dnl > +blackhole 172.16.1.150 proto 84 metric 1000]) > + > +# Add distributed router connected through "join" LS and ensure > +# that its LB VIPs are redistributed by R1. LS join has no > +# IP config, routers reach each other over IPv6 LLAs > +check ovn-nbctl --wait=sb \ > + -- \ > + remove Logical_Router_Port rp-public options > dynamic-routing-redistribute \ > + -- \ > + lrp-add R1 r1-join 00:00:00:00:ff:02 \ > + -- \ > + lrp-set-options r1-join dynamic-routing-redistribute=lb \ > + -- \ > + ls-add ls-join \ > + -- \ > + lsp-add ls-join lsp-join-to-r1 \ > + -- \ > + lsp-set-type lsp-join-to-r1 router \ > + -- \ > + lsp-set-options lsp-join-to-r1 router-port=r1-join \ > + -- \ > + lsp-set-addresses lsp-join-to-r1 router \ > + -- \ > + lr-add R2 \ > + -- \ > + lrp-add R2 r2-join 00:00:00:00:fe:02 \ > + -- \ > + lsp-add ls-join lsp-join-to-r2 \ > + -- \ > + lsp-set-type lsp-join-to-r2 router \ > + -- \ > + lsp-set-options lsp-join-to-r2 router-port=r2-join \ > + -- \ > + lrp-set-gateway-chassis r2-join hv1 > + > +# Create a load balancer and associate to R2 > +check ovn-nbctl lb-add lb2 10.42.10.10:80 192.168.123.10:80 > +check ovn-nbctl lr-lb-add R2 lb2 > + > +check ovn-nbctl --wait=hv sync > + > +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) > + > +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > [dnl > +blackhole 10.42.10.10 proto 84 metric 100 > +blackhole 172.16.1.150 proto 84 metric 1000]) > + > +# Move DGW of R2 to another chassis to verify that route priority will > decrease > +check ovn-nbctl --wait=hv \ > + -- \ > + lrp-del-gateway-chassis r2-join hv1 > + -- \ > + lrp-set-gateway-chassis r2-join hv123 > + > +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > [dnl > +blackhole 10.42.10.10 proto 84 metric 1000 > +blackhole 172.16.1.150 proto 84 metric 1000]) > + > +OVS_APP_EXIT_AND_WAIT([ovn-controller]) > + > +# Ensure system resources are cleaned up > +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) > +AT_CHECK([test `ip route show table $vrf | wc -l` -eq 1], [1]) > + > +as ovn-sb > +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > + > +as ovn-nb > +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > + > +as northd > +OVS_APP_EXIT_AND_WAIT([ovn-northd]) > + > +as > +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d > +/Failed to acquire.*/d > +/connection dropped.*/d"]) > +AT_CLEANUP > +]) > + > +OVN_FOR_EACH_NORTHD([ > +AT_SETUP([dynamic-routing - LB VIPs on gateway router IPv6]) > +AT_KEYWORDS([dynamic-routing]) > + > +CHECK_VRF() > +CHECK_CONNTRACK() > +CHECK_CONNTRACK_NAT() > + > +vrf=1001 > +VRF_RESERVE([$vrf]) > +ovn_start > +OVS_TRAFFIC_VSWITCHD_START() > +ADD_BR([br-int]) > +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone]) > + > +# Set external-ids in br-int needed for ovn-controller > +ovs-vsctl \ > + -- set Open_vSwitch . external-ids:system-id=hv1 \ > + -- set Open_vSwitch . > external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ > + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ > + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ > + -- set bridge br-int fail-mode=secure > other-config:disable-in-band=true > + > +# Start ovn-controller > +start_daemon ovn-controller > + > +ovn-appctl vlog/set route_exchange > +check ovn-nbctl -- lr-add R1 \ > + -- set Logical_Router R1 options:requested-tnl-key=$vrf > options:dynamic-routing=true > + > +check ovn-nbctl ls-add sw0 > +check ovn-nbctl ls-add public > + > +check ovn-nbctl --wait=hv sync > + > +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) > + > +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 2001:db8:100::1/64 > +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 > 2001:db8:1001::1/64 \ > + -- lrp-set-options rp-public \ > + dynamic-routing-maintain-vrf=true \ > + dynamic-routing-redistribute=lb > + > +check ovn-nbctl set logical_router R1 options:chassis=hv1 > + > +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \ > + type=router options:router-port=rp-sw0 \ > + -- lsp-set-addresses sw0-rp router > + > +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port > public-rp \ > + type=router options:router-port=rp-public \ > + -- lsp-set-addresses public-rp router > + > +check ovs-vsctl set Open_vSwitch . > external-ids:ovn-bridge-mappings=phynet:br-ext > + > +check ovn-nbctl lsp-add public public1 \ > + -- lsp-set-addresses public1 unknown \ > + -- lsp-set-type public1 localnet \ > + -- lsp-set-options public1 network_name=phynet > + > +check ovn-nbctl --wait=hv sync > + > +AT_CHECK([test `ip -6 route show table $vrf | wc -l` -eq 1], [1]) > + > +# Create a load balancer and associate to R1 > +check ovn-nbctl lb-add lb1 [[2001:db8:1001::150]]:80 > [[2001:db8:2001::100]]:80 > +check ovn-nbctl lr-lb-add R1 lb1 > + > +check ovn-nbctl --wait=hv sync > + > +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP]) > +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1]) > + > +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > [dnl > +blackhole 2001:db8:1001::150 dev lo proto 84 metric 1000 pref medium]) > + > +# Add distributed router connected through "join" LS and ensure > +# that its LB VIPs are redistributed by R1. LS join has no > +# IP config, routers reach each other over IPv6 LLAs > +check ovn-nbctl --wait=sb \ > + -- \ > + remove Logical_Router_Port rp-public options > dynamic-routing-redistribute \ > + -- \ > + lrp-add R1 r1-join 00:00:00:00:ff:02 \ > + -- \ > + lrp-set-options r1-join dynamic-routing-redistribute=lb \ > + -- \ > + ls-add ls-join \ > + -- \ > + lsp-add ls-join lsp-join-to-r1 \ > + -- \ > + lsp-set-type lsp-join-to-r1 router \ > + -- \ > + lsp-set-options lsp-join-to-r1 router-port=r1-join \ > + -- \ > + lsp-set-addresses lsp-join-to-r1 router \ > + -- \ > + lr-add R2 \ > + -- \ > + lrp-add R2 r2-join 00:00:00:00:fe:02 \ > + -- \ > + lsp-add ls-join lsp-join-to-r2 \ > + -- \ > + lsp-set-type lsp-join-to-r2 router \ > + -- \ > + lsp-set-options lsp-join-to-r2 router-port=r2-join \ > + -- \ > + lrp-set-gateway-chassis r2-join hv1 > + > +# Create a load balancer and associate to R2 > +check ovn-nbctl lb-add lb2 [[2001:db8:3001::150]]:80 > [[2001:db8:4001::100]]:80 > +check ovn-nbctl lr-lb-add R2 lb2 > + > +check ovn-nbctl --wait=hv sync > + > +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) > + > +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > [dnl > +blackhole 2001:db8:1001::150 dev lo proto 84 metric 1000 pref medium > +blackhole 2001:db8:3001::150 dev lo proto 84 metric 100 pref medium]) > + > +# Move DGW of R2 to another chassis to verify that route priority will > decrease > +check ovn-nbctl --wait=hv \ > + -- \ > + lrp-del-gateway-chassis r2-join hv1 > + -- \ > + lrp-set-gateway-chassis r2-join hv123 > + > +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > [dnl > +blackhole 2001:db8:1001::150 dev lo proto 84 metric 1000 pref medium > +blackhole 2001:db8:3001::150 dev lo proto 84 metric 1000 pref medium]) > + > +OVS_APP_EXIT_AND_WAIT([ovn-controller]) > + > +# Ensure system resources are cleaned up > +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP], [1]) > +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1], [1]) > + > +as ovn-sb > +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > + > +as ovn-nb > +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > + > +as northd > +OVS_APP_EXIT_AND_WAIT([ovn-northd]) > + > +as > +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d > +/Failed to acquire.*/d > +/connection dropped.*/d"]) > +AT_CLEANUP > +]) > + > +OVN_FOR_EACH_NORTHD([ > +AT_SETUP([dynamic-routing - DNAT and DNAT_AND_SNAT on gateway router > IPv4]) > +AT_KEYWORDS([dynamic-routing]) > + > +CHECK_VRF() > +CHECK_CONNTRACK() > +CHECK_CONNTRACK_NAT() > + > +vrf=1002 > +VRF_RESERVE([$vrf]) > +ovn_start > +OVS_TRAFFIC_VSWITCHD_START() > +ADD_BR([br-int]) > +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone]) > + > +# Set external-ids in br-int needed for ovn-controller > +ovs-vsctl \ > + -- set Open_vSwitch . external-ids:system-id=hv1 \ > + -- set Open_vSwitch . > external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ > + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ > + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ > + -- set bridge br-int fail-mode=secure > other-config:disable-in-band=true > + > +# Start ovn-controller > +start_daemon ovn-controller > + > +ovn-appctl vlog/set route_exchange > +check ovn-nbctl -- lr-add R1 \ > + -- set Logical_Router R1 options:requested-tnl-key=$vrf > options:dynamic-routing=true > + > +check ovn-nbctl ls-add sw0 > +check ovn-nbctl ls-add public > + > +check ovn-nbctl --wait=hv sync > + > +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) > + > +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 192.168.1.1/24 > +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 172.16.1.1/24 \ > + -- lrp-set-options rp-public \ > + dynamic-routing-maintain-vrf=true \ > + dynamic-routing-redistribute=nat > + > +check ovn-nbctl set logical_router R1 options:chassis=hv1 > + > +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \ > + type=router options:router-port=rp-sw0 \ > + -- lsp-set-addresses sw0-rp router > + > +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port > public-rp \ > + type=router options:router-port=rp-public \ > + -- lsp-set-addresses public-rp router > + > +check ovs-vsctl set Open_vSwitch . > external-ids:ovn-bridge-mappings=phynet:br-ext > + > +check ovn-nbctl lsp-add public public1 \ > + -- lsp-set-addresses public1 unknown \ > + -- lsp-set-type public1 localnet \ > + -- lsp-set-options public1 network_name=phynet > + > +check ovn-nbctl --wait=hv sync > + > +AT_CHECK([test `ip route show table $vrf | wc -l` -eq 2], [1]) > + > +# Create dnat_and_snat, dnat rules in R1 > +check ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.10 192.168.1.10 > +check ovn-nbctl lr-nat-add R1 dnat 172.16.1.11 192.168.1.11 > + > +check ovn-nbctl --wait=hv sync > + > +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) > + > +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > [dnl > +blackhole 172.16.1.10 proto 84 metric 1000 > +blackhole 172.16.1.11 proto 84 metric 1000]) > + > +# Add distributed router connected through "join" LS and ensure > +# that its external NAT IPs are redistributed by R1. LS join has no > +# IP config, routers reach each other over IPv6 LLAs > +check ovn-nbctl --wait=sb \ > + -- \ > + remove Logical_Router_Port rp-public options > dynamic-routing-redistribute \ > + -- \ > + lrp-add R1 r1-join 00:00:00:00:ff:02 \ > + -- \ > + lrp-set-options r1-join dynamic-routing-redistribute=nat \ > + -- \ > + ls-add ls-join \ > + -- \ > + lsp-add ls-join lsp-join-to-r1 \ > + -- \ > + lsp-set-type lsp-join-to-r1 router \ > + -- \ > + lsp-set-options lsp-join-to-r1 router-port=r1-join \ > + -- \ > + lsp-set-addresses lsp-join-to-r1 router \ > + -- \ > + lr-add R2 \ > + -- \ > + lrp-add R2 r2-join 00:00:00:00:fe:02 \ > + -- \ > + lsp-add ls-join lsp-join-to-r2 \ > + -- \ > + lsp-set-type lsp-join-to-r2 router \ > + -- \ > + lsp-set-options lsp-join-to-r2 router-port=r2-join \ > + -- \ > + ha-chassis-group-add g1 \ > + -- \ > + ha-chassis-group-add-chassis g1 hv1 10 > + > +group_uuid=$(ovn-nbctl get ha_chassis_group g1 _uuid) > +check ovn-nbctl set logical_router_port r2-join > ha_chassis_group="$group_uuid" > + > +# Create NAT on R2 > +check ovn-nbctl lr-nat-add R2 dnat_and_snat 10.42.10.10 192.168.1.10 > +check ovn-nbctl lr-nat-add R2 dnat_and_snat 10.42.10.11 192.168.1.11 > + > +check ovn-nbctl --wait=hv sync > + > +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) > + > +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > [dnl > +blackhole 10.42.10.10 proto 84 metric 100 > +blackhole 10.42.10.11 proto 84 metric 100 > +blackhole 172.16.1.10 proto 84 metric 1000 > +blackhole 172.16.1.11 proto 84 metric 1000]) > + > +# Move DGW of R2 to another chassis to verify that route priority will > decrease > +check ovn-nbctl --wait=hv \ > + -- \ > + ha-chassis-group-add-chassis g1 hv2 20 \ > + -- \ > + ha-chassis-group-remove-chassis g1 hv1 > + > +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > [dnl > +blackhole 10.42.10.10 proto 84 metric 1000 > +blackhole 10.42.10.11 proto 84 metric 1000 > +blackhole 172.16.1.10 proto 84 metric 1000 > +blackhole 172.16.1.11 proto 84 metric 1000]) > + > +# Add "guest" LS connected the distributed router R2 and one "VM" called > +# guest1. > +# Also, connect R2 to ls-join via nother DGW > +check ovn-nbctl --wait=sb \ > + -- \ > + lrp-add R2 r2-guest 00:00:00:00:fd:02 192.168.2.1/24 \ > + -- \ > + lrp-add R2 r2-join-dgw2 00:00:00:00:fd:02 \ > + -- \ > + lsp-add ls-join lsp-join-to-r2-dgw2 \ > + -- \ > + lsp-set-type lsp-join-to-r2-dgw2 router \ > + -- \ > + lsp-set-options lsp-join-to-r2-dgw2 router-port=r2-join-dgw2 \ > + -- \ > + ls-add ls-guest \ > + -- \ > + lsp-add ls-guest lsp-guest-to-r2 \ > + -- \ > + lsp-set-type lsp-guest-to-r2 router \ > + -- \ > + lsp-set-options lsp-guest-to-r2 router-port=r2-guest \ > + -- \ > + lsp-set-addresses lsp-guest-to-r2 router \ > + -- \ > + lsp-add ls-guest guest1 \ > + -- \ > + lsp-set-addresses guest1 '00:00:00:00:fc:03 192.168.2.10' > + > +check ovn-nbctl set logical_router_port r2-join-dgw2 \ > + ha_chassis_group="$group_uuid" > +# Bind guest1 on hv1 > +ADD_NAMESPACES(guest1) > +ADD_VETH(guest1, guest1, br-int, "192.168.2.10/24", "00:00:00:00:fd:03", > \ > + "192.168.2.1") > + > +# Add distributed NAT on R2 for guest1 and ensure that it's advertised > +# with better metric on chassis hv1 even when DGP for R2 is elsewhere. > +check ovn-nbctl --gateway-port \ > + r2-join-dgw2 --add-route lr-nat-add R2 dnat_and_snat 10.42.20.11 \ > + 192.168.2.10 guest1 00:00:00:00:fc:04 > +check ovn-nbctl --wait=hv sync > + > +ovn-sbctl list advertised_route > +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > [dnl > +blackhole 10.42.10.10 proto 84 metric 1000 > +blackhole 10.42.10.11 proto 84 metric 1000 > +blackhole 10.42.20.11 proto 84 metric 100 > +blackhole 172.16.1.10 proto 84 metric 1000 > +blackhole 172.16.1.11 proto 84 metric 1000]) > + > +OVS_APP_EXIT_AND_WAIT([ovn-controller]) > + > +# Ensure system resources are cleaned up > +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) > +AT_CHECK([test `ip route show table $vrf | wc -l` -eq 1], [1]) > + > +as ovn-sb > +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > + > +as ovn-nb > +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > + > +as northd > +OVS_APP_EXIT_AND_WAIT([ovn-northd]) > + > +as > +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d > +/Failed to acquire.*/d > +/connection dropped.*/d"]) > +AT_CLEANUP > +]) > + > +OVN_FOR_EACH_NORTHD([ > +AT_SETUP([dynamic-routing - DNAT and DNAT_AND_SNAT on gateway router > IPv6]) > +AT_KEYWORDS([dynamic-routing]) > + > +CHECK_VRF() > +CHECK_CONNTRACK() > +CHECK_CONNTRACK_NAT() > + > +vrf=1003 > +VRF_RESERVE([$vrf]) > +ovn_start > +OVS_TRAFFIC_VSWITCHD_START() > +ADD_BR([br-int]) > +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone]) > + > +# Set external-ids in br-int needed for ovn-controller > +ovs-vsctl \ > + -- set Open_vSwitch . external-ids:system-id=hv1 \ > + -- set Open_vSwitch . > external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ > + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ > + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ > + -- set bridge br-int fail-mode=secure > other-config:disable-in-band=true > + > +# Start ovn-controller > +start_daemon ovn-controller > + > +ovn-appctl vlog/set route_exchange > +check ovn-nbctl -- lr-add R1 \ > + -- set Logical_Router R1 options:requested-tnl-key=$vrf > options:dynamic-routing=true > + > +check ovn-nbctl ls-add sw0 > +check ovn-nbctl ls-add public > + > +check ovn-nbctl --wait=hv sync > + > +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) > + > +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 2001:db8:100::1/64 > +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 > 2001:db8:1003::1/64 \ > + -- lrp-set-options rp-public \ > + dynamic-routing-maintain-vrf=true \ > + dynamic-routing-redistribute=nat > + > +check ovn-nbctl set logical_router R1 options:chassis=hv1 > + > +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \ > + type=router options:router-port=rp-sw0 \ > + -- lsp-set-addresses sw0-rp router > + > +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port > public-rp \ > + type=router options:router-port=rp-public \ > + -- lsp-set-addresses public-rp router > + > +check ovs-vsctl set Open_vSwitch . > external-ids:ovn-bridge-mappings=phynet:br-ext > + > +check ovn-nbctl lsp-add public public1 \ > + -- lsp-set-addresses public1 unknown \ > + -- lsp-set-type public1 localnet \ > + -- lsp-set-options public1 network_name=phynet > + > +check ovn-nbctl --wait=hv sync > + > +AT_CHECK([test `ip -6 route show table $vrf | wc -l` -eq 2], [1]) > + > +# Create dnat_and_snat, dnat rules in R1 > +check ovn-nbctl lr-nat-add R1 \ > + dnat_and_snat 2001:db8:1003::150 2001:db8:100::100 > +check ovn-nbctl lr-nat-add R1 \ > + dnat 2001:db8:1003::151 2001:db8:100::100 > + > +check ovn-nbctl --wait=hv sync > + > +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) > +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > [dnl > +blackhole 2001:db8:1003::150 dev lo proto 84 metric 1000 pref medium > +blackhole 2001:db8:1003::151 dev lo proto 84 metric 1000 pref medium]) > + > +# Add distributed router connected through "join" LS and ensure > +# that its external NAT IPs are redistributed by R1. LS join has no > +# IP config, routers reach each other over IPv6 LLAs > +check ovn-nbctl --wait=sb \ > + -- \ > + remove Logical_Router_Port rp-public options > dynamic-routing-redistribute \ > + -- \ > + lrp-add R1 r1-join 00:00:00:00:ff:02 \ > + -- \ > + lrp-set-options r1-join dynamic-routing-redistribute=nat \ > + -- \ > + ls-add ls-join \ > + -- \ > + lsp-add ls-join lsp-join-to-r1 \ > + -- \ > + lsp-set-type lsp-join-to-r1 router \ > + -- \ > + lsp-set-options lsp-join-to-r1 router-port=r1-join \ > + -- \ > + lsp-set-addresses lsp-join-to-r1 router \ > + -- \ > + lr-add R2 \ > + -- \ > + lrp-add R2 r2-join 00:00:00:00:fe:02 \ > + -- \ > + lsp-add ls-join lsp-join-to-r2 \ > + -- \ > + lsp-set-type lsp-join-to-r2 router \ > + -- \ > + lsp-set-options lsp-join-to-r2 router-port=r2-join \ > + -- \ > + ha-chassis-group-add g1 \ > + -- \ > + ha-chassis-group-add-chassis g1 hv1 10 > + > +group_uuid=$(ovn-nbctl get ha_chassis_group g1 _uuid) > +check ovn-nbctl set logical_router_port r2-join > ha_chassis_group="$group_uuid" > + > +# Create NAT on R2 > +check ovn-nbctl lr-nat-add R2 \ > + dnat_and_snat 2001:db8:1004::150 2001:db8:201::100 > +check ovn-nbctl lr-nat-add R2 \ > + dnat 2001:db8:1004::151 2001:db8:201::101 > + > +check ovn-nbctl --wait=hv sync > + > +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) > + > +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > [dnl > +blackhole 2001:db8:1003::150 dev lo proto 84 metric 1000 pref medium > +blackhole 2001:db8:1003::151 dev lo proto 84 metric 1000 pref medium > +blackhole 2001:db8:1004::150 dev lo proto 84 metric 100 pref medium > +blackhole 2001:db8:1004::151 dev lo proto 84 metric 100 pref medium]) > + > +# Move DGW of R2 to another chassis to verify that route priority will > decrease > +check ovn-nbctl --wait=hv \ > + -- \ > + ha-chassis-group-add-chassis g1 hv2 20 \ > + -- \ > + ha-chassis-group-remove-chassis g1 hv1 > + > +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > [dnl > +blackhole 2001:db8:1003::150 dev lo proto 84 metric 1000 pref medium > +blackhole 2001:db8:1003::151 dev lo proto 84 metric 1000 pref medium > +blackhole 2001:db8:1004::150 dev lo proto 84 metric 1000 pref medium > +blackhole 2001:db8:1004::151 dev lo proto 84 metric 1000 pref medium]) > + > +# Add "guest" LS connected the distributed router R2 and one "VM" called > +# guest1. > +# Also, connect R2 to ls-join via nother DGW > +check ovn-nbctl --wait=sb \ > + -- \ > + lrp-add R2 r2-guest 00:00:00:00:fd:02 2001:db9:2000::1/64 \ > + -- \ > + lrp-add R2 r2-join-dgw2 00:00:00:00:fd:02 \ > + -- \ > + lsp-add ls-join lsp-join-to-r2-dgw2 \ > + -- \ > + lsp-set-type lsp-join-to-r2-dgw2 router \ > + -- \ > + lsp-set-options lsp-join-to-r2-dgw2 router-port=r2-join-dgw2 \ > + -- \ > + ls-add ls-guest \ > + -- \ > + lsp-add ls-guest lsp-guest-to-r2 \ > + -- \ > + lsp-set-type lsp-guest-to-r2 router \ > + -- \ > + lsp-set-options lsp-guest-to-r2 router-port=r2-guest \ > + -- \ > + lsp-set-addresses lsp-guest-to-r2 router \ > + -- \ > + lsp-add ls-guest guest1 \ > + -- \ > + lsp-set-addresses guest1 '00:00:00:00:fc:03 2001:db9:2000::10' > + > +check ovn-nbctl set logical_router_port r2-join-dgw2 > ha_chassis_group="$group_uuid" > +# Bind guest1 on hv1 > +ADD_NAMESPACES(guest1) > +ADD_VETH(guest1, guest1, br-int, "2001:db9:2000::10/64", > "00:00:00:00:fd:03", \ > + "2001:db9:2000::1", "nodad") > + > +# Add distributed NAT on R2 for guest1 and ensure that it's advertised > +# with better metric on chassis hv1 even when DGP for R2 is elsewhere. > +check ovn-nbctl --gateway-port \ > + r2-join-dgw2 --add-route lr-nat-add R2 dnat_and_snat \ > + 2001:db8:1005::150 2001:db9:2000::10 guest1 00:00:00:00:fc:04 > +check ovn-nbctl --wait=hv sync > + > +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > [dnl > +blackhole 2001:db8:1003::150 dev lo proto 84 metric 1000 pref medium > +blackhole 2001:db8:1003::151 dev lo proto 84 metric 1000 pref medium > +blackhole 2001:db8:1004::150 dev lo proto 84 metric 1000 pref medium > +blackhole 2001:db8:1004::151 dev lo proto 84 metric 1000 pref medium > +blackhole 2001:db8:1005::150 dev lo proto 84 metric 100 pref medium]) > + > +OVS_APP_EXIT_AND_WAIT([ovn-controller]) > +# Ensure system resources are cleaned up > +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP], [1]) > +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2], [1]) > + > +as ovn-sb > +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > + > +as ovn-nb > +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > + > +as northd > +OVS_APP_EXIT_AND_WAIT([ovn-northd]) > + > +as > +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d > +/Failed to acquire.*/d > +/connection dropped.*/d"]) > +AT_CLEANUP > +]) > + > + > -- > 2.43.0 > > Thanks, Ales _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
