On Fri, Feb 14, 2025 at 11:14 AM Dumitru Ceara <[email protected]> wrote:
> On 2/14/25 9:57 AM, Ales Musil wrote: > > On Fri, Feb 14, 2025 at 7:34 AM Martin Kalcok < > [email protected]> > > wrote: > > > >> This change builds on top of the new "dynamic routing" OVN feature > >> that allows advertising routes to the fabric network. New accepted > >> values are added to "dynamic-routing-redistribute" option. > >> > >> * nat - When included in the option, ovn-controller will advertise > >> routes for external NAT IPs of the LR, as well as those > >> of the neighboring routers (directly connected or through > >> the shared LS). > >> * lb - When included in the option, ovn-controller will advertise > >> routes for LB VIPs of the LR, as well as those > >> of the neighboring routers (directly connected or through > >> the shared LS). > >> > >> Co-authored-by: Frode Nordahl <[email protected]> > >> Co-authored-by: Dumitru Ceara <[email protected]> > >> Signed-off-by: Frode Nordahl <[email protected]> > >> Signed-off-by: Dumitru Ceara <[email protected]> > >> Signed-off-by: Martin Kalcok <[email protected]> > >> --- > > > > > > Hi Martin, > > > > Hi Ales, > > Thanks for the review! > > > I have a couple of comments down below. > > > > TODO.rst | 10 + > >> lib/stopwatch-names.h | 1 + > >> northd/en-advertised-route-sync.c | 198 ++++++-- > >> northd/en-advertised-route-sync.h | 3 + > >> northd/en-learned-route-sync.c | 3 +- > >> northd/inc-proc-northd.c | 5 + > >> northd/northd.c | 254 +++++++++- > >> northd/northd.h | 34 +- > >> ovn-nb.xml | 36 ++ > >> tests/ovn-northd.at | 628 +++++++++++++++++++++++++ > >> tests/system-ovn.at | 738 ++++++++++++++++++++++++++++++ > >> 11 files changed, 1861 insertions(+), 49 deletions(-) > >> > >> diff --git a/TODO.rst b/TODO.rst > >> index d75db3152..593b9a580 100644 > >> --- a/TODO.rst > >> +++ b/TODO.rst > >> @@ -146,6 +146,9 @@ OVN To-do List > >> * Add incremental processing for northd when the Learned_Route table > >> changes. > >> Currently en_lflow is fully recomputed whenever such a change > happens. > >> > >> + * Add incremental processing of en_dynamic_routes for stateful > >> configuration > >> + changes. > >> + > >> * The ovn-controller currently loads all Advertised_Route entries on > >> startup. > >> This is to prevent deleting our routes on restart. If we defer > >> updating > >> routes until we are sure to have loaded all necessary > >> Advertised_Routes > >> @@ -156,3 +159,10 @@ OVN To-do List > >> monitoring conditions to update before we actually try to learn > >> routes. > >> Otherwise we could try to add duplicated Learned_Routes and the > ovnsb > >> commit would fail. > >> + > >> + * Consider splitting parsed_route structure. When creating parsed > routes > >> + with tracked_port explicitly set, other members of this structure > are > >> + usually unused/default. A new structure dedicated to routes with > >> + explicitly defined tracked_port would be more efficient. > >> + More details in > >> + > >> > https://mail.openvswitch.org/pipermail/ovs-dev/2025-February/420985.html > >> diff --git a/lib/stopwatch-names.h b/lib/stopwatch-names.h > >> index c12dd28d0..13aa5e7bf 100644 > >> --- a/lib/stopwatch-names.h > >> +++ b/lib/stopwatch-names.h > >> @@ -36,5 +36,6 @@ > >> #define LS_STATEFUL_RUN_STOPWATCH_NAME "ls_stateful" > >> #define ADVERTISED_ROUTE_SYNC_RUN_STOPWATCH_NAME > "advertised_route_sync" > >> #define LEARNED_ROUTE_SYNC_RUN_STOPWATCH_NAME "learned_route_sync" > >> +#define DYNAMIC_ROUTES_RUN_STOPWATCH_NAME "dynamic_routes" > > This one is not used and should be. > > >> > >> #endif > >> diff --git a/northd/en-advertised-route-sync.c > >> b/northd/en-advertised-route-sync.c > >> index cfea7c39b..a984e70a8 100644 > >> --- a/northd/en-advertised-route-sync.c > >> +++ b/northd/en-advertised-route-sync.c > >> @@ -30,7 +30,8 @@ advertised_route_table_sync( > >> struct ovsdb_idl_txn *ovnsb_txn, > >> const struct sbrec_advertised_route_table > >> *sbrec_advertised_route_table, > >> const struct lr_stateful_table *lr_stateful_table, > >> - const struct hmap *parsed_routes, > >> + const struct hmap *routes, > >> + const struct hmap *dynamic_routes, > >> struct advertised_route_sync_data *data); > >> > >> bool > >> @@ -141,6 +142,8 @@ en_advertised_route_sync_run(struct engine_node > *node, > >> void *data OVS_UNUSED) > >> struct advertised_route_sync_data *routes_sync_data = data; > >> struct routes_data *routes_data > >> = engine_get_input_data("routes", node); > >> + struct dynamic_routes_data *dynamic_routes_data > >> + = engine_get_input_data("dynamic_routes", node); > >> const struct engine_context *eng_ctx = engine_get_context(); > >> const struct sbrec_advertised_route_table > >> *sbrec_advertised_route_table = > >> EN_OVSDB_GET(engine_get_input("SB_advertised_route", node)); > >> @@ -153,12 +156,75 @@ en_advertised_route_sync_run(struct engine_node > >> *node, void *data OVS_UNUSED) > >> sbrec_advertised_route_table, > >> &lr_stateful_data->table, > >> &routes_data->parsed_routes, > >> + &dynamic_routes_data->parsed_routes, > >> routes_sync_data); > >> > >> stopwatch_stop(ADVERTISED_ROUTE_SYNC_RUN_STOPWATCH_NAME, > time_msec()); > >> engine_set_node_state(node, EN_UPDATED); > >> } > >> > >> +void > >> +*en_dynamic_routes_init(struct engine_node *node OVS_UNUSED, > > Nit: * should be on the previous line. > > >> + struct engine_arg *arg OVS_UNUSED) > > Nit: indentation. > > >> +{ > >> + struct dynamic_routes_data *data = xmalloc(sizeof *data); > >> + *data = (struct dynamic_routes_data) { > >> + .parsed_routes = HMAP_INITIALIZER(&data->parsed_routes), > >> + }; > >> + > >> + return data; > >> +} > >> + > >> +static void > >> +en_dynamic_routes_clean(struct dynamic_routes_data *data) > >> +{ > >> + struct parsed_route *r; > >> + HMAP_FOR_EACH_POP (r, key_node, &data->parsed_routes) { > >> + parsed_route_free(r); > >> + } > >> +} > >> +void > >> +en_dynamic_routes_cleanup(void *data_) > >> +{ > >> + struct dynamic_routes_data *data = data_; > >> + > >> + en_dynamic_routes_clean(data); > >> + hmap_destroy(&data->parsed_routes); > >> +} > >> + > >> +void > >> +en_dynamic_routes_run(struct engine_node *node, void *data) > >> +{ > >> + struct dynamic_routes_data *dynamic_routes_data = data; > >> + struct northd_data *northd_data = engine_get_input_data("northd", > >> node); > >> + struct ed_type_lr_stateful *lr_stateful_data = > >> + engine_get_input_data("lr_stateful", node); > >> + > >> + en_dynamic_routes_clean(data); > >> + const struct lr_stateful_record *lr_stateful_rec; > >> + HMAP_FOR_EACH (lr_stateful_rec, key_node, > >> + &lr_stateful_data->table.entries) { > >> + const struct ovn_datapath *od = > >> + ovn_datapaths_find_by_index(&northd_data->lr_datapaths, > >> + lr_stateful_rec->lr_index); > >> + if (!od->dynamic_routing) { > >> + continue; > >> + } > >> + build_nat_parsed_routes(od, lr_stateful_rec->lrnat_rec, > >> + &northd_data->ls_ports, > >> + &dynamic_routes_data->parsed_routes); > >> + build_nat_connected_parsed_routes(od, &lr_stateful_data->table, > >> + &northd_data->ls_ports, > >> + > >> &dynamic_routes_data->parsed_routes); > >> + > >> + build_lb_parsed_routes(od, lr_stateful_rec->lb_ips, > >> + &dynamic_routes_data->parsed_routes); > >> + build_lb_connected_parsed_routes(od, &lr_stateful_data->table, > >> + > >> &dynamic_routes_data->parsed_routes); > >> + } > >> + engine_set_node_state(node, EN_UPDATED); > >> +} > >> + > >> struct ar_entry { > >> struct hmap_node hmap_node; > >> > >> @@ -333,12 +399,89 @@ publish_host_routes(struct hmap *sync_routes, > >> } > >> } > >> > >> +static void > >> +advertised_route_table_sync_route_add( > >> + const struct lr_stateful_table *lr_stateful_table, > >> + struct advertised_route_sync_data *data, > >> + struct uuidset *host_route_lrps, > >> + struct hmap *sync_routes, > >> + const struct parsed_route *route) > >> +{ > >> + if (route->is_discard_route) { > >> + return; > >> + } > >> + if (prefix_is_link_local(&route->prefix, route->plen)) { > >> + return; > >> + } > >> + if (!route->od->dynamic_routing) { > >> + return; > >> + } > >> + > >> + enum dynamic_routing_redistribute_mode drr = > >> + route->out_port->dynamic_routing_redistribute; > >> + if (route->source == ROUTE_SOURCE_CONNECTED) { > >> + if (!drr_mode_CONNECTED_is_set(drr)) { > >> + return; > >> + } > >> + /* If we advertise host routes, we only need to do so once per > >> + * LRP. */ > >> + const struct uuid *lrp_uuid = > >> &route->out_port->nbrp->header_.uuid; > >> + if (drr_mode_CONNECTED_AS_HOST_is_set(drr) && > >> + !uuidset_contains(host_route_lrps, lrp_uuid)) { > >> + uuidset_insert(host_route_lrps, lrp_uuid); > >> + publish_host_routes(sync_routes, lr_stateful_table, route, > >> data); > >> + return; > >> + } > >> + } > >> + if (route->source == ROUTE_SOURCE_STATIC && > >> !drr_mode_STATIC_is_set(drr)) { > >> + return; > >> + } > >> + if (route->source == ROUTE_SOURCE_NAT) { > >> + if (!drr_mode_NAT_is_set(drr)) { > >> + return; > >> + } > >> + /* If NAT route tracks port on a different DP than the one that > >> + * advertises the route, we need to watch for changes on that > DP > >> as > >> + * well. */ > >> + if (route->tracked_port && route->tracked_port->od != > route->od) { > >> + if (route->tracked_port->od->nbr) { > >> + uuidset_insert(&data->nb_lr, > >> + > >> &route->tracked_port->od->nbr->header_.uuid); > >> + } else if (route->tracked_port->od->nbs) { > >> + uuidset_insert(&data->nb_ls, > >> + > >> &route->tracked_port->od->nbs->header_.uuid); > >> + } > >> + } > >> + } > >> + if (route->source == ROUTE_SOURCE_LB) { > >> + if (!drr_mode_LB_is_set(drr)) { > >> + return; > >> + } > >> + /* If LB route tracks port on a different DP than the one that > >> + * advertises the route, we need to watch for changes on that > DP > >> as > >> + * well. */ > >> + if (route->tracked_port && route->tracked_port->od != > route->od) { > >> + uuidset_insert(&data->nb_lr, > >> + > &route->tracked_port->od->nbr->header_.uuid); > >> + } > >> + } > >> + > >> + char *ip_prefix = normalize_v46_prefix(&route->prefix, > route->plen); > >> + const struct sbrec_port_binding *tracked_port = NULL; > >> + if (route->tracked_port) { > >> + tracked_port = route->tracked_port->sb; > >> + } > >> + ar_add_entry(sync_routes, route->od->sb, route->out_port->sb, > >> + ip_prefix, tracked_port); > >> +} > >> + > >> static void > >> advertised_route_table_sync( > >> struct ovsdb_idl_txn *ovnsb_txn, > >> const struct sbrec_advertised_route_table > >> *sbrec_advertised_route_table, > >> const struct lr_stateful_table *lr_stateful_table, > >> - const struct hmap *parsed_routes, > >> + const struct hmap *routes, > >> + const struct hmap *dynamic_routes, > >> struct advertised_route_sync_data *data) > >> { > >> struct hmap sync_routes = HMAP_INITIALIZER(&sync_routes); > >> @@ -346,47 +489,24 @@ advertised_route_table_sync( > >> const struct parsed_route *route; > >> > >> struct ar_entry *route_e; > >> - const struct sbrec_advertised_route *sb_route; > >> - HMAP_FOR_EACH (route, key_node, parsed_routes) { > >> - if (route->is_discard_route) { > >> - continue; > >> - } > >> - if (prefix_is_link_local(&route->prefix, route->plen)) { > >> - continue; > >> - } > >> - if (!route->od->dynamic_routing) { > >> - continue; > >> - } > >> > >> - enum dynamic_routing_redistribute_mode drr = > >> - route->out_port->dynamic_routing_redistribute; > >> - if (route->source == ROUTE_SOURCE_CONNECTED) { > >> - if (!drr_mode_CONNECTED_is_set(drr)) { > >> - continue; > >> - } > >> - /* If we advertise host routes, we only need to do so once > per > >> - * LRP. */ > >> - const struct uuid *lrp_uuid = > >> - &route->out_port->nbrp->header_.uuid; > >> - if (drr_mode_CONNECTED_AS_HOST_is_set(drr) && > >> - !uuidset_contains(&host_route_lrps, lrp_uuid)) { > >> - uuidset_insert(&host_route_lrps, lrp_uuid); > >> - publish_host_routes(&sync_routes, lr_stateful_table, > >> - route, data); > >> - continue; > >> - } > >> - } > >> - if (route->source == ROUTE_SOURCE_STATIC && > >> - !drr_mode_STATIC_is_set(drr)) { > >> - continue; > >> - } > >> - > >> - char *ip_prefix = normalize_v46_prefix(&route->prefix, > >> route->plen); > >> - route_e = ar_add_entry(&sync_routes, route->od->sb, > >> - route->out_port->sb, ip_prefix, NULL); > >> + /* First build the set of non-dynamic routes that need sync-ing. */ > >> + HMAP_FOR_EACH (route, key_node, routes) { > >> + advertised_route_table_sync_route_add(lr_stateful_table, > >> + data, &host_route_lrps, > >> + &sync_routes, > >> + route); > >> + } > >> + /* Then add the set of dynamic routes that need sync-ing. */ > >> + HMAP_FOR_EACH (route, key_node, dynamic_routes) { > >> + advertised_route_table_sync_route_add(lr_stateful_table, > >> + data, &host_route_lrps, > >> + &sync_routes, > >> + route); > >> } > >> uuidset_destroy(&host_route_lrps); > >> > >> + const struct sbrec_advertised_route *sb_route; > >> SBREC_ADVERTISED_ROUTE_TABLE_FOR_EACH_SAFE (sb_route, > >> > >> sbrec_advertised_route_table) { > >> route_e = ar_find(&sync_routes, sb_route->datapath, > >> diff --git a/northd/en-advertised-route-sync.h > >> b/northd/en-advertised-route-sync.h > >> index 1f24fd329..0290f08d2 100644 > >> --- a/northd/en-advertised-route-sync.h > >> +++ b/northd/en-advertised-route-sync.h > >> @@ -36,4 +36,7 @@ void *en_advertised_route_sync_init(struct engine_node > >> *, struct engine_arg *); > >> void en_advertised_route_sync_cleanup(void *data); > >> void en_advertised_route_sync_run(struct engine_node *, void *data); > >> > >> +void *en_dynamic_routes_init(struct engine_node *, struct engine_arg > *); > >> +void en_dynamic_routes_cleanup(void *data); > >> +void en_dynamic_routes_run(struct engine_node *, void *data); > >> #endif /* EN_ADVERTISED_ROUTE_SYNC_H */ > >> diff --git a/northd/en-learned-route-sync.c > >> b/northd/en-learned-route-sync.c > >> index 406f1551f..4e87b3265 100644 > >> --- a/northd/en-learned-route-sync.c > >> +++ b/northd/en-learned-route-sync.c > >> @@ -181,7 +181,8 @@ parse_route_from_sbrec_route(struct hmap > >> *parsed_routes_out, > >> > >> parsed_route_add(od, nexthop, &prefix, plen, false, lrp_addr_s, > >> out_port, 0, false, false, NULL, > >> - ROUTE_SOURCE_LEARNED, &route->header_, > >> parsed_routes_out); > >> + ROUTE_SOURCE_LEARNED, &route->header_, NULL, > >> + parsed_routes_out); > >> } > >> > >> static void > >> diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c > >> index 438daf1c6..e94e98f78 100644 > >> --- a/northd/inc-proc-northd.c > >> +++ b/northd/inc-proc-northd.c > >> @@ -175,6 +175,7 @@ static ENGINE_NODE(multicast_igmp, > "multicast_igmp"); > >> static ENGINE_NODE(acl_id, "acl_id"); > >> static ENGINE_NODE(advertised_route_sync, "advertised_route_sync"); > >> static ENGINE_NODE(learned_route_sync, "learned_route_sync"); > >> +static ENGINE_NODE(dynamic_routes, "dynamic_routes"); > >> > >> void inc_proc_northd_init(struct ovsdb_idl_loop *nb, > >> struct ovsdb_idl_loop *sb) > >> @@ -289,7 +290,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop > *nb, > >> engine_add_input(&en_ecmp_nexthop, &en_sb_mac_binding, > >> ecmp_nexthop_mac_binding_handler); > >> > >> + engine_add_input(&en_dynamic_routes, &en_lr_stateful, NULL); > >> > > + engine_add_input(&en_dynamic_routes, &en_northd, > engine_noop_handler); > >> > > > > We usually add a comment explaining why noop is fine. > > > > Good point. I'm thinking of: > > /* No need for an explicit handler for northd changes. Stateful > * configuration changes are passed through the en_lr_stateful input > * dependency. We do need to access en_northd (input) data, i.e., to > * lookup OVN ports. */ > > > > >> + > >> engine_add_input(&en_advertised_route_sync, &en_routes, NULL); > >> + engine_add_input(&en_advertised_route_sync, &en_dynamic_routes, > NULL); > >> engine_add_input(&en_advertised_route_sync, > &en_sb_advertised_route, > >> NULL); > >> engine_add_input(&en_advertised_route_sync, &en_lr_stateful, > >> diff --git a/northd/northd.c b/northd/northd.c > >> index 1d9f0675e..1558654a0 100644 > >> --- a/northd/northd.c > >> +++ b/northd/northd.c > >> @@ -848,6 +848,14 @@ parse_dynamic_routing_redistribute( > >> out |= DRRM_STATIC; > >> continue; > >> } > >> + if (!strcmp(token, "nat")) { > >> + out |= DRRM_NAT; > >> + continue; > >> + } > >> + if (!strcmp(token, "lb")) { > >> + out |= DRRM_LB; > >> + continue; > >> + } > >> static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); > >> VLOG_WARN_RL(&rl, > >> "unkown dynamic-routing-redistribute option '%s' > on > >> %s", > >> @@ -11000,6 +11008,7 @@ parsed_route_init(const struct ovn_datapath *od, > >> bool ecmp_symmetric_reply, > >> const struct sset *ecmp_selection_fields, > >> enum route_source source, > >> + const struct ovn_port *tracked_port, > >> const struct ovsdb_idl_row *source_hint) > >> { > >> > >> @@ -11015,6 +11024,7 @@ parsed_route_init(const struct ovn_datapath *od, > >> new_pr->is_discard_route = is_discard_route; > >> new_pr->lrp_addr_s = nullable_xstrdup(lrp_addr_s); > >> new_pr->out_port = out_port; > >> + new_pr->tracked_port = tracked_port; > >> new_pr->source = source; > >> if (ecmp_selection_fields) { > >> sset_clone(&new_pr->ecmp_selection_fields, > ecmp_selection_fields); > >> @@ -11040,7 +11050,7 @@ parsed_route_clone(const struct parsed_route > *pr) > >> pr->od, nexthop, pr->prefix, pr->plen, pr->is_discard_route, > >> pr->lrp_addr_s, pr->out_port, pr->route_table_id, > >> pr->is_src_route, > >> pr->ecmp_symmetric_reply, &pr->ecmp_selection_fields, > pr->source, > >> - pr->source_hint); > >> + pr->tracked_port, pr->source_hint); > >> > >> new_pr->hash = pr->hash; > >> return new_pr; > >> @@ -11083,13 +11093,14 @@ parsed_route_add(const struct ovn_datapath > *od, > >> const struct sset *ecmp_selection_fields, > >> enum route_source source, > >> const struct ovsdb_idl_row *source_hint, > >> + const struct ovn_port *tracked_port, > >> struct hmap *routes) > >> { > >> > >> struct parsed_route *new_pr = parsed_route_init( > >> od, nexthop, *prefix, plen, is_discard_route, lrp_addr_s, > >> out_port, > >> route_table_id, is_src_route, ecmp_symmetric_reply, > >> - ecmp_selection_fields, source, source_hint); > >> + ecmp_selection_fields, source, tracked_port, source_hint); > >> > >> new_pr->hash = route_hash(new_pr); > >> > >> @@ -11226,7 +11237,7 @@ parsed_routes_add_static(const struct > ovn_datapath > >> *od, > >> parsed_route_add(od, nexthop, &prefix, plen, is_discard_route, > >> lrp_addr_s, > >> out_port, route_table_id, is_src_route, > >> ecmp_symmetric_reply, &ecmp_selection_fields, > source, > >> - &route->header_, routes); > >> + &route->header_, NULL, routes); > >> sset_destroy(&ecmp_selection_fields); > >> } > >> > >> @@ -11244,7 +11255,7 @@ parsed_routes_add_connected(const struct > >> ovn_datapath *od, > >> false, addr->addr_s, op, > >> 0, false, > >> false, NULL, ROUTE_SOURCE_CONNECTED, > >> - &op->nbrp->header_, routes); > >> + &op->nbrp->header_, NULL, routes); > >> } > >> > >> for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { > >> @@ -11256,7 +11267,7 @@ parsed_routes_add_connected(const struct > >> ovn_datapath *od, > >> false, addr->addr_s, op, > >> 0, false, > >> false, NULL, ROUTE_SOURCE_CONNECTED, > >> - &op->nbrp->header_, routes); > >> + &op->nbrp->header_, NULL, routes); > >> } > >> } > >> > >> @@ -11294,6 +11305,237 @@ build_parsed_routes(const struct ovn_datapath > >> *od, const struct hmap *lr_ports, > >> } > >> } > >> > >> +/* This function adds new parsed route for each entry in lr_nat record > >> + * to "routes". Logical port of the route is set to "advertising_op" > and > >> + * tracked port is set to NAT's distributed gw port. If NAT doesn't > have > >> + * DGP (for example if it's set on gateway router), no tracked port > will > >> + * be set.*/ > >> +static void > >> +build_nat_parsed_route_for_port(const struct ovn_port *advertising_op, > >> + const struct lr_nat_record *lr_nat, > >> + const struct hmap *ls_ports, > >> + struct hmap *routes) > >> +{ > >> + const struct ovn_datapath *advertising_od = advertising_op->od; > >> + > >> + for (size_t i = 0; i < lr_nat->n_nat_entries; i++) { > >> + const struct ovn_nat *nat = &lr_nat->nat_entries[i]; > >> + int plen = nat_entry_is_v6(nat) ? 128 : 32; > >> + struct in6_addr prefix; > >> + ip46_parse(nat->nb->external_ip, &prefix); > >> + > >> + const struct ovn_port *tracked_port = > >> + nat->is_distributed && nat->is_valid > >> + ? ovn_port_find(ls_ports, nat->nb->logical_port) > >> + : nat->l3dgw_port; > >> > > > > Why do we still advertise NAT entries if they're not valid? There won't > be > > any flow installed for invalid NAT. IMO we should skip invalid > completely. > > > > +1 this is a bug. I can change it to check validity first and continue > if not valid. > > > > >> + parsed_route_add(advertising_od, NULL, &prefix, plen, false, > >> + nat->nb->external_ip, advertising_op, 0, > false, > >> + false, NULL, ROUTE_SOURCE_NAT, > &nat->nb->header_, > >> + tracked_port, routes); > >> + } > >> +} > >> + > >> +/* Generate parsed routes for NAT external IPs in lr_nat, for each ovn > >> port > >> + * in "od" that has enabled redistribution of NAT adresses.*/ > >> +void > >> +build_nat_parsed_routes(const struct ovn_datapath *od, > >> + const struct lr_nat_record *lr_nat, > >> + const struct hmap *ls_ports, > >> + struct hmap *routes) > >> +{ > >> + const struct ovn_port *op; > >> + HMAP_FOR_EACH (op, dp_node, &od->ports) { > >> + if (!drr_mode_NAT_is_set(op->dynamic_routing_redistribute)) { > >> + continue; > >> + } > >> + > >> + build_nat_parsed_route_for_port(op, lr_nat, ls_ports, routes); > >> + } > >> +} > >> + > >> +/* Similar to build_nat_parsed_routes, this function generates parsed > >> routes > >> + * for nat records in neighboring routers. For each ovn port in "od" > that > >> has > >> + * enabled redistribution of NAT adresses, look up their neighbors > (either > >> + * directly routers, or routers connected through common LS) and > advertise > >> + * thier external NAT IPs too.*/ > >> +void > >> +build_nat_connected_parsed_routes( > >> + const struct ovn_datapath *od, > >> + const struct lr_stateful_table *lr_stateful_table, > >> + const struct hmap *ls_ports, > >> + struct hmap *routes) > >> +{ > >> + const struct ovn_port *op; > >> + HMAP_FOR_EACH (op, dp_node, &od->ports) { > >> + if (!drr_mode_NAT_is_set(op->dynamic_routing_redistribute)) { > >> + continue; > >> + } > >> + > >> + if (!op->peer) { > >> + continue; > >> + } > >> + > >> + struct ovn_datapath *peer_od = op->peer->od; > >> + if (!peer_od->nbs && !peer_od->nbr) { > >> + continue; > >> + } > >> > > > > This shouldn't be possible e.g. the datapath will always be either router > > or switch. > > > > True, changing it into an assertion. > > > > >> + > >> + const struct ovn_port *peer_port = NULL; > >> + /* This is a directly connected LR peer. */ > >> + if (peer_od->nbr) { > >> + peer_port = op->peer; > >> + > >> + const struct lr_stateful_record *peer_lr_stateful = > >> + lr_stateful_table_find_by_index(lr_stateful_table, > >> + peer_od->index); > >> + if (!peer_lr_stateful) { > >> + continue; > >> + } > >> + > >> + /* Advertise peer's NAT routes via the local port too. */ > >> + build_nat_parsed_route_for_port(op, > >> peer_lr_stateful->lrnat_rec, > >> + ls_ports, routes); > >> + return; > >> + } > >> + > >> + /* This peer is LSP, we need to check all connected router > ports > >> + * for NAT.*/ > >> + for (size_t i = 0; i < peer_od->n_router_ports; i++) { > >> + peer_port = peer_od->router_ports[i]->peer; > >> + if (peer_port == op) { > >> + /* Skip advertising router. */ > >> + continue; > >> + } > >> + > >> + const struct lr_stateful_record *peer_lr_stateful = > >> + lr_stateful_table_find_by_index(lr_stateful_table, > >> + peer_port->od->index); > >> + if (!peer_lr_stateful) { > >> + continue; > >> + } > >> + > >> + /* Advertise peer's NAT routes via the local port too. */ > >> + build_nat_parsed_route_for_port(op, > >> peer_lr_stateful->lrnat_rec, > >> + ls_ports, routes); > >> + } > >> + } > >> +} > >> + > >> +/* This function adds new parsed route for each IP in lb_ips to > >> "routes".*/ > >> +static void > >> +build_lb_parsed_route_for_port(const struct ovn_port *advertising_op, > >> + const struct ovn_port *tracked_port, > >> + const struct ovn_lb_ip_set *lb_ips, > >> + struct hmap *routes) > >> +{ > >> + const struct ovn_datapath *advertising_od = advertising_op->od; > >> + > >> + const char *ip_address; > >> + SSET_FOR_EACH (ip_address, &lb_ips->ips_v4) { > >> + struct in6_addr prefix; > >> + ip46_parse(ip_address, &prefix); > >> + parsed_route_add(advertising_od, NULL, &prefix, 32, false, > >> + ip_address, advertising_op, 0, false, false, > >> + NULL, ROUTE_SOURCE_LB, > >> &advertising_op->nbrp->header_, > >> + tracked_port, routes); > >> + } > >> + SSET_FOR_EACH (ip_address, &lb_ips->ips_v6) { > >> + struct in6_addr prefix; > >> + ip46_parse(ip_address, &prefix); > >> + parsed_route_add(advertising_od, NULL, &prefix, 128, false, > >> + ip_address, advertising_op, 0, false, false, > >> + NULL, ROUTE_SOURCE_LB, > >> &advertising_op->nbrp->header_, > >> + tracked_port, routes); > >> + } > >> +} > >> + > >> +/* Similar to build_lb_parsed_routes, this function generates parsed > >> routes > >> + * for LB VIPs of neighboring routers. For each ovn port in "od" that > has > >> + * enabled redistribution of LB VIPs, look up their neighbors (either > >> + * directly routers, or routers connected through common LS) and > advertise > >> + * thier LB VIPs too.*/ > >> +void > >> +build_lb_connected_parsed_routes( > >> + const struct ovn_datapath *od, > >> + const struct lr_stateful_table *lr_stateful_table, > >> + struct hmap *routes) > >> +{ > >> + const struct ovn_port *op; > >> + HMAP_FOR_EACH (op, dp_node, &od->ports) { > >> + if (!drr_mode_LB_is_set(op->dynamic_routing_redistribute)) { > >> + continue; > >> + } > >> + > >> + if (!op->peer) { > >> + continue; > >> + } > >> + > >> + struct ovn_datapath *peer_od = op->peer->od; > >> + if (!peer_od->nbs && !peer_od->nbr) { > >> + continue; > >> + } > >> + > >> + const struct lr_stateful_record *lr_stateful_rec; > >> + const struct ovn_port *peer_port = NULL; > >> + /* This is directly connected LR peer. */ > >> + if (peer_od->nbr) { > >> + lr_stateful_rec = lr_stateful_table_find_by_index( > >> + lr_stateful_table, peer_od->index); > >> + peer_port = op->peer; > >> + build_lb_parsed_route_for_port(op, peer_port, > >> + lr_stateful_rec->lb_ips, > >> routes); > >> + return; > >> + } > >> + > >> + /* This peer is LSP, we need to check all connected router > ports > >> for > >> + * LBs.*/ > >> + for (size_t i = 0; i < peer_od->n_router_ports; i++) { > >> + peer_port = peer_od->router_ports[i]->peer; > >> + if (peer_port == op) { > >> + /* no need to check for LBs on ovn_port that initiated > >> this > >> + * function.*/ > >> + continue; > >> + } > >> + lr_stateful_rec = lr_stateful_table_find_by_index( > >> + lr_stateful_table, peer_port->od->index); > >> + > >> + build_lb_parsed_route_for_port(op, peer_port, > >> + lr_stateful_rec->lb_ips, > >> routes); > >> + } > >> + } > >> +} > >> + > >> +void > >> +build_lb_parsed_routes(const struct ovn_datapath *od, > >> + const struct ovn_lb_ip_set *lb_ips, > >> + struct hmap *routes) > >> +{ > >> + const struct ovn_port *op; > >> + HMAP_FOR_EACH (op, dp_node, &od->ports) { > >> + if (!drr_mode_LB_is_set(op->dynamic_routing_redistribute)) { > >> + continue; > >> + } > >> + > >> + /* Traffic processed by a load balancer is: > >> + * - handled by the chassis where a gateway router is bound > >> + * OR > >> + * - always redirected to a distributed gateway router port > >> + * > >> + * Advertise the LB IPs via all 'op' if this is a gateway > router > >> or > >> + * throuh all DGPs of this distributed router otherwise. */ > >> + struct ovn_port *op_ = NULL; > >> + size_t n_tracked_ports = !od->is_gw_router ? od->n_l3dgw_ports > : > >> 1; > >> + struct ovn_port **tracked_ports = !od->is_gw_router > >> + ? od->l3dgw_ports > >> + : &op_; > >> + > >> + for (size_t i = 0; i < n_tracked_ports; i++) { > >> + build_lb_parsed_route_for_port(op, tracked_ports[i], > lb_ips, > >> + routes); > >> + } > >> + } > >> + > >> +} > >> struct ecmp_route_list_node { > >> struct ovs_list list_node; > >> uint16_t id; /* starts from 1 */ > >> @@ -11468,6 +11710,8 @@ route_source_to_offset(enum route_source source) > >> return ROUTE_PRIO_OFFSET_STATIC; > >> case ROUTE_SOURCE_LEARNED: > >> return ROUTE_PRIO_OFFSET_LEARNED; > >> + case ROUTE_SOURCE_NAT: > >> + case ROUTE_SOURCE_LB: > >> default: > >> OVS_NOT_REACHED(); > >> } > >> diff --git a/northd/northd.h b/northd/northd.h > >> index b984e124d..1bb0cbd6f 100644 > >> --- a/northd/northd.h > >> +++ b/northd/northd.h > >> @@ -186,11 +186,15 @@ struct route_policy { > >> }; > >> > >> struct routes_data { > >> - struct hmap parsed_routes; > >> + struct hmap parsed_routes; /* Stores struct parsed_route. */ > >> struct simap route_tables; > >> struct hmap bfd_active_connections; > >> }; > >> > >> +struct dynamic_routes_data { > >> + struct hmap parsed_routes; /* Stores struct parsed_route. */ > >> +}; > >> + > >> struct route_policies_data { > >> struct hmap route_policies; > >> struct hmap bfd_active_connections; > >> @@ -308,10 +312,12 @@ struct mcast_port_info { > >> * (e.g., IGMP join/leave). */ > >> }; > >> > >> -#define DRR_MODES \ > >> - DRR_MODE(CONNECTED, 0) \ > >> +#define DRR_MODES \ > >> + DRR_MODE(CONNECTED, 0) \ > >> DRR_MODE(CONNECTED_AS_HOST, 1) \ > >> - DRR_MODE(STATIC, 2) > >> + DRR_MODE(STATIC, 2) \ > >> + DRR_MODE(NAT, 3) \ > >> + DRR_MODE(LB, 4) > >> > >> enum dynamic_routing_redistribute_mode_bits { > >> #define DRR_MODE(PROTOCOL, BIT) DRRM_##PROTOCOL##_BIT = BIT, > >> @@ -746,6 +752,10 @@ enum route_source { > >> ROUTE_SOURCE_STATIC, > >> /* The route is dynamically learned by an ovn-controller. */ > >> ROUTE_SOURCE_LEARNED, > >> + /* The route is derived from a NAT's external IP. */ > >> + ROUTE_SOURCE_NAT, > >> + /* The route is derived from a LB's VIP. */ > >> + ROUTE_SOURCE_LB, > >> }; > >> > >> struct parsed_route { > >> @@ -765,6 +775,7 @@ struct parsed_route { > >> const struct ovsdb_idl_row *source_hint; > >> char *lrp_addr_s; > >> const struct ovn_port *out_port; > >> + const struct ovn_port *tracked_port; /* May be NULL. */ > >> }; > >> > >> struct parsed_route *parsed_route_clone(const struct parsed_route *); > >> @@ -784,6 +795,7 @@ void parsed_route_add(const struct ovn_datapath *od, > >> const struct sset *ecmp_selection_fields, > >> enum route_source source, > >> const struct ovsdb_idl_row *source_hint, > >> + const struct ovn_port *tracked_port, > >> struct hmap *routes); > >> > >> bool > >> @@ -818,6 +830,20 @@ void route_policies_destroy(struct > >> route_policies_data *); > >> void build_parsed_routes(const struct ovn_datapath *, const struct > hmap *, > >> const struct hmap *, struct hmap *, struct > simap > >> *, > >> struct hmap *); > >> +void build_nat_parsed_routes(const struct ovn_datapath *, > >> + const struct lr_nat_record *, > >> + const struct hmap *ls_ports, > >> + struct hmap *routes); > >> +void build_nat_connected_parsed_routes(const struct ovn_datapath *, > >> + const struct lr_stateful_table > *, > >> + const struct hmap *ls_ports, > >> + struct hmap *routes); > >> +void build_lb_parsed_routes(const struct ovn_datapath *, > >> + const struct ovn_lb_ip_set *, > >> + struct hmap *); > >> +void build_lb_connected_parsed_routes(const struct ovn_datapath *, > >> + const struct lr_stateful_table *, > >> + struct hmap *routes); > >> uint32_t get_route_table_id(struct simap *, const char *); > >> void routes_init(struct routes_data *); > >> void routes_destroy(struct routes_data *); > >> diff --git a/ovn-nb.xml b/ovn-nb.xml > >> index 20d30dd58..f10c1e738 100644 > >> --- a/ovn-nb.xml > >> +++ b/ovn-nb.xml > >> @@ -3101,6 +3101,24 @@ or > >> <ref table="Advertised_Route" db="OVN_Southbound"/> table. > >> </p> > >> > >> + <p> > >> + If <code>lb</code> is in the list then northd will create > >> entries in > >> + <ref table="Advertised_Route" db="OVN_Southbound"/> table for > >> each > >> + Load Balancer VIP on this router and it's neighboring > routers. > >> + Neighboring routers are those that are either directly > >> connected, > >> + via Logical Router Port, or those that are connected via > shared > >> + Logical Switch. > >> + </p> > >> + > >> + <p> > >> + If <code>nat</code> is in the list then northd will create > >> entries in > >> + <ref table="Advertised_Route" db="OVN_Southbound"/> table for > >> each > >> + NAT's external IP on this router and it's neighboring > routers. > >> + Neighboring routers are those that are either directly > >> connected, > >> + via Logical Router Port, or those that are connected via > shared > >> + Logical Switch. > >> + </p> > >> + > >> <p> > >> This value can be overwritten on a per LRP basis using > >> <ref column="options" key="dynamic-routing-redistribute" > >> @@ -3950,6 +3968,24 @@ or > >> <ref table="Advertised_Route" db="OVN_Southbound"/> table. > >> </p> > >> > >> + <p> > >> + If <code>lb</code> is in the list then northd will create > >> entries in > >> + <ref table="Advertised_Route" db="OVN_Southbound"/> table for > >> each > >> + Load Balancer VIP on this port's router, and it's neighboring > >> + routers. Neighboring routers are those that are either > directly > >> + connected to this Logical Router Port, or those that are > >> connected > >> + via shared Logical Switch. > >> + </p> > >> + > >> + <p> > >> + If <code>nat</code> is in the list then northd will create > >> entries in > >> + <ref table="Advertised_Route" db="OVN_Southbound"/> table for > >> each > >> + NAT's external IP on this port's router, and it's neighboring > >> + routers. Neighboring routers are those that are either > directly > >> + connected to this Logical Router Port, or those that are > >> connected > >> + via shared Logical Switch. > >> + </p> > >> + > >> <p> > >> If not set the value from <ref column="options" > >> key="dynamic-routing-redistribute" table="Logical_Router"/> > on > >> the > >> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at > >> index 64991ff75..a18321818 100644 > >> --- a/tests/ovn-northd.at > >> +++ b/tests/ovn-northd.at > >> @@ -15542,3 +15542,631 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > >> > >> AT_CLEANUP > >> ]) > >> + > >> +OVN_FOR_EACH_NORTHD_NO_HV([ > >> +AT_SETUP([dynamic-routing - nat sync to sb IPv4]) > >> +AT_KEYWORDS([dynamic-routing]) > >> +ovn_start > >> + > >> +# Start with GW router and a single LRP. > >> +check ovn-nbctl lr-add lr0 > >> +check ovn-nbctl set Logical_Router lr0 \ > >> + options:dynamic-routing=true \ > >> + options:chassis=hv1 > >> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +check_row_count Advertised_Route 0 > >> + > >> +datapath=$(fetch_column Datapath_Binding _uuid external_ids:name=lr0) > >> +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-sw0) > >> + > >> +# Adding LRP dynamic-routing-redistribute=nat option and NAT rule > >> advertises a > >> +# route entry. > >> +check ovn-nbctl \ > >> + -- lrp-set-options lr0-sw0 dynamic-routing-redistribute=nat \ > >> + -- lr-nat-add lr0 dnat_and_snat 172.16.1.10 192.168.1.10 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +check_row_count Advertised_Route 1 > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="172.16.1.10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=[[]] > >> + > >> +# Add LR with distributed LRP connected to GW router through join LS > >> +# and switch dynamic-routing-redistribute to lrp-guest-join so that > >> +# it advertises NAT routes from the neighbors in the join LS. > >> +check ovn-nbctl \ > >> + -- remove Logical_Router_Port lr0-sw0 options > >> dynamic-routing-redistribute \ > >> + -- lrp-add lr0 lr0-join 00:00:00:00:ff:02 10.42.0.1/24 \ > >> + -- lrp-set-options lr0-join dynamic-routing-redistribute=nat \ > >> + -- ls-add ls-join \ > >> + -- lsp-add ls-join lsp-join-to-lr0 \ > >> + -- lsp-set-type lsp-join-to-lr0 router \ > >> + -- lsp-set-options lsp-join-to-lr0 router-port=lr0-join \ > >> + -- lsp-set-addresses lsp-join-to-lr0 router \ > >> + -- lr-add lr-guest0 \ > >> + -- lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 10.51.0.1/24 > \ > >> + -- lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 > 10.42.0.2/24 \ > >> + -- lsp-add ls-join lsp-join-to-guest0 \ > >> + -- lsp-set-type lsp-join-to-guest0 router \ > >> + -- lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \ > >> + -- lrp-set-gateway-chassis lrp-guest0-join hv1 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-join) > >> +pb2=$(fetch_column Port_Binding _uuid logical_port=lrp-guest0-join) > >> + > >> +check ovn-nbctl \ > >> + --add-route lr-nat-add lr-guest0 dnat_and_snat 172.16.2.10 > >> 192.168.2.10 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +check_row_count Advertised_Route 2 > >> + > >> +# Routes to local NAT addresses are advertised without tracked port. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="172.16.1.10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=[[]] > >> + > >> +# Routes to NAT addresses of neighboring routers are advertised with > >> +# tracked port pointing to the LRP of the neighboring router. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="172.16.2.10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=$pb2 > >> + > >> +# Add nonlocal LR with distributed LRP connected to GW router through > >> join LS. > >> +check ovn-nbctl \ > >> + -- lr-add lr-guest1 \ > >> + -- lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 10.51.1.1/24 > \ > >> + -- lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 > 10.42.0.3/24 \ > >> + -- lsp-add ls-join lsp-join-to-guest1 \ > >> + -- lsp-set-type lsp-join-to-guest1 router \ > >> + -- lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \ > >> + -- lrp-set-gateway-chassis lrp-guest1-join nonlocalhv > >> +check ovn-nbctl --wait=sb sync > >> + > >> +pb3=$(fetch_column Port_Binding _uuid logical_port=lrp-guest1-join) > >> + > >> +check ovn-nbctl \ > >> + -- --add-route lr-nat-add lr-guest1 dnat_and_snat 172.16.3.10 > >> 192.168.3.10 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +# Add a distributed router with distributed NAT connected to a > different > >> +# public LS. > >> +check ovn-nbctl \ > >> + -- ls-add ls-pub \ > >> + -- lsp-add ls-pub ls-pub-ln \ > >> + -- lsp-set-type ls-pub-ln localnet \ > >> + -- lsp-set-addresses ls-pub-ln unknown \ > >> + -- lsp-set-options ls-pub-ln network_name=public \ > >> + -- lr-add dr \ > >> + -- set logical_router dr options:dynamic-routing=true \ > >> + -- lrp-add dr lrp-dr-sw1 00:00:00:00:fd:01 10.52.1.1/24 \ > >> + -- lrp-add dr lrp-dr-pub 00:00:00:00:fd:02 10.42.0.4/24 \ > >> + -- lrp-set-options lrp-dr-pub dynamic-routing-redistribute=nat \ > >> + -- lsp-add ls-pub lsp-pub-to-guest2 \ > >> + -- lsp-set-type lsp-pub-to-guest2 router \ > >> + -- lsp-set-options lsp-pub-to-guest2 router-port=lrp-dr-pub \ > >> + -- lrp-set-gateway-chassis lrp-dr-pub nonlocalhv \ > >> + -- ls-add sw1 \ > >> + -- lsp-add sw1 sw1-lrp-dr \ > >> + -- lsp-set-type sw1-lrp-dr router \ > >> + -- lsp-set-addresses sw1-lrp-dr router \ > >> + -- lsp-set-options sw1-lrp-dr router-port=lrp-dr-sw1 \ > >> + -- lsp-add sw1 vm > >> +check ovn-nbctl --wait=sb sync > >> + > >> +datapath_dr=$(fetch_column Datapath_Binding _uuid external_ids:name=dr) > >> +pb_dr=$(fetch_column Port_Binding _uuid logical_port=lrp-dr-pub) > >> +pb_vm=$(fetch_column Port_Binding _uuid logical_port=vm) > >> + > >> +check ovn-nbctl \ > >> + -- --add-route lr-nat-add dr dnat_and_snat 172.16.4.10 > 192.168.3.20 \ > >> + vm 00:00:00:00:fd:03 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +check_row_count Advertised_Route 4 > >> + > >> +# Routes to local NAT addresses are advertised without tracked port. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="172.16.1.10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=[[]] > >> + > >> +# Routes to NAT addresses of neighboring routers are advertised with > >> +# tracked port pointing to the LRP of the neighboring router. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="172.16.2.10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=$pb2 > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="172.16.3.10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=$pb3 > >> + > >> +# Routes to distributed NAT addresses are advertised with tracked port > set > >> +# to the logical_port of the NAT entry. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="172.16.4.10" \ > >> + datapath=$datapath_dr \ > >> + logical_port=$pb_dr \ > >> + tracked_port=$pb_vm > >> + > >> +# Removing the option:dynamic-routing removes all routes. > >> +check ovn-nbctl --wait=sb remove Logical_Router lr0 option > dynamic-routing > >> +check_row_count Advertised_Route 1 > >> + > >> +check ovn-nbctl --wait=sb remove Logical_Router dr option > dynamic-routing > >> +check_row_count Advertised_Route 0 > >> + > >> +# And setting it again adds them again. > >> +check ovn-nbctl --wait=sb set Logical_Router lr0 > >> option:dynamic-routing=true > >> +check_row_count Advertised_Route 3 > >> + > >> +check ovn-nbctl --wait=sb set Logical_Router dr > >> option:dynamic-routing=true > >> +check_row_count Advertised_Route 4 > >> + > >> +# Removing the lr will remove all routes. > >> +check ovn-nbctl lr-del lr0 -- lr-del dr > >> +check ovn-nbctl --wait=sb sync > >> +check_row_count Advertised_Route 0 > >> + > >> +AT_CLEANUP > >> +]) > >> + > >> +OVN_FOR_EACH_NORTHD_NO_HV([ > >> +AT_SETUP([dynamic-routing - nat sync to sb IPv6]) > >> +AT_KEYWORDS([dynamic-routing]) > >> +ovn_start > >> + > >> +# Start with GW router and a single LRP. > >> +check ovn-nbctl lr-add lr0 > >> +check ovn-nbctl set Logical_Router lr0 \ > >> + options:dynamic-routing=true \ > >> + options:chassis=hv1 > >> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 2001:db0::1/64 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +check_row_count Advertised_Route 0 > >> + > >> +datapath=$(fetch_column Datapath_Binding _uuid external_ids:name=lr0) > >> +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-sw0) > >> + > >> +# Adding LRP dynamic-routing-redistribute=nat option and NAT rule > >> advertises > >> +# a route entry. > >> +check ovn-nbctl \ > >> + -- lrp-set-options lr0-sw0 dynamic-routing-redistribute=nat \ > >> + -- lr-nat-add lr0 dnat_and_snat 2001:db1::10 2001:db2::10 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +check_row_count Advertised_Route 1 > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="2001\:db1\:\:10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=[[]] > >> + > >> +# Add LR with distributed LRP connected to GW router through join LS > >> +# and switch dynamic-routing-redistribute to lrp-guest-join so that > >> +# it advertises NAT routes from the neighbors in the join LS. > >> +check ovn-nbctl \ > >> + -- remove Logical_Router_Port lr0-sw0 options > >> dynamic-routing-redistribute \ > >> + -- lrp-add lr0 lr0-join 00:00:00:00:ff:02 2001:db42::1/64 \ > >> + -- lrp-set-options lr0-join dynamic-routing-redistribute=nat \ > >> + -- ls-add ls-join \ > >> + -- lsp-add ls-join lsp-join-to-lr0 \ > >> + -- lsp-set-type lsp-join-to-lr0 router \ > >> + -- lsp-set-options lsp-join-to-lr0 router-port=lr0-join \ > >> + -- lsp-set-addresses lsp-join-to-lr0 router \ > >> + -- lr-add lr-guest0 \ > >> + -- lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 > 2001:db51::1/64 > >> \ > >> + -- lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 > >> 2001:db42::2/64 \ > >> + -- lsp-add ls-join lsp-join-to-guest0 \ > >> + -- lsp-set-type lsp-join-to-guest0 router \ > >> + -- lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \ > >> + -- lrp-set-gateway-chassis lrp-guest0-join hv1 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-join) > >> +pb2=$(fetch_column Port_Binding _uuid logical_port=lrp-guest0-join) > >> + > >> +check ovn-nbctl \ > >> + --add-route lr-nat-add lr-guest0 dnat_and_snat 2001:db3::10 > >> 2001:db4::10 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +check_row_count Advertised_Route 2 > >> + > >> +# Routes to local NAT addresses are advertised without tracked port. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="2001\:db1\:\:10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=[[]] > >> + > >> +# Routes to NAT addresses of neighboring routers are advertised with > >> +# tracked port pointing to the LRP of the neighboring router. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="2001\:db3\:\:10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=$pb2 > >> + > >> +# Add nonlocal LR with distributed LRP connected to GW router through > >> join LS. > >> +check ovn-nbctl \ > >> + -- lr-add lr-guest1 \ > >> + -- lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 > 2001:db52::1/64 > >> \ > >> + -- lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 > >> 2001:db42::3/64 \ > >> + -- lsp-add ls-join lsp-join-to-guest1 \ > >> + -- lsp-set-type lsp-join-to-guest1 router \ > >> + -- lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \ > >> + -- lrp-set-gateway-chassis lrp-guest1-join nonlocalhv > >> +check ovn-nbctl --wait=sb sync > >> + > >> +pb3=$(fetch_column Port_Binding _uuid logical_port=lrp-guest1-join) > >> + > >> +check ovn-nbctl \ > >> + --add-route lr-nat-add lr-guest1 dnat_and_snat 2001:db5::10 > >> 2001:db6::10 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +# Add a distributed router with distributed NAT connected to a > different > >> +# public LS. > >> +check ovn-nbctl \ > >> + -- ls-add ls-pub \ > >> + -- lsp-add ls-pub ls-pub-ln \ > >> + -- lsp-set-type ls-pub-ln localnet \ > >> + -- lsp-set-addresses ls-pub-ln unknown \ > >> + -- lsp-set-options ls-pub-ln network_name=public \ > >> + -- lr-add dr \ > >> + -- set logical_router dr options:dynamic-routing=true \ > >> + -- lrp-add dr lrp-dr-sw1 00:00:00:00:fd:01 2001:db53::1/64 \ > >> + -- lrp-add dr lrp-dr-pub 00:00:00:00:fd:02 2001:db42::4/64 \ > >> + -- lrp-set-options lrp-dr-pub dynamic-routing-redistribute=nat \ > >> + -- lsp-add ls-pub lsp-pub-to-guest2 \ > >> + -- lsp-set-type lsp-pub-to-guest2 router \ > >> + -- lsp-set-options lsp-pub-to-guest2 router-port=lrp-dr-pub \ > >> + -- lrp-set-gateway-chassis lrp-dr-pub nonlocalhv \ > >> + -- ls-add sw1 \ > >> + -- lsp-add sw1 sw1-lrp-dr \ > >> + -- lsp-set-type sw1-lrp-dr router \ > >> + -- lsp-set-addresses sw1-lrp-dr router \ > >> + -- lsp-set-options sw1-lrp-dr router-port=lrp-dr-sw1 \ > >> + -- lsp-add sw1 vm > >> +check ovn-nbctl --wait=sb sync > >> + > >> +datapath_dr=$(fetch_column Datapath_Binding _uuid external_ids:name=dr) > >> +pb_dr=$(fetch_column Port_Binding _uuid logical_port=lrp-dr-pub) > >> +pb_vm=$(fetch_column Port_Binding _uuid logical_port=vm) > >> + > >> +check ovn-nbctl \ > >> + -- --add-route lr-nat-add dr dnat_and_snat 2001:db6::10 > 2001:db6::20 \ > >> + vm 00:00:00:00:fd:03 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +check_row_count Advertised_Route 4 > >> + > >> +# Routes to local NAT addresses are advertised without tracked port. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="2001\:db1\:\:10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=[[]] > >> + > >> +# Routes to NAT addresses of neighboring routers are advertised with > >> +# tracked port pointing to the LRP of the neighboring router. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="2001\:db3\:\:10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=$pb2 > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="2001\:db5\:\:10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=$pb3 > >> + > >> +# Routes to distributed NAT addresses are advertised with tracked port > set > >> +# to the logical_port of the NAT entry. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="2001\:db6\:\:10" \ > >> + datapath=$datapath_dr \ > >> + logical_port=$pb_dr \ > >> + tracked_port=$pb_vm > >> + > >> +# Removing the option:dynamic-routing removes all routes. > >> +check ovn-nbctl --wait=sb remove Logical_Router lr0 option > dynamic-routing > >> +check_row_count Advertised_Route 1 > >> + > >> +check ovn-nbctl --wait=sb remove Logical_Router dr option > dynamic-routing > >> +check_row_count Advertised_Route 0 > >> + > >> +# And setting it again adds them again. > >> +check ovn-nbctl --wait=sb set Logical_Router lr0 > >> option:dynamic-routing=true > >> +check_row_count Advertised_Route 3 > >> + > >> +check ovn-nbctl --wait=sb set Logical_Router dr > >> option:dynamic-routing=true > >> +check_row_count Advertised_Route 4 > >> + > >> +# Removing the lr will remove all routes. > >> +check ovn-nbctl lr-del lr0 -- lr-del dr > >> +check ovn-nbctl --wait=sb sync > >> +check_row_count Advertised_Route 0 > >> + > >> +AT_CLEANUP > >> +]) > >> + > >> +OVN_FOR_EACH_NORTHD_NO_HV([ > >> +AT_SETUP([dynamic-routing - LB sync to sb IPv4]) > >> +AT_KEYWORDS([dynamic-routing]) > >> +ovn_start > >> + > >> +# Start with GW router and a single LRP, > >> +check ovn-nbctl lr-add lr0 > >> +check ovn-nbctl set Logical_Router lr0 \ > >> + options:dynamic-routing=true \ > >> + options:chassis=hv1 > >> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +check_row_count Advertised_Route 0 > >> + > >> +datapath=$(fetch_column Datapath_Binding _uuid external_ids:name=lr0) > >> +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-sw0) > >> + > >> +# Adding LRP dynamic-routing-lb-vips option and LB VIP rule advertises > >> +# a route entry. > >> +check ovn-nbctl lrp-set-options lr0-sw0 > dynamic-routing-redistribute=lb \ > >> + -- lb-add lb0 172.16.1.10:80 192.168.1.10:80,192.168.1.11:80 > \ > >> + -- lr-lb-add lr0 lb0 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +check_row_count Advertised_Route 1 > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="172.16.1.10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=[[]] > >> + > >> +# Add LR with distributed LRP connected to GW router through join LS > >> +# and switch dynamic-routing-redistribute to lrp-guest-join so that > >> +# it advertises LB VIP routes from the neighbors in the join LS. > >> +check ovn-nbctl \ > >> + -- remove Logical_Router_Port lr0-sw0 options > >> dynamic-routing-redistribute \ > >> + -- lrp-add lr0 lr0-join 00:00:00:00:ff:02 10.42.0.1/24 \ > >> + -- lrp-set-options lr0-join dynamic-routing-redistribute=lb \ > >> + -- ls-add ls-join \ > >> + -- lsp-add ls-join lsp-join-to-lr0 \ > >> + -- lsp-set-type lsp-join-to-lr0 router \ > >> + -- lsp-set-options lsp-join-to-lr0 router-port=lr0-join \ > >> + -- lsp-set-addresses lsp-join-to-lr0 router \ > >> + -- lr-add lr-guest0 \ > >> + -- lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 10.51.0.1/24 > \ > >> + -- lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 > 10.42.0.2/24 \ > >> + -- lrp-set-options lrp-guest0-join dynamic-routing-lb-vips=true \ > >> + -- lsp-add ls-join lsp-join-to-guest0 \ > >> + -- lsp-set-type lsp-join-to-guest0 router \ > >> + -- lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \ > >> + -- lrp-set-gateway-chassis lrp-guest0-join hv1 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-join) > >> +pb2=$(fetch_column Port_Binding _uuid logical_port=lrp-guest0-join) > >> + > >> +check ovn-nbctl \ > >> + --add-route lb-add lb1 172.16.2.10:80 192.168.2.10:80, > 192.168.2.11:80 > >> \ > >> + -- lr-lb-add lr-guest0 lb1 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +check_row_count Advertised_Route 2 > >> + > >> +# Routes to local LB VIPs are advertised without tracked port. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="172.16.1.10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=[[]] > >> + > >> +# Routes to LB VIPs of neighboring routers are advertised with > >> +# tracked port pointing to the LRP of the neighboring router. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="172.16.2.10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=$pb2 > >> + > >> +# Add nonlocal LR with distributed LRP connected to GW router through > >> join LS. > >> +check ovn-nbctl \ > >> + -- lr-add lr-guest1 \ > >> + -- lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 10.51.1.1/24 > \ > >> + -- lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 > 10.42.0.3/24 \ > >> + -- lsp-add ls-join lsp-join-to-guest1 \ > >> + -- lsp-set-type lsp-join-to-guest1 router \ > >> + -- lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \ > >> + -- lrp-set-gateway-chassis lrp-guest1-join nonlocalhv > >> +check ovn-nbctl --wait=sb sync > >> + > >> +pb3=$(fetch_column Port_Binding _uuid logical_port=lrp-guest1-join) > >> + > >> +check ovn-nbctl \ > >> + -- --add-route lb-add lb2 172.16.3.10:80 192.168.3.10:80, > >> 192.168.3.11:80 \ > >> + -- lr-lb-add lr-guest1 lb2 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +check_row_count Advertised_Route 3 > >> + > >> +# Routes to local LB VIPs are advertised without tracked port. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="172.16.1.10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=[[]] > >> + > >> +# Routes to LB VIPs of neighboring routers are advertised with > >> +# tracked port pointing to the LRP of the neighboring router. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="172.16.2.10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=$pb2 > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="172.16.3.10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=$pb3 > >> + > >> +# Removing the option:dynamic-routing removes all routes. > >> +check ovn-nbctl --wait=sb remove Logical_Router lr0 option > dynamic-routing > >> +check_row_count Advertised_Route 0 > >> + > >> +# And setting it again adds them again. > >> +check ovn-nbctl --wait=sb set Logical_Router lr0 > >> option:dynamic-routing=true > >> +check_row_count Advertised_Route 3 > >> + > >> +# Removing the lr will remove all routes. > >> +check ovn-nbctl --wait=sb lr-del lr0 > >> +check_row_count Advertised_Route 0 > >> + > >> +AT_CLEANUP > >> +]) > >> + > >> +OVN_FOR_EACH_NORTHD_NO_HV([ > >> +AT_SETUP([dynamic-routing - LB sync to sb IPv6]) > >> +AT_KEYWORDS([dynamic-routing]) > >> +ovn_start > >> + > >> +# Start with GW router and a single LRP. > >> +check ovn-nbctl lr-add lr0 > >> +check ovn-nbctl set Logical_Router lr0 \ > >> + options:dynamic-routing=true \ > >> + options:chassis=hv1 > >> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 2001:db0::1/64 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +check_row_count Advertised_Route 0 > >> + > >> +datapath=$(fetch_column Datapath_Binding _uuid external_ids:name=lr0) > >> +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-sw0) > >> + > >> +# Adding LRP dynamic-routing-lb-vips option and LB VIP rule advertises > >> +# a route entry. > >> +check ovn-nbctl lrp-set-options lr0-sw0 dynamic-routing-redistribute=lb > >> \ > >> + -- lb-add lb0 [[2001:db1::10]]:80 > >> [[2001:db2::10]]:80,[[2001:db2::11]]:80 \ > >> + -- lr-lb-add lr0 lb0 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +check_row_count Advertised_Route 1 > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="2001\:db1\:\:10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=[[]] > >> + > >> +# Add LR with distributed LRP connected to GW router through join LS > >> +# and switch dynamic-routing-redistribute to lrp-guest-join so that > >> +# it advertises LB VIP routes from the neighbors in the join LS. > >> +check ovn-nbctl \ > >> + -- remove Logical_Router_Port lr0-sw0 options > >> dynamic-routing-redistribute \ > >> + -- lrp-add lr0 lr0-join 00:00:00:00:ff:02 2001:db42::1/64 \ > >> + -- lrp-set-options lr0-join dynamic-routing-redistribute=lb \ > >> + -- ls-add ls-join \ > >> + -- lsp-add ls-join lsp-join-to-lr0 \ > >> + -- lsp-set-type lsp-join-to-lr0 router \ > >> + -- lsp-set-options lsp-join-to-lr0 router-port=lr0-join \ > >> + -- lsp-set-addresses lsp-join-to-lr0 router \ > >> + -- lr-add lr-guest0 \ > >> + -- lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 > 2001:db52::1/64 > >> \ > >> + -- lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 > >> 2001:db42::2/64 \ > >> + -- lrp-set-options lrp-guest0-join dynamic-routing-lb-vips=true \ > >> + -- lsp-add ls-join lsp-join-to-guest0 \ > >> + -- lsp-set-type lsp-join-to-guest0 router \ > >> + -- lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \ > >> + -- lrp-set-gateway-chassis lrp-guest0-join hv1 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-join) > >> +pb2=$(fetch_column Port_Binding _uuid logical_port=lrp-guest0-join) > >> + > >> +check ovn-nbctl \ > >> + -- --add-route lb-add lb1 [[2001:db3::10]]:80 > >> [[2001:db4::10]]:80,[[2001:db4::11]]:80 \ > >> + -- lr-lb-add lr-guest0 lb1 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +check_row_count Advertised_Route 2 > >> + > >> +# Routes to local LB VIPs are advertised without tracked port. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="2001\:db1\:\:10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=[[]] > >> + > >> +# Routes to LB VIPs of neighboring routers are advertised with > >> +# tracked port pointing to the LRP of the neighboring router. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="2001\:db3\:\:10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=$pb2 > >> + > >> +# Add nonlocal LR with distributed LRP connected to GW router through > >> join LS. > >> +check ovn-nbctl \ > >> + -- lr-add lr-guest1 \ > >> + -- lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 > 2001:db2::1/64 \ > >> + -- lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 > >> 2001:db42::3/64 \ > >> + -- lsp-add ls-join lsp-join-to-guest1 \ > >> + -- lsp-set-type lsp-join-to-guest1 router \ > >> + -- lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \ > >> + -- lrp-set-gateway-chassis lrp-guest1-join nonlocalhv > >> +check ovn-nbctl --wait=sb sync > >> + > >> +pb3=$(fetch_column Port_Binding _uuid logical_port=lrp-guest1-join) > >> + > >> +check ovn-nbctl \ > >> + -- --add-route lb-add lb2 [[2001:db5::10]]:80 > >> [[2001:db4::10]]:80,[[2001:db4::11]]:80 \ > >> + -- lr-lb-add lr-guest1 lb2 > >> +check ovn-nbctl --wait=sb sync > >> + > >> +check_row_count Advertised_Route 3 > >> + > >> +# Routes to local LB VIPs are advertised without tracked port. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="2001\:db1\:\:10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=[[]] > >> + > >> +# Routes to LB VIPs of neighboring routers are advertised with > >> +# tracked port pointing to the LRP of the neighboring router. > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="2001\:db3\:\:10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=$pb2 > >> +check_row_count Advertised_Route 1 \ > >> + ip_prefix="2001\:db5\:\:10" \ > >> + datapath=$datapath \ > >> + logical_port=$pb \ > >> + tracked_port=$pb3 > >> + > >> +# Removing the option:dynamic-routing removes all routes. > >> +check ovn-nbctl --wait=sb remove Logical_Router lr0 option > dynamic-routing > >> +check_row_count Advertised_Route 0 > >> + > >> +# And setting it again adds them again. > >> +check ovn-nbctl --wait=sb set Logical_Router lr0 > >> option:dynamic-routing=true > >> +check_row_count Advertised_Route 3 > >> + > >> +# Removing the lr will remove all routes. > >> +check ovn-nbctl --wait=sb lr-del lr0 > >> +check_row_count Advertised_Route 0 > >> + > >> +AT_CLEANUP > >> +]) > >> diff --git a/tests/system-ovn.at b/tests/system-ovn.at > >> index d24e50974..d7b16deda 100644 > >> --- a/tests/system-ovn.at > >> +++ b/tests/system-ovn.at > >> @@ -16561,3 +16561,741 @@ OVS_TRAFFIC_VSWITCHD_STOP(["/.*error > >> receiving.*/d > >> /.*terminating with signal 15.*/d"]) > >> AT_CLEANUP > >> ]) > >> + > >> +OVN_FOR_EACH_NORTHD([ > >> +AT_SETUP([dynamic-routing - LB VIPs IPv4]) > >> +AT_KEYWORDS([dynamic-routing]) > >> + > >> +CHECK_VRF() > >> +CHECK_CONNTRACK() > >> +CHECK_CONNTRACK_NAT() > >> + > >> +vrf=1000 > >> +VRF_RESERVE([$vrf]) > >> +ovn_start > >> +OVS_TRAFFIC_VSWITCHD_START() > >> +ADD_BR([br-int]) > >> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone]) > >> + > >> +# Set external-ids in br-int needed for ovn-controller > >> +ovs-vsctl \ > >> + -- set Open_vSwitch . external-ids:system-id=hv1 \ > >> + -- set Open_vSwitch . > >> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ > >> + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ > >> + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ > >> + -- set bridge br-int fail-mode=secure > >> other-config:disable-in-band=true > >> + > >> +# Start ovn-controller > >> +start_daemon ovn-controller > >> + > >> +ovn-appctl vlog/set route_exchange > >> +check ovn-nbctl -- lr-add R1 \ > >> + -- set Logical_Router R1 options:requested-tnl-key=$vrf > >> options:dynamic-routing=true > >> + > >> +check ovn-nbctl ls-add sw0 > >> +check ovn-nbctl ls-add public > >> + > >> +check ovn-nbctl --wait=hv sync > >> > > > > Missing in all tests: > > > > OVN_POPULATE_ARP > > wait_for_ports_up > > > > Good point, adding that. > > > > >> + > >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) > >> > > > > Has this potential to flake? Wouldn't OVS_WAIT_UNTIL be better here? > > > > Hmm, it should be OVS_WAIT_WHILE I think: > > OVS_WAIT_WHILE([ip link | grep -q ovnvrf$vrf:.*UP]) > > > > >> + > >> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 192.168.1.1/24 > >> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 > 172.16.1.1/24 \ > >> + -- lrp-set-options rp-public \ > >> + dynamic-routing-maintain-vrf=true \ > >> + dynamic-routing-redistribute=lb > >> + > >> +check ovn-nbctl set logical_router R1 options:chassis=hv1 > >> + > >> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \ > >> + type=router options:router-port=rp-sw0 \ > >> + -- lsp-set-addresses sw0-rp router > >> + > >> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port > >> public-rp \ > >> + type=router options:router-port=rp-public \ > >> + -- lsp-set-addresses public-rp router > >> + > >> +check ovs-vsctl set Open_vSwitch . > >> external-ids:ovn-bridge-mappings=phynet:br-ext > >> + > >> +check ovn-nbctl lsp-add public public1 \ > >> + -- lsp-set-addresses public1 unknown \ > >> + -- lsp-set-type public1 localnet \ > >> + -- lsp-set-options public1 network_name=phynet > >> + > >> +check ovn-nbctl --wait=hv sync > >> + > >> +AT_CHECK([test `ip route show table $vrf | wc -l` -eq 1], [1]) > >> > > + > >> + > >> +# Create a load balancer and associate to R1 > >> +check ovn-nbctl lb-add lb1 172.16.1.150:80 172.16.1.100:80 > >> +check ovn-nbctl lr-lb-add R1 lb1 > >> + > >> +check ovn-nbctl --wait=hv sync > >> + > >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) > >> + > >> +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > >> [dnl > >> +blackhole 172.16.1.150 proto 84 metric 1000]) > >> + > >> +# Add distributed router connected through "join" LS and ensure > >> +# that its LB VIPs are redistributed by R1. LS join has no > >> +# IP config, routers reach each other over IPv6 LLAs > >> +check ovn-nbctl --wait=sb \ > >> + -- \ > >> + remove Logical_Router_Port rp-public options > >> dynamic-routing-redistribute \ > >> + -- \ > >> + lrp-add R1 r1-join 00:00:00:00:ff:02 \ > >> + -- \ > >> + lrp-set-options r1-join dynamic-routing-redistribute=lb \ > >> + -- \ > >> + ls-add ls-join \ > >> + -- \ > >> + lsp-add ls-join lsp-join-to-r1 \ > >> + -- \ > >> + lsp-set-type lsp-join-to-r1 router \ > >> + -- \ > >> + lsp-set-options lsp-join-to-r1 router-port=r1-join \ > >> + -- \ > >> + lsp-set-addresses lsp-join-to-r1 router \ > >> + -- \ > >> + lr-add R2 \ > >> + -- \ > >> + lrp-add R2 r2-join 00:00:00:00:fe:02 \ > >> + -- \ > >> + lsp-add ls-join lsp-join-to-r2 \ > >> + -- \ > >> + lsp-set-type lsp-join-to-r2 router \ > >> + -- \ > >> + lsp-set-options lsp-join-to-r2 router-port=r2-join \ > >> + -- \ > >> + lrp-set-gateway-chassis r2-join hv1 > >> + > >> +# Create a load balancer and associate to R2 > >> +check ovn-nbctl lb-add lb2 10.42.10.10:80 192.168.123.10:80 > >> +check ovn-nbctl lr-lb-add R2 lb2 > >> + > >> +check ovn-nbctl --wait=hv sync > >> + > >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) > >> + > >> +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > >> [dnl > >> +blackhole 10.42.10.10 proto 84 metric 100 > >> +blackhole 172.16.1.150 proto 84 metric 1000]) > >> + > >> +# Move DGW of R2 to another chassis to verify that route priority will > >> decrease > >> +check ovn-nbctl --wait=hv \ > >> + -- \ > >> + lrp-del-gateway-chassis r2-join hv1 > >> + -- \ > >> + lrp-set-gateway-chassis r2-join hv123 > >> + > >> +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > >> [dnl > >> +blackhole 10.42.10.10 proto 84 metric 1000 > >> +blackhole 172.16.1.150 proto 84 metric 1000]) > >> + > >> +OVS_APP_EXIT_AND_WAIT([ovn-controller]) > >> + > >> +# Ensure system resources are cleaned up > >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) > >> +AT_CHECK([test `ip route show table $vrf | wc -l` -eq 1], [1]) > >> + > >> +as ovn-sb > >> +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > >> + > >> +as ovn-nb > >> +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > >> + > >> +as northd > >> +OVS_APP_EXIT_AND_WAIT([ovn-northd]) > >> + > >> +as > >> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d > >> +/Failed to acquire.*/d > >> +/connection dropped.*/d"]) > >> +AT_CLEANUP > >> +]) > >> + > >> +OVN_FOR_EACH_NORTHD([ > >> +AT_SETUP([dynamic-routing - LB VIPs on gateway router IPv6]) > >> +AT_KEYWORDS([dynamic-routing]) > >> + > >> +CHECK_VRF() > >> +CHECK_CONNTRACK() > >> +CHECK_CONNTRACK_NAT() > >> + > >> +vrf=1001 > >> +VRF_RESERVE([$vrf]) > >> +ovn_start > >> +OVS_TRAFFIC_VSWITCHD_START() > >> +ADD_BR([br-int]) > >> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone]) > >> + > >> +# Set external-ids in br-int needed for ovn-controller > >> +ovs-vsctl \ > >> + -- set Open_vSwitch . external-ids:system-id=hv1 \ > >> + -- set Open_vSwitch . > >> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ > >> + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ > >> + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ > >> + -- set bridge br-int fail-mode=secure > >> other-config:disable-in-band=true > >> + > >> +# Start ovn-controller > >> +start_daemon ovn-controller > >> + > >> +ovn-appctl vlog/set route_exchange > >> +check ovn-nbctl -- lr-add R1 \ > >> + -- set Logical_Router R1 options:requested-tnl-key=$vrf > >> options:dynamic-routing=true > >> + > >> +check ovn-nbctl ls-add sw0 > >> +check ovn-nbctl ls-add public > >> + > >> +check ovn-nbctl --wait=hv sync > >> + > >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) > >> + > >> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 2001:db8:100::1/64 > >> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 > >> 2001:db8:1001::1/64 \ > >> + -- lrp-set-options rp-public \ > >> + dynamic-routing-maintain-vrf=true \ > >> + dynamic-routing-redistribute=lb > >> + > >> +check ovn-nbctl set logical_router R1 options:chassis=hv1 > >> + > >> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \ > >> + type=router options:router-port=rp-sw0 \ > >> + -- lsp-set-addresses sw0-rp router > >> + > >> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port > >> public-rp \ > >> + type=router options:router-port=rp-public \ > >> + -- lsp-set-addresses public-rp router > >> + > >> +check ovs-vsctl set Open_vSwitch . > >> external-ids:ovn-bridge-mappings=phynet:br-ext > >> + > >> +check ovn-nbctl lsp-add public public1 \ > >> + -- lsp-set-addresses public1 unknown \ > >> + -- lsp-set-type public1 localnet \ > >> + -- lsp-set-options public1 network_name=phynet > >> + > >> +check ovn-nbctl --wait=hv sync > >> + > >> +AT_CHECK([test `ip -6 route show table $vrf | wc -l` -eq 1], [1]) > >> + > >> +# Create a load balancer and associate to R1 > >> +check ovn-nbctl lb-add lb1 [[2001:db8:1001::150]]:80 > >> [[2001:db8:2001::100]]:80 > >> +check ovn-nbctl lr-lb-add R1 lb1 > >> + > >> +check ovn-nbctl --wait=hv sync > >> + > >> +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP]) > >> +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1]) > >> + > >> +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk > '{$1=$1};1'], > >> [dnl > >> +blackhole 2001:db8:1001::150 dev lo proto 84 metric 1000 pref medium]) > >> + > >> +# Add distributed router connected through "join" LS and ensure > >> +# that its LB VIPs are redistributed by R1. LS join has no > >> +# IP config, routers reach each other over IPv6 LLAs > >> +check ovn-nbctl --wait=sb \ > >> + -- \ > >> + remove Logical_Router_Port rp-public options > >> dynamic-routing-redistribute \ > >> + -- \ > >> + lrp-add R1 r1-join 00:00:00:00:ff:02 \ > >> + -- \ > >> + lrp-set-options r1-join dynamic-routing-redistribute=lb \ > >> + -- \ > >> + ls-add ls-join \ > >> + -- \ > >> + lsp-add ls-join lsp-join-to-r1 \ > >> + -- \ > >> + lsp-set-type lsp-join-to-r1 router \ > >> + -- \ > >> + lsp-set-options lsp-join-to-r1 router-port=r1-join \ > >> + -- \ > >> + lsp-set-addresses lsp-join-to-r1 router \ > >> + -- \ > >> + lr-add R2 \ > >> + -- \ > >> + lrp-add R2 r2-join 00:00:00:00:fe:02 \ > >> + -- \ > >> + lsp-add ls-join lsp-join-to-r2 \ > >> + -- \ > >> + lsp-set-type lsp-join-to-r2 router \ > >> + -- \ > >> + lsp-set-options lsp-join-to-r2 router-port=r2-join \ > >> + -- \ > >> + lrp-set-gateway-chassis r2-join hv1 > >> + > >> +# Create a load balancer and associate to R2 > >> +check ovn-nbctl lb-add lb2 [[2001:db8:3001::150]]:80 > >> [[2001:db8:4001::100]]:80 > >> +check ovn-nbctl lr-lb-add R2 lb2 > >> + > >> +check ovn-nbctl --wait=hv sync > >> + > >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) > >> + > >> +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk > '{$1=$1};1'], > >> [dnl > >> +blackhole 2001:db8:1001::150 dev lo proto 84 metric 1000 pref medium > >> +blackhole 2001:db8:3001::150 dev lo proto 84 metric 100 pref medium]) > >> + > >> +# Move DGW of R2 to another chassis to verify that route priority will > >> decrease > >> +check ovn-nbctl --wait=hv \ > >> + -- \ > >> + lrp-del-gateway-chassis r2-join hv1 > >> + -- \ > >> + lrp-set-gateway-chassis r2-join hv123 > >> + > >> +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk > '{$1=$1};1'], > >> [dnl > >> +blackhole 2001:db8:1001::150 dev lo proto 84 metric 1000 pref medium > >> +blackhole 2001:db8:3001::150 dev lo proto 84 metric 1000 pref medium]) > >> + > >> +OVS_APP_EXIT_AND_WAIT([ovn-controller]) > >> + > >> +# Ensure system resources are cleaned up > >> +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP], [1]) > >> +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1], [1]) > >> + > >> +as ovn-sb > >> +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > >> + > >> +as ovn-nb > >> +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > >> + > >> +as northd > >> +OVS_APP_EXIT_AND_WAIT([ovn-northd]) > >> + > >> +as > >> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d > >> +/Failed to acquire.*/d > >> +/connection dropped.*/d"]) > >> +AT_CLEANUP > >> +]) > >> + > >> +OVN_FOR_EACH_NORTHD([ > >> +AT_SETUP([dynamic-routing - DNAT and DNAT_AND_SNAT on gateway router > >> IPv4]) > >> +AT_KEYWORDS([dynamic-routing]) > >> + > >> +CHECK_VRF() > >> +CHECK_CONNTRACK() > >> +CHECK_CONNTRACK_NAT() > >> + > >> +vrf=1002 > >> +VRF_RESERVE([$vrf]) > >> +ovn_start > >> +OVS_TRAFFIC_VSWITCHD_START() > >> +ADD_BR([br-int]) > >> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone]) > >> + > >> +# Set external-ids in br-int needed for ovn-controller > >> +ovs-vsctl \ > >> + -- set Open_vSwitch . external-ids:system-id=hv1 \ > >> + -- set Open_vSwitch . > >> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ > >> + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ > >> + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ > >> + -- set bridge br-int fail-mode=secure > >> other-config:disable-in-band=true > >> + > >> +# Start ovn-controller > >> +start_daemon ovn-controller > >> + > >> +ovn-appctl vlog/set route_exchange > >> +check ovn-nbctl -- lr-add R1 \ > >> + -- set Logical_Router R1 options:requested-tnl-key=$vrf > >> options:dynamic-routing=true > >> + > >> +check ovn-nbctl ls-add sw0 > >> +check ovn-nbctl ls-add public > >> + > >> +check ovn-nbctl --wait=hv sync > >> + > >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) > >> + > >> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 192.168.1.1/24 > >> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 > 172.16.1.1/24 \ > >> + -- lrp-set-options rp-public \ > >> + dynamic-routing-maintain-vrf=true \ > >> + dynamic-routing-redistribute=nat > >> + > >> +check ovn-nbctl set logical_router R1 options:chassis=hv1 > >> + > >> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \ > >> + type=router options:router-port=rp-sw0 \ > >> + -- lsp-set-addresses sw0-rp router > >> + > >> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port > >> public-rp \ > >> + type=router options:router-port=rp-public \ > >> + -- lsp-set-addresses public-rp router > >> + > >> +check ovs-vsctl set Open_vSwitch . > >> external-ids:ovn-bridge-mappings=phynet:br-ext > >> + > >> +check ovn-nbctl lsp-add public public1 \ > >> + -- lsp-set-addresses public1 unknown \ > >> + -- lsp-set-type public1 localnet \ > >> + -- lsp-set-options public1 network_name=phynet > >> + > >> +check ovn-nbctl --wait=hv sync > >> + > >> +AT_CHECK([test `ip route show table $vrf | wc -l` -eq 2], [1]) > >> + > >> +# Create dnat_and_snat, dnat rules in R1 > >> +check ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.10 192.168.1.10 > >> +check ovn-nbctl lr-nat-add R1 dnat 172.16.1.11 192.168.1.11 > >> + > >> +check ovn-nbctl --wait=hv sync > >> + > >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) > >> + > >> +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > >> [dnl > >> +blackhole 172.16.1.10 proto 84 metric 1000 > >> +blackhole 172.16.1.11 proto 84 metric 1000]) > >> + > >> +# Add distributed router connected through "join" LS and ensure > >> +# that its external NAT IPs are redistributed by R1. LS join has no > >> +# IP config, routers reach each other over IPv6 LLAs > >> +check ovn-nbctl --wait=sb \ > >> + -- \ > >> + remove Logical_Router_Port rp-public options > >> dynamic-routing-redistribute \ > >> + -- \ > >> + lrp-add R1 r1-join 00:00:00:00:ff:02 \ > >> + -- \ > >> + lrp-set-options r1-join dynamic-routing-redistribute=nat \ > >> + -- \ > >> + ls-add ls-join \ > >> + -- \ > >> + lsp-add ls-join lsp-join-to-r1 \ > >> + -- \ > >> + lsp-set-type lsp-join-to-r1 router \ > >> + -- \ > >> + lsp-set-options lsp-join-to-r1 router-port=r1-join \ > >> + -- \ > >> + lsp-set-addresses lsp-join-to-r1 router \ > >> + -- \ > >> + lr-add R2 \ > >> + -- \ > >> + lrp-add R2 r2-join 00:00:00:00:fe:02 \ > >> + -- \ > >> + lsp-add ls-join lsp-join-to-r2 \ > >> + -- \ > >> + lsp-set-type lsp-join-to-r2 router \ > >> + -- \ > >> + lsp-set-options lsp-join-to-r2 router-port=r2-join \ > >> + -- \ > >> + ha-chassis-group-add g1 \ > >> + -- \ > >> + ha-chassis-group-add-chassis g1 hv1 10 > >> + > >> +group_uuid=$(ovn-nbctl get ha_chassis_group g1 _uuid) > >> +check ovn-nbctl set logical_router_port r2-join > >> ha_chassis_group="$group_uuid" > >> + > >> +# Create NAT on R2 > >> +check ovn-nbctl lr-nat-add R2 dnat_and_snat 10.42.10.10 192.168.1.10 > >> +check ovn-nbctl lr-nat-add R2 dnat_and_snat 10.42.10.11 192.168.1.11 > >> + > >> +check ovn-nbctl --wait=hv sync > >> + > >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) > >> + > >> +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > >> [dnl > >> +blackhole 10.42.10.10 proto 84 metric 100 > >> +blackhole 10.42.10.11 proto 84 metric 100 > >> +blackhole 172.16.1.10 proto 84 metric 1000 > >> +blackhole 172.16.1.11 proto 84 metric 1000]) > >> + > >> +# Move DGW of R2 to another chassis to verify that route priority will > >> decrease > >> +check ovn-nbctl --wait=hv \ > >> + -- \ > >> + ha-chassis-group-add-chassis g1 hv2 20 \ > >> + -- \ > >> + ha-chassis-group-remove-chassis g1 hv1 > >> + > >> +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > >> [dnl > >> +blackhole 10.42.10.10 proto 84 metric 1000 > >> +blackhole 10.42.10.11 proto 84 metric 1000 > >> +blackhole 172.16.1.10 proto 84 metric 1000 > >> +blackhole 172.16.1.11 proto 84 metric 1000]) > >> + > >> +# Add "guest" LS connected the distributed router R2 and one "VM" > called > >> +# guest1. > >> +# Also, connect R2 to ls-join via nother DGW > >> +check ovn-nbctl --wait=sb \ > >> + -- \ > >> + lrp-add R2 r2-guest 00:00:00:00:fd:02 192.168.2.1/24 \ > >> + -- \ > >> + lrp-add R2 r2-join-dgw2 00:00:00:00:fd:02 \ > >> + -- \ > >> + lsp-add ls-join lsp-join-to-r2-dgw2 \ > >> + -- \ > >> + lsp-set-type lsp-join-to-r2-dgw2 router \ > >> + -- \ > >> + lsp-set-options lsp-join-to-r2-dgw2 router-port=r2-join-dgw2 \ > >> + -- \ > >> + ls-add ls-guest \ > >> + -- \ > >> + lsp-add ls-guest lsp-guest-to-r2 \ > >> + -- \ > >> + lsp-set-type lsp-guest-to-r2 router \ > >> + -- \ > >> + lsp-set-options lsp-guest-to-r2 router-port=r2-guest \ > >> + -- \ > >> + lsp-set-addresses lsp-guest-to-r2 router \ > >> + -- \ > >> + lsp-add ls-guest guest1 \ > >> + -- \ > >> + lsp-set-addresses guest1 '00:00:00:00:fc:03 192.168.2.10' > >> + > >> +check ovn-nbctl set logical_router_port r2-join-dgw2 \ > >> + ha_chassis_group="$group_uuid" > >> +# Bind guest1 on hv1 > >> +ADD_NAMESPACES(guest1) > >> +ADD_VETH(guest1, guest1, br-int, "192.168.2.10/24", > "00:00:00:00:fd:03", > >> \ > >> + "192.168.2.1") > >> + > >> +# Add distributed NAT on R2 for guest1 and ensure that it's advertised > >> +# with better metric on chassis hv1 even when DGP for R2 is elsewhere. > >> +check ovn-nbctl --gateway-port \ > >> + r2-join-dgw2 --add-route lr-nat-add R2 dnat_and_snat 10.42.20.11 \ > >> + 192.168.2.10 guest1 00:00:00:00:fc:04 > >> +check ovn-nbctl --wait=hv sync > >> + > >> +ovn-sbctl list advertised_route > >> +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], > >> [dnl > >> +blackhole 10.42.10.10 proto 84 metric 1000 > >> +blackhole 10.42.10.11 proto 84 metric 1000 > >> +blackhole 10.42.20.11 proto 84 metric 100 > >> +blackhole 172.16.1.10 proto 84 metric 1000 > >> +blackhole 172.16.1.11 proto 84 metric 1000]) > >> + > >> +OVS_APP_EXIT_AND_WAIT([ovn-controller]) > >> + > >> +# Ensure system resources are cleaned up > >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) > >> +AT_CHECK([test `ip route show table $vrf | wc -l` -eq 1], [1]) > >> + > >> +as ovn-sb > >> +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > >> + > >> +as ovn-nb > >> +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > >> + > >> +as northd > >> +OVS_APP_EXIT_AND_WAIT([ovn-northd]) > >> + > >> +as > >> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d > >> +/Failed to acquire.*/d > >> +/connection dropped.*/d"]) > >> +AT_CLEANUP > >> +]) > >> + > >> +OVN_FOR_EACH_NORTHD([ > >> +AT_SETUP([dynamic-routing - DNAT and DNAT_AND_SNAT on gateway router > >> IPv6]) > >> +AT_KEYWORDS([dynamic-routing]) > >> + > >> +CHECK_VRF() > >> +CHECK_CONNTRACK() > >> +CHECK_CONNTRACK_NAT() > >> + > >> +vrf=1003 > >> +VRF_RESERVE([$vrf]) > >> +ovn_start > >> +OVS_TRAFFIC_VSWITCHD_START() > >> +ADD_BR([br-int]) > >> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone]) > >> + > >> +# Set external-ids in br-int needed for ovn-controller > >> +ovs-vsctl \ > >> + -- set Open_vSwitch . external-ids:system-id=hv1 \ > >> + -- set Open_vSwitch . > >> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ > >> + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ > >> + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ > >> + -- set bridge br-int fail-mode=secure > >> other-config:disable-in-band=true > >> + > >> +# Start ovn-controller > >> +start_daemon ovn-controller > >> + > >> +ovn-appctl vlog/set route_exchange > >> +check ovn-nbctl -- lr-add R1 \ > >> + -- set Logical_Router R1 options:requested-tnl-key=$vrf > >> options:dynamic-routing=true > >> + > >> +check ovn-nbctl ls-add sw0 > >> +check ovn-nbctl ls-add public > >> + > >> +check ovn-nbctl --wait=hv sync > >> + > >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) > >> + > >> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 2001:db8:100::1/64 > >> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 > >> 2001:db8:1003::1/64 \ > >> + -- lrp-set-options rp-public \ > >> + dynamic-routing-maintain-vrf=true \ > >> + dynamic-routing-redistribute=nat > >> + > >> +check ovn-nbctl set logical_router R1 options:chassis=hv1 > >> + > >> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \ > >> + type=router options:router-port=rp-sw0 \ > >> + -- lsp-set-addresses sw0-rp router > >> + > >> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port > >> public-rp \ > >> + type=router options:router-port=rp-public \ > >> + -- lsp-set-addresses public-rp router > >> + > >> +check ovs-vsctl set Open_vSwitch . > >> external-ids:ovn-bridge-mappings=phynet:br-ext > >> + > >> +check ovn-nbctl lsp-add public public1 \ > >> + -- lsp-set-addresses public1 unknown \ > >> + -- lsp-set-type public1 localnet \ > >> + -- lsp-set-options public1 network_name=phynet > >> + > >> +check ovn-nbctl --wait=hv sync > >> + > >> +AT_CHECK([test `ip -6 route show table $vrf | wc -l` -eq 2], [1]) > >> + > >> +# Create dnat_and_snat, dnat rules in R1 > >> +check ovn-nbctl lr-nat-add R1 \ > >> + dnat_and_snat 2001:db8:1003::150 2001:db8:100::100 > >> +check ovn-nbctl lr-nat-add R1 \ > >> + dnat 2001:db8:1003::151 2001:db8:100::100 > >> + > >> +check ovn-nbctl --wait=hv sync > >> + > >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) > >> +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk > '{$1=$1};1'], > >> [dnl > >> +blackhole 2001:db8:1003::150 dev lo proto 84 metric 1000 pref medium > >> +blackhole 2001:db8:1003::151 dev lo proto 84 metric 1000 pref medium]) > >> + > >> +# Add distributed router connected through "join" LS and ensure > >> +# that its external NAT IPs are redistributed by R1. LS join has no > >> +# IP config, routers reach each other over IPv6 LLAs > >> +check ovn-nbctl --wait=sb \ > >> + -- \ > >> + remove Logical_Router_Port rp-public options > >> dynamic-routing-redistribute \ > >> + -- \ > >> + lrp-add R1 r1-join 00:00:00:00:ff:02 \ > >> + -- \ > >> + lrp-set-options r1-join dynamic-routing-redistribute=nat \ > >> + -- \ > >> + ls-add ls-join \ > >> + -- \ > >> + lsp-add ls-join lsp-join-to-r1 \ > >> + -- \ > >> + lsp-set-type lsp-join-to-r1 router \ > >> + -- \ > >> + lsp-set-options lsp-join-to-r1 router-port=r1-join \ > >> + -- \ > >> + lsp-set-addresses lsp-join-to-r1 router \ > >> + -- \ > >> + lr-add R2 \ > >> + -- \ > >> + lrp-add R2 r2-join 00:00:00:00:fe:02 \ > >> + -- \ > >> + lsp-add ls-join lsp-join-to-r2 \ > >> + -- \ > >> + lsp-set-type lsp-join-to-r2 router \ > >> + -- \ > >> + lsp-set-options lsp-join-to-r2 router-port=r2-join \ > >> + -- \ > >> + ha-chassis-group-add g1 \ > >> + -- \ > >> + ha-chassis-group-add-chassis g1 hv1 10 > >> + > >> +group_uuid=$(ovn-nbctl get ha_chassis_group g1 _uuid) > >> +check ovn-nbctl set logical_router_port r2-join > >> ha_chassis_group="$group_uuid" > >> + > >> +# Create NAT on R2 > >> +check ovn-nbctl lr-nat-add R2 \ > >> + dnat_and_snat 2001:db8:1004::150 2001:db8:201::100 > >> +check ovn-nbctl lr-nat-add R2 \ > >> + dnat 2001:db8:1004::151 2001:db8:201::101 > >> + > >> +check ovn-nbctl --wait=hv sync > >> + > >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) > >> + > >> +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk > '{$1=$1};1'], > >> [dnl > >> +blackhole 2001:db8:1003::150 dev lo proto 84 metric 1000 pref medium > >> +blackhole 2001:db8:1003::151 dev lo proto 84 metric 1000 pref medium > >> +blackhole 2001:db8:1004::150 dev lo proto 84 metric 100 pref medium > >> +blackhole 2001:db8:1004::151 dev lo proto 84 metric 100 pref medium]) > >> + > >> +# Move DGW of R2 to another chassis to verify that route priority will > >> decrease > >> +check ovn-nbctl --wait=hv \ > >> + -- \ > >> + ha-chassis-group-add-chassis g1 hv2 20 \ > >> + -- \ > >> + ha-chassis-group-remove-chassis g1 hv1 > >> + > >> +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk > '{$1=$1};1'], > >> [dnl > >> +blackhole 2001:db8:1003::150 dev lo proto 84 metric 1000 pref medium > >> +blackhole 2001:db8:1003::151 dev lo proto 84 metric 1000 pref medium > >> +blackhole 2001:db8:1004::150 dev lo proto 84 metric 1000 pref medium > >> +blackhole 2001:db8:1004::151 dev lo proto 84 metric 1000 pref medium]) > >> + > >> +# Add "guest" LS connected the distributed router R2 and one "VM" > called > >> +# guest1. > >> +# Also, connect R2 to ls-join via nother DGW > >> +check ovn-nbctl --wait=sb \ > >> + -- \ > >> + lrp-add R2 r2-guest 00:00:00:00:fd:02 2001:db9:2000::1/64 \ > >> + -- \ > >> + lrp-add R2 r2-join-dgw2 00:00:00:00:fd:02 \ > >> + -- \ > >> + lsp-add ls-join lsp-join-to-r2-dgw2 \ > >> + -- \ > >> + lsp-set-type lsp-join-to-r2-dgw2 router \ > >> + -- \ > >> + lsp-set-options lsp-join-to-r2-dgw2 router-port=r2-join-dgw2 \ > >> + -- \ > >> + ls-add ls-guest \ > >> + -- \ > >> + lsp-add ls-guest lsp-guest-to-r2 \ > >> + -- \ > >> + lsp-set-type lsp-guest-to-r2 router \ > >> + -- \ > >> + lsp-set-options lsp-guest-to-r2 router-port=r2-guest \ > >> + -- \ > >> + lsp-set-addresses lsp-guest-to-r2 router \ > >> + -- \ > >> + lsp-add ls-guest guest1 \ > >> + -- \ > >> + lsp-set-addresses guest1 '00:00:00:00:fc:03 2001:db9:2000::10' > >> + > >> +check ovn-nbctl set logical_router_port r2-join-dgw2 > >> ha_chassis_group="$group_uuid" > >> +# Bind guest1 on hv1 > >> +ADD_NAMESPACES(guest1) > >> +ADD_VETH(guest1, guest1, br-int, "2001:db9:2000::10/64", > >> "00:00:00:00:fd:03", \ > >> + "2001:db9:2000::1", "nodad") > >> + > >> +# Add distributed NAT on R2 for guest1 and ensure that it's advertised > >> +# with better metric on chassis hv1 even when DGP for R2 is elsewhere. > >> +check ovn-nbctl --gateway-port \ > >> + r2-join-dgw2 --add-route lr-nat-add R2 dnat_and_snat \ > >> + 2001:db8:1005::150 2001:db9:2000::10 guest1 00:00:00:00:fc:04 > >> +check ovn-nbctl --wait=hv sync > >> + > >> +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk > '{$1=$1};1'], > >> [dnl > >> +blackhole 2001:db8:1003::150 dev lo proto 84 metric 1000 pref medium > >> +blackhole 2001:db8:1003::151 dev lo proto 84 metric 1000 pref medium > >> +blackhole 2001:db8:1004::150 dev lo proto 84 metric 1000 pref medium > >> +blackhole 2001:db8:1004::151 dev lo proto 84 metric 1000 pref medium > >> +blackhole 2001:db8:1005::150 dev lo proto 84 metric 100 pref medium]) > >> + > >> +OVS_APP_EXIT_AND_WAIT([ovn-controller]) > >> +# Ensure system resources are cleaned up > >> +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP], [1]) > >> +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2], [1]) > >> + > >> +as ovn-sb > >> +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > >> + > >> +as ovn-nb > >> +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > >> + > >> +as northd > >> +OVS_APP_EXIT_AND_WAIT([ovn-northd]) > >> + > >> +as > >> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d > >> +/Failed to acquire.*/d > >> +/connection dropped.*/d"]) > >> +AT_CLEANUP > >> +]) > >> + > >> + > >> -- > >> 2.43.0 > >> > >> > > Thanks, > > Ales > > > > I tried to address Ales' review comments here: > https://github.com/dceara/ovn/commit/3655aa3 > > But I'm also going to reindent and do some minor style changes in the > system tests. > > In the meantime, Ales, Martin, please let me know if the incremental > changes look OK to you, I can squash them in when applying the series. > > Thanks, > Dumitru > > > The incremental change looks good to me. You can put Acked-by: Ales Musil <[email protected]> on top of that. Thanks, Ales _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
