On 2/14/25 9:57 AM, Ales Musil wrote: > On Fri, Feb 14, 2025 at 7:34 AM Martin Kalcok <[email protected]> > wrote: > >> This change builds on top of the new "dynamic routing" OVN feature >> that allows advertising routes to the fabric network. New accepted >> values are added to "dynamic-routing-redistribute" option. >> >> * nat - When included in the option, ovn-controller will advertise >> routes for external NAT IPs of the LR, as well as those >> of the neighboring routers (directly connected or through >> the shared LS). >> * lb - When included in the option, ovn-controller will advertise >> routes for LB VIPs of the LR, as well as those >> of the neighboring routers (directly connected or through >> the shared LS). >> >> Co-authored-by: Frode Nordahl <[email protected]> >> Co-authored-by: Dumitru Ceara <[email protected]> >> Signed-off-by: Frode Nordahl <[email protected]> >> Signed-off-by: Dumitru Ceara <[email protected]> >> Signed-off-by: Martin Kalcok <[email protected]> >> --- > > > Hi Martin, >
Hi Ales, Thanks for the review! > I have a couple of comments down below. > > TODO.rst | 10 + >> lib/stopwatch-names.h | 1 + >> northd/en-advertised-route-sync.c | 198 ++++++-- >> northd/en-advertised-route-sync.h | 3 + >> northd/en-learned-route-sync.c | 3 +- >> northd/inc-proc-northd.c | 5 + >> northd/northd.c | 254 +++++++++- >> northd/northd.h | 34 +- >> ovn-nb.xml | 36 ++ >> tests/ovn-northd.at | 628 +++++++++++++++++++++++++ >> tests/system-ovn.at | 738 ++++++++++++++++++++++++++++++ >> 11 files changed, 1861 insertions(+), 49 deletions(-) >> >> diff --git a/TODO.rst b/TODO.rst >> index d75db3152..593b9a580 100644 >> --- a/TODO.rst >> +++ b/TODO.rst >> @@ -146,6 +146,9 @@ OVN To-do List >> * Add incremental processing for northd when the Learned_Route table >> changes. >> Currently en_lflow is fully recomputed whenever such a change happens. >> >> + * Add incremental processing of en_dynamic_routes for stateful >> configuration >> + changes. >> + >> * The ovn-controller currently loads all Advertised_Route entries on >> startup. >> This is to prevent deleting our routes on restart. If we defer >> updating >> routes until we are sure to have loaded all necessary >> Advertised_Routes >> @@ -156,3 +159,10 @@ OVN To-do List >> monitoring conditions to update before we actually try to learn >> routes. >> Otherwise we could try to add duplicated Learned_Routes and the ovnsb >> commit would fail. >> + >> + * Consider splitting parsed_route structure. When creating parsed routes >> + with tracked_port explicitly set, other members of this structure are >> + usually unused/default. A new structure dedicated to routes with >> + explicitly defined tracked_port would be more efficient. >> + More details in >> + >> https://mail.openvswitch.org/pipermail/ovs-dev/2025-February/420985.html >> diff --git a/lib/stopwatch-names.h b/lib/stopwatch-names.h >> index c12dd28d0..13aa5e7bf 100644 >> --- a/lib/stopwatch-names.h >> +++ b/lib/stopwatch-names.h >> @@ -36,5 +36,6 @@ >> #define LS_STATEFUL_RUN_STOPWATCH_NAME "ls_stateful" >> #define ADVERTISED_ROUTE_SYNC_RUN_STOPWATCH_NAME "advertised_route_sync" >> #define LEARNED_ROUTE_SYNC_RUN_STOPWATCH_NAME "learned_route_sync" >> +#define DYNAMIC_ROUTES_RUN_STOPWATCH_NAME "dynamic_routes" This one is not used and should be. >> >> #endif >> diff --git a/northd/en-advertised-route-sync.c >> b/northd/en-advertised-route-sync.c >> index cfea7c39b..a984e70a8 100644 >> --- a/northd/en-advertised-route-sync.c >> +++ b/northd/en-advertised-route-sync.c >> @@ -30,7 +30,8 @@ advertised_route_table_sync( >> struct ovsdb_idl_txn *ovnsb_txn, >> const struct sbrec_advertised_route_table >> *sbrec_advertised_route_table, >> const struct lr_stateful_table *lr_stateful_table, >> - const struct hmap *parsed_routes, >> + const struct hmap *routes, >> + const struct hmap *dynamic_routes, >> struct advertised_route_sync_data *data); >> >> bool >> @@ -141,6 +142,8 @@ en_advertised_route_sync_run(struct engine_node *node, >> void *data OVS_UNUSED) >> struct advertised_route_sync_data *routes_sync_data = data; >> struct routes_data *routes_data >> = engine_get_input_data("routes", node); >> + struct dynamic_routes_data *dynamic_routes_data >> + = engine_get_input_data("dynamic_routes", node); >> const struct engine_context *eng_ctx = engine_get_context(); >> const struct sbrec_advertised_route_table >> *sbrec_advertised_route_table = >> EN_OVSDB_GET(engine_get_input("SB_advertised_route", node)); >> @@ -153,12 +156,75 @@ en_advertised_route_sync_run(struct engine_node >> *node, void *data OVS_UNUSED) >> sbrec_advertised_route_table, >> &lr_stateful_data->table, >> &routes_data->parsed_routes, >> + &dynamic_routes_data->parsed_routes, >> routes_sync_data); >> >> stopwatch_stop(ADVERTISED_ROUTE_SYNC_RUN_STOPWATCH_NAME, time_msec()); >> engine_set_node_state(node, EN_UPDATED); >> } >> >> +void >> +*en_dynamic_routes_init(struct engine_node *node OVS_UNUSED, Nit: * should be on the previous line. >> + struct engine_arg *arg OVS_UNUSED) Nit: indentation. >> +{ >> + struct dynamic_routes_data *data = xmalloc(sizeof *data); >> + *data = (struct dynamic_routes_data) { >> + .parsed_routes = HMAP_INITIALIZER(&data->parsed_routes), >> + }; >> + >> + return data; >> +} >> + >> +static void >> +en_dynamic_routes_clean(struct dynamic_routes_data *data) >> +{ >> + struct parsed_route *r; >> + HMAP_FOR_EACH_POP (r, key_node, &data->parsed_routes) { >> + parsed_route_free(r); >> + } >> +} >> +void >> +en_dynamic_routes_cleanup(void *data_) >> +{ >> + struct dynamic_routes_data *data = data_; >> + >> + en_dynamic_routes_clean(data); >> + hmap_destroy(&data->parsed_routes); >> +} >> + >> +void >> +en_dynamic_routes_run(struct engine_node *node, void *data) >> +{ >> + struct dynamic_routes_data *dynamic_routes_data = data; >> + struct northd_data *northd_data = engine_get_input_data("northd", >> node); >> + struct ed_type_lr_stateful *lr_stateful_data = >> + engine_get_input_data("lr_stateful", node); >> + >> + en_dynamic_routes_clean(data); >> + const struct lr_stateful_record *lr_stateful_rec; >> + HMAP_FOR_EACH (lr_stateful_rec, key_node, >> + &lr_stateful_data->table.entries) { >> + const struct ovn_datapath *od = >> + ovn_datapaths_find_by_index(&northd_data->lr_datapaths, >> + lr_stateful_rec->lr_index); >> + if (!od->dynamic_routing) { >> + continue; >> + } >> + build_nat_parsed_routes(od, lr_stateful_rec->lrnat_rec, >> + &northd_data->ls_ports, >> + &dynamic_routes_data->parsed_routes); >> + build_nat_connected_parsed_routes(od, &lr_stateful_data->table, >> + &northd_data->ls_ports, >> + >> &dynamic_routes_data->parsed_routes); >> + >> + build_lb_parsed_routes(od, lr_stateful_rec->lb_ips, >> + &dynamic_routes_data->parsed_routes); >> + build_lb_connected_parsed_routes(od, &lr_stateful_data->table, >> + >> &dynamic_routes_data->parsed_routes); >> + } >> + engine_set_node_state(node, EN_UPDATED); >> +} >> + >> struct ar_entry { >> struct hmap_node hmap_node; >> >> @@ -333,12 +399,89 @@ publish_host_routes(struct hmap *sync_routes, >> } >> } >> >> +static void >> +advertised_route_table_sync_route_add( >> + const struct lr_stateful_table *lr_stateful_table, >> + struct advertised_route_sync_data *data, >> + struct uuidset *host_route_lrps, >> + struct hmap *sync_routes, >> + const struct parsed_route *route) >> +{ >> + if (route->is_discard_route) { >> + return; >> + } >> + if (prefix_is_link_local(&route->prefix, route->plen)) { >> + return; >> + } >> + if (!route->od->dynamic_routing) { >> + return; >> + } >> + >> + enum dynamic_routing_redistribute_mode drr = >> + route->out_port->dynamic_routing_redistribute; >> + if (route->source == ROUTE_SOURCE_CONNECTED) { >> + if (!drr_mode_CONNECTED_is_set(drr)) { >> + return; >> + } >> + /* If we advertise host routes, we only need to do so once per >> + * LRP. */ >> + const struct uuid *lrp_uuid = >> &route->out_port->nbrp->header_.uuid; >> + if (drr_mode_CONNECTED_AS_HOST_is_set(drr) && >> + !uuidset_contains(host_route_lrps, lrp_uuid)) { >> + uuidset_insert(host_route_lrps, lrp_uuid); >> + publish_host_routes(sync_routes, lr_stateful_table, route, >> data); >> + return; >> + } >> + } >> + if (route->source == ROUTE_SOURCE_STATIC && >> !drr_mode_STATIC_is_set(drr)) { >> + return; >> + } >> + if (route->source == ROUTE_SOURCE_NAT) { >> + if (!drr_mode_NAT_is_set(drr)) { >> + return; >> + } >> + /* If NAT route tracks port on a different DP than the one that >> + * advertises the route, we need to watch for changes on that DP >> as >> + * well. */ >> + if (route->tracked_port && route->tracked_port->od != route->od) { >> + if (route->tracked_port->od->nbr) { >> + uuidset_insert(&data->nb_lr, >> + >> &route->tracked_port->od->nbr->header_.uuid); >> + } else if (route->tracked_port->od->nbs) { >> + uuidset_insert(&data->nb_ls, >> + >> &route->tracked_port->od->nbs->header_.uuid); >> + } >> + } >> + } >> + if (route->source == ROUTE_SOURCE_LB) { >> + if (!drr_mode_LB_is_set(drr)) { >> + return; >> + } >> + /* If LB route tracks port on a different DP than the one that >> + * advertises the route, we need to watch for changes on that DP >> as >> + * well. */ >> + if (route->tracked_port && route->tracked_port->od != route->od) { >> + uuidset_insert(&data->nb_lr, >> + &route->tracked_port->od->nbr->header_.uuid); >> + } >> + } >> + >> + char *ip_prefix = normalize_v46_prefix(&route->prefix, route->plen); >> + const struct sbrec_port_binding *tracked_port = NULL; >> + if (route->tracked_port) { >> + tracked_port = route->tracked_port->sb; >> + } >> + ar_add_entry(sync_routes, route->od->sb, route->out_port->sb, >> + ip_prefix, tracked_port); >> +} >> + >> static void >> advertised_route_table_sync( >> struct ovsdb_idl_txn *ovnsb_txn, >> const struct sbrec_advertised_route_table >> *sbrec_advertised_route_table, >> const struct lr_stateful_table *lr_stateful_table, >> - const struct hmap *parsed_routes, >> + const struct hmap *routes, >> + const struct hmap *dynamic_routes, >> struct advertised_route_sync_data *data) >> { >> struct hmap sync_routes = HMAP_INITIALIZER(&sync_routes); >> @@ -346,47 +489,24 @@ advertised_route_table_sync( >> const struct parsed_route *route; >> >> struct ar_entry *route_e; >> - const struct sbrec_advertised_route *sb_route; >> - HMAP_FOR_EACH (route, key_node, parsed_routes) { >> - if (route->is_discard_route) { >> - continue; >> - } >> - if (prefix_is_link_local(&route->prefix, route->plen)) { >> - continue; >> - } >> - if (!route->od->dynamic_routing) { >> - continue; >> - } >> >> - enum dynamic_routing_redistribute_mode drr = >> - route->out_port->dynamic_routing_redistribute; >> - if (route->source == ROUTE_SOURCE_CONNECTED) { >> - if (!drr_mode_CONNECTED_is_set(drr)) { >> - continue; >> - } >> - /* If we advertise host routes, we only need to do so once per >> - * LRP. */ >> - const struct uuid *lrp_uuid = >> - &route->out_port->nbrp->header_.uuid; >> - if (drr_mode_CONNECTED_AS_HOST_is_set(drr) && >> - !uuidset_contains(&host_route_lrps, lrp_uuid)) { >> - uuidset_insert(&host_route_lrps, lrp_uuid); >> - publish_host_routes(&sync_routes, lr_stateful_table, >> - route, data); >> - continue; >> - } >> - } >> - if (route->source == ROUTE_SOURCE_STATIC && >> - !drr_mode_STATIC_is_set(drr)) { >> - continue; >> - } >> - >> - char *ip_prefix = normalize_v46_prefix(&route->prefix, >> route->plen); >> - route_e = ar_add_entry(&sync_routes, route->od->sb, >> - route->out_port->sb, ip_prefix, NULL); >> + /* First build the set of non-dynamic routes that need sync-ing. */ >> + HMAP_FOR_EACH (route, key_node, routes) { >> + advertised_route_table_sync_route_add(lr_stateful_table, >> + data, &host_route_lrps, >> + &sync_routes, >> + route); >> + } >> + /* Then add the set of dynamic routes that need sync-ing. */ >> + HMAP_FOR_EACH (route, key_node, dynamic_routes) { >> + advertised_route_table_sync_route_add(lr_stateful_table, >> + data, &host_route_lrps, >> + &sync_routes, >> + route); >> } >> uuidset_destroy(&host_route_lrps); >> >> + const struct sbrec_advertised_route *sb_route; >> SBREC_ADVERTISED_ROUTE_TABLE_FOR_EACH_SAFE (sb_route, >> >> sbrec_advertised_route_table) { >> route_e = ar_find(&sync_routes, sb_route->datapath, >> diff --git a/northd/en-advertised-route-sync.h >> b/northd/en-advertised-route-sync.h >> index 1f24fd329..0290f08d2 100644 >> --- a/northd/en-advertised-route-sync.h >> +++ b/northd/en-advertised-route-sync.h >> @@ -36,4 +36,7 @@ void *en_advertised_route_sync_init(struct engine_node >> *, struct engine_arg *); >> void en_advertised_route_sync_cleanup(void *data); >> void en_advertised_route_sync_run(struct engine_node *, void *data); >> >> +void *en_dynamic_routes_init(struct engine_node *, struct engine_arg *); >> +void en_dynamic_routes_cleanup(void *data); >> +void en_dynamic_routes_run(struct engine_node *, void *data); >> #endif /* EN_ADVERTISED_ROUTE_SYNC_H */ >> diff --git a/northd/en-learned-route-sync.c >> b/northd/en-learned-route-sync.c >> index 406f1551f..4e87b3265 100644 >> --- a/northd/en-learned-route-sync.c >> +++ b/northd/en-learned-route-sync.c >> @@ -181,7 +181,8 @@ parse_route_from_sbrec_route(struct hmap >> *parsed_routes_out, >> >> parsed_route_add(od, nexthop, &prefix, plen, false, lrp_addr_s, >> out_port, 0, false, false, NULL, >> - ROUTE_SOURCE_LEARNED, &route->header_, >> parsed_routes_out); >> + ROUTE_SOURCE_LEARNED, &route->header_, NULL, >> + parsed_routes_out); >> } >> >> static void >> diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c >> index 438daf1c6..e94e98f78 100644 >> --- a/northd/inc-proc-northd.c >> +++ b/northd/inc-proc-northd.c >> @@ -175,6 +175,7 @@ static ENGINE_NODE(multicast_igmp, "multicast_igmp"); >> static ENGINE_NODE(acl_id, "acl_id"); >> static ENGINE_NODE(advertised_route_sync, "advertised_route_sync"); >> static ENGINE_NODE(learned_route_sync, "learned_route_sync"); >> +static ENGINE_NODE(dynamic_routes, "dynamic_routes"); >> >> void inc_proc_northd_init(struct ovsdb_idl_loop *nb, >> struct ovsdb_idl_loop *sb) >> @@ -289,7 +290,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, >> engine_add_input(&en_ecmp_nexthop, &en_sb_mac_binding, >> ecmp_nexthop_mac_binding_handler); >> >> + engine_add_input(&en_dynamic_routes, &en_lr_stateful, NULL); >> > + engine_add_input(&en_dynamic_routes, &en_northd, engine_noop_handler); >> > > We usually add a comment explaining why noop is fine. > Good point. I'm thinking of: /* No need for an explicit handler for northd changes. Stateful * configuration changes are passed through the en_lr_stateful input * dependency. We do need to access en_northd (input) data, i.e., to * lookup OVN ports. */ > >> + >> engine_add_input(&en_advertised_route_sync, &en_routes, NULL); >> + engine_add_input(&en_advertised_route_sync, &en_dynamic_routes, NULL); >> engine_add_input(&en_advertised_route_sync, &en_sb_advertised_route, >> NULL); >> engine_add_input(&en_advertised_route_sync, &en_lr_stateful, >> diff --git a/northd/northd.c b/northd/northd.c >> index 1d9f0675e..1558654a0 100644 >> --- a/northd/northd.c >> +++ b/northd/northd.c >> @@ -848,6 +848,14 @@ parse_dynamic_routing_redistribute( >> out |= DRRM_STATIC; >> continue; >> } >> + if (!strcmp(token, "nat")) { >> + out |= DRRM_NAT; >> + continue; >> + } >> + if (!strcmp(token, "lb")) { >> + out |= DRRM_LB; >> + continue; >> + } >> static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); >> VLOG_WARN_RL(&rl, >> "unkown dynamic-routing-redistribute option '%s' on >> %s", >> @@ -11000,6 +11008,7 @@ parsed_route_init(const struct ovn_datapath *od, >> bool ecmp_symmetric_reply, >> const struct sset *ecmp_selection_fields, >> enum route_source source, >> + const struct ovn_port *tracked_port, >> const struct ovsdb_idl_row *source_hint) >> { >> >> @@ -11015,6 +11024,7 @@ parsed_route_init(const struct ovn_datapath *od, >> new_pr->is_discard_route = is_discard_route; >> new_pr->lrp_addr_s = nullable_xstrdup(lrp_addr_s); >> new_pr->out_port = out_port; >> + new_pr->tracked_port = tracked_port; >> new_pr->source = source; >> if (ecmp_selection_fields) { >> sset_clone(&new_pr->ecmp_selection_fields, ecmp_selection_fields); >> @@ -11040,7 +11050,7 @@ parsed_route_clone(const struct parsed_route *pr) >> pr->od, nexthop, pr->prefix, pr->plen, pr->is_discard_route, >> pr->lrp_addr_s, pr->out_port, pr->route_table_id, >> pr->is_src_route, >> pr->ecmp_symmetric_reply, &pr->ecmp_selection_fields, pr->source, >> - pr->source_hint); >> + pr->tracked_port, pr->source_hint); >> >> new_pr->hash = pr->hash; >> return new_pr; >> @@ -11083,13 +11093,14 @@ parsed_route_add(const struct ovn_datapath *od, >> const struct sset *ecmp_selection_fields, >> enum route_source source, >> const struct ovsdb_idl_row *source_hint, >> + const struct ovn_port *tracked_port, >> struct hmap *routes) >> { >> >> struct parsed_route *new_pr = parsed_route_init( >> od, nexthop, *prefix, plen, is_discard_route, lrp_addr_s, >> out_port, >> route_table_id, is_src_route, ecmp_symmetric_reply, >> - ecmp_selection_fields, source, source_hint); >> + ecmp_selection_fields, source, tracked_port, source_hint); >> >> new_pr->hash = route_hash(new_pr); >> >> @@ -11226,7 +11237,7 @@ parsed_routes_add_static(const struct ovn_datapath >> *od, >> parsed_route_add(od, nexthop, &prefix, plen, is_discard_route, >> lrp_addr_s, >> out_port, route_table_id, is_src_route, >> ecmp_symmetric_reply, &ecmp_selection_fields, source, >> - &route->header_, routes); >> + &route->header_, NULL, routes); >> sset_destroy(&ecmp_selection_fields); >> } >> >> @@ -11244,7 +11255,7 @@ parsed_routes_add_connected(const struct >> ovn_datapath *od, >> false, addr->addr_s, op, >> 0, false, >> false, NULL, ROUTE_SOURCE_CONNECTED, >> - &op->nbrp->header_, routes); >> + &op->nbrp->header_, NULL, routes); >> } >> >> for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { >> @@ -11256,7 +11267,7 @@ parsed_routes_add_connected(const struct >> ovn_datapath *od, >> false, addr->addr_s, op, >> 0, false, >> false, NULL, ROUTE_SOURCE_CONNECTED, >> - &op->nbrp->header_, routes); >> + &op->nbrp->header_, NULL, routes); >> } >> } >> >> @@ -11294,6 +11305,237 @@ build_parsed_routes(const struct ovn_datapath >> *od, const struct hmap *lr_ports, >> } >> } >> >> +/* This function adds new parsed route for each entry in lr_nat record >> + * to "routes". Logical port of the route is set to "advertising_op" and >> + * tracked port is set to NAT's distributed gw port. If NAT doesn't have >> + * DGP (for example if it's set on gateway router), no tracked port will >> + * be set.*/ >> +static void >> +build_nat_parsed_route_for_port(const struct ovn_port *advertising_op, >> + const struct lr_nat_record *lr_nat, >> + const struct hmap *ls_ports, >> + struct hmap *routes) >> +{ >> + const struct ovn_datapath *advertising_od = advertising_op->od; >> + >> + for (size_t i = 0; i < lr_nat->n_nat_entries; i++) { >> + const struct ovn_nat *nat = &lr_nat->nat_entries[i]; >> + int plen = nat_entry_is_v6(nat) ? 128 : 32; >> + struct in6_addr prefix; >> + ip46_parse(nat->nb->external_ip, &prefix); >> + >> + const struct ovn_port *tracked_port = >> + nat->is_distributed && nat->is_valid >> + ? ovn_port_find(ls_ports, nat->nb->logical_port) >> + : nat->l3dgw_port; >> > > Why do we still advertise NAT entries if they're not valid? There won't be > any flow installed for invalid NAT. IMO we should skip invalid completely. > +1 this is a bug. I can change it to check validity first and continue if not valid. > >> + parsed_route_add(advertising_od, NULL, &prefix, plen, false, >> + nat->nb->external_ip, advertising_op, 0, false, >> + false, NULL, ROUTE_SOURCE_NAT, &nat->nb->header_, >> + tracked_port, routes); >> + } >> +} >> + >> +/* Generate parsed routes for NAT external IPs in lr_nat, for each ovn >> port >> + * in "od" that has enabled redistribution of NAT adresses.*/ >> +void >> +build_nat_parsed_routes(const struct ovn_datapath *od, >> + const struct lr_nat_record *lr_nat, >> + const struct hmap *ls_ports, >> + struct hmap *routes) >> +{ >> + const struct ovn_port *op; >> + HMAP_FOR_EACH (op, dp_node, &od->ports) { >> + if (!drr_mode_NAT_is_set(op->dynamic_routing_redistribute)) { >> + continue; >> + } >> + >> + build_nat_parsed_route_for_port(op, lr_nat, ls_ports, routes); >> + } >> +} >> + >> +/* Similar to build_nat_parsed_routes, this function generates parsed >> routes >> + * for nat records in neighboring routers. For each ovn port in "od" that >> has >> + * enabled redistribution of NAT adresses, look up their neighbors (either >> + * directly routers, or routers connected through common LS) and advertise >> + * thier external NAT IPs too.*/ >> +void >> +build_nat_connected_parsed_routes( >> + const struct ovn_datapath *od, >> + const struct lr_stateful_table *lr_stateful_table, >> + const struct hmap *ls_ports, >> + struct hmap *routes) >> +{ >> + const struct ovn_port *op; >> + HMAP_FOR_EACH (op, dp_node, &od->ports) { >> + if (!drr_mode_NAT_is_set(op->dynamic_routing_redistribute)) { >> + continue; >> + } >> + >> + if (!op->peer) { >> + continue; >> + } >> + >> + struct ovn_datapath *peer_od = op->peer->od; >> + if (!peer_od->nbs && !peer_od->nbr) { >> + continue; >> + } >> > > This shouldn't be possible e.g. the datapath will always be either router > or switch. > True, changing it into an assertion. > >> + >> + const struct ovn_port *peer_port = NULL; >> + /* This is a directly connected LR peer. */ >> + if (peer_od->nbr) { >> + peer_port = op->peer; >> + >> + const struct lr_stateful_record *peer_lr_stateful = >> + lr_stateful_table_find_by_index(lr_stateful_table, >> + peer_od->index); >> + if (!peer_lr_stateful) { >> + continue; >> + } >> + >> + /* Advertise peer's NAT routes via the local port too. */ >> + build_nat_parsed_route_for_port(op, >> peer_lr_stateful->lrnat_rec, >> + ls_ports, routes); >> + return; >> + } >> + >> + /* This peer is LSP, we need to check all connected router ports >> + * for NAT.*/ >> + for (size_t i = 0; i < peer_od->n_router_ports; i++) { >> + peer_port = peer_od->router_ports[i]->peer; >> + if (peer_port == op) { >> + /* Skip advertising router. */ >> + continue; >> + } >> + >> + const struct lr_stateful_record *peer_lr_stateful = >> + lr_stateful_table_find_by_index(lr_stateful_table, >> + peer_port->od->index); >> + if (!peer_lr_stateful) { >> + continue; >> + } >> + >> + /* Advertise peer's NAT routes via the local port too. */ >> + build_nat_parsed_route_for_port(op, >> peer_lr_stateful->lrnat_rec, >> + ls_ports, routes); >> + } >> + } >> +} >> + >> +/* This function adds new parsed route for each IP in lb_ips to >> "routes".*/ >> +static void >> +build_lb_parsed_route_for_port(const struct ovn_port *advertising_op, >> + const struct ovn_port *tracked_port, >> + const struct ovn_lb_ip_set *lb_ips, >> + struct hmap *routes) >> +{ >> + const struct ovn_datapath *advertising_od = advertising_op->od; >> + >> + const char *ip_address; >> + SSET_FOR_EACH (ip_address, &lb_ips->ips_v4) { >> + struct in6_addr prefix; >> + ip46_parse(ip_address, &prefix); >> + parsed_route_add(advertising_od, NULL, &prefix, 32, false, >> + ip_address, advertising_op, 0, false, false, >> + NULL, ROUTE_SOURCE_LB, >> &advertising_op->nbrp->header_, >> + tracked_port, routes); >> + } >> + SSET_FOR_EACH (ip_address, &lb_ips->ips_v6) { >> + struct in6_addr prefix; >> + ip46_parse(ip_address, &prefix); >> + parsed_route_add(advertising_od, NULL, &prefix, 128, false, >> + ip_address, advertising_op, 0, false, false, >> + NULL, ROUTE_SOURCE_LB, >> &advertising_op->nbrp->header_, >> + tracked_port, routes); >> + } >> +} >> + >> +/* Similar to build_lb_parsed_routes, this function generates parsed >> routes >> + * for LB VIPs of neighboring routers. For each ovn port in "od" that has >> + * enabled redistribution of LB VIPs, look up their neighbors (either >> + * directly routers, or routers connected through common LS) and advertise >> + * thier LB VIPs too.*/ >> +void >> +build_lb_connected_parsed_routes( >> + const struct ovn_datapath *od, >> + const struct lr_stateful_table *lr_stateful_table, >> + struct hmap *routes) >> +{ >> + const struct ovn_port *op; >> + HMAP_FOR_EACH (op, dp_node, &od->ports) { >> + if (!drr_mode_LB_is_set(op->dynamic_routing_redistribute)) { >> + continue; >> + } >> + >> + if (!op->peer) { >> + continue; >> + } >> + >> + struct ovn_datapath *peer_od = op->peer->od; >> + if (!peer_od->nbs && !peer_od->nbr) { >> + continue; >> + } >> + >> + const struct lr_stateful_record *lr_stateful_rec; >> + const struct ovn_port *peer_port = NULL; >> + /* This is directly connected LR peer. */ >> + if (peer_od->nbr) { >> + lr_stateful_rec = lr_stateful_table_find_by_index( >> + lr_stateful_table, peer_od->index); >> + peer_port = op->peer; >> + build_lb_parsed_route_for_port(op, peer_port, >> + lr_stateful_rec->lb_ips, >> routes); >> + return; >> + } >> + >> + /* This peer is LSP, we need to check all connected router ports >> for >> + * LBs.*/ >> + for (size_t i = 0; i < peer_od->n_router_ports; i++) { >> + peer_port = peer_od->router_ports[i]->peer; >> + if (peer_port == op) { >> + /* no need to check for LBs on ovn_port that initiated >> this >> + * function.*/ >> + continue; >> + } >> + lr_stateful_rec = lr_stateful_table_find_by_index( >> + lr_stateful_table, peer_port->od->index); >> + >> + build_lb_parsed_route_for_port(op, peer_port, >> + lr_stateful_rec->lb_ips, >> routes); >> + } >> + } >> +} >> + >> +void >> +build_lb_parsed_routes(const struct ovn_datapath *od, >> + const struct ovn_lb_ip_set *lb_ips, >> + struct hmap *routes) >> +{ >> + const struct ovn_port *op; >> + HMAP_FOR_EACH (op, dp_node, &od->ports) { >> + if (!drr_mode_LB_is_set(op->dynamic_routing_redistribute)) { >> + continue; >> + } >> + >> + /* Traffic processed by a load balancer is: >> + * - handled by the chassis where a gateway router is bound >> + * OR >> + * - always redirected to a distributed gateway router port >> + * >> + * Advertise the LB IPs via all 'op' if this is a gateway router >> or >> + * throuh all DGPs of this distributed router otherwise. */ >> + struct ovn_port *op_ = NULL; >> + size_t n_tracked_ports = !od->is_gw_router ? od->n_l3dgw_ports : >> 1; >> + struct ovn_port **tracked_ports = !od->is_gw_router >> + ? od->l3dgw_ports >> + : &op_; >> + >> + for (size_t i = 0; i < n_tracked_ports; i++) { >> + build_lb_parsed_route_for_port(op, tracked_ports[i], lb_ips, >> + routes); >> + } >> + } >> + >> +} >> struct ecmp_route_list_node { >> struct ovs_list list_node; >> uint16_t id; /* starts from 1 */ >> @@ -11468,6 +11710,8 @@ route_source_to_offset(enum route_source source) >> return ROUTE_PRIO_OFFSET_STATIC; >> case ROUTE_SOURCE_LEARNED: >> return ROUTE_PRIO_OFFSET_LEARNED; >> + case ROUTE_SOURCE_NAT: >> + case ROUTE_SOURCE_LB: >> default: >> OVS_NOT_REACHED(); >> } >> diff --git a/northd/northd.h b/northd/northd.h >> index b984e124d..1bb0cbd6f 100644 >> --- a/northd/northd.h >> +++ b/northd/northd.h >> @@ -186,11 +186,15 @@ struct route_policy { >> }; >> >> struct routes_data { >> - struct hmap parsed_routes; >> + struct hmap parsed_routes; /* Stores struct parsed_route. */ >> struct simap route_tables; >> struct hmap bfd_active_connections; >> }; >> >> +struct dynamic_routes_data { >> + struct hmap parsed_routes; /* Stores struct parsed_route. */ >> +}; >> + >> struct route_policies_data { >> struct hmap route_policies; >> struct hmap bfd_active_connections; >> @@ -308,10 +312,12 @@ struct mcast_port_info { >> * (e.g., IGMP join/leave). */ >> }; >> >> -#define DRR_MODES \ >> - DRR_MODE(CONNECTED, 0) \ >> +#define DRR_MODES \ >> + DRR_MODE(CONNECTED, 0) \ >> DRR_MODE(CONNECTED_AS_HOST, 1) \ >> - DRR_MODE(STATIC, 2) >> + DRR_MODE(STATIC, 2) \ >> + DRR_MODE(NAT, 3) \ >> + DRR_MODE(LB, 4) >> >> enum dynamic_routing_redistribute_mode_bits { >> #define DRR_MODE(PROTOCOL, BIT) DRRM_##PROTOCOL##_BIT = BIT, >> @@ -746,6 +752,10 @@ enum route_source { >> ROUTE_SOURCE_STATIC, >> /* The route is dynamically learned by an ovn-controller. */ >> ROUTE_SOURCE_LEARNED, >> + /* The route is derived from a NAT's external IP. */ >> + ROUTE_SOURCE_NAT, >> + /* The route is derived from a LB's VIP. */ >> + ROUTE_SOURCE_LB, >> }; >> >> struct parsed_route { >> @@ -765,6 +775,7 @@ struct parsed_route { >> const struct ovsdb_idl_row *source_hint; >> char *lrp_addr_s; >> const struct ovn_port *out_port; >> + const struct ovn_port *tracked_port; /* May be NULL. */ >> }; >> >> struct parsed_route *parsed_route_clone(const struct parsed_route *); >> @@ -784,6 +795,7 @@ void parsed_route_add(const struct ovn_datapath *od, >> const struct sset *ecmp_selection_fields, >> enum route_source source, >> const struct ovsdb_idl_row *source_hint, >> + const struct ovn_port *tracked_port, >> struct hmap *routes); >> >> bool >> @@ -818,6 +830,20 @@ void route_policies_destroy(struct >> route_policies_data *); >> void build_parsed_routes(const struct ovn_datapath *, const struct hmap *, >> const struct hmap *, struct hmap *, struct simap >> *, >> struct hmap *); >> +void build_nat_parsed_routes(const struct ovn_datapath *, >> + const struct lr_nat_record *, >> + const struct hmap *ls_ports, >> + struct hmap *routes); >> +void build_nat_connected_parsed_routes(const struct ovn_datapath *, >> + const struct lr_stateful_table *, >> + const struct hmap *ls_ports, >> + struct hmap *routes); >> +void build_lb_parsed_routes(const struct ovn_datapath *, >> + const struct ovn_lb_ip_set *, >> + struct hmap *); >> +void build_lb_connected_parsed_routes(const struct ovn_datapath *, >> + const struct lr_stateful_table *, >> + struct hmap *routes); >> uint32_t get_route_table_id(struct simap *, const char *); >> void routes_init(struct routes_data *); >> void routes_destroy(struct routes_data *); >> diff --git a/ovn-nb.xml b/ovn-nb.xml >> index 20d30dd58..f10c1e738 100644 >> --- a/ovn-nb.xml >> +++ b/ovn-nb.xml >> @@ -3101,6 +3101,24 @@ or >> <ref table="Advertised_Route" db="OVN_Southbound"/> table. >> </p> >> >> + <p> >> + If <code>lb</code> is in the list then northd will create >> entries in >> + <ref table="Advertised_Route" db="OVN_Southbound"/> table for >> each >> + Load Balancer VIP on this router and it's neighboring routers. >> + Neighboring routers are those that are either directly >> connected, >> + via Logical Router Port, or those that are connected via shared >> + Logical Switch. >> + </p> >> + >> + <p> >> + If <code>nat</code> is in the list then northd will create >> entries in >> + <ref table="Advertised_Route" db="OVN_Southbound"/> table for >> each >> + NAT's external IP on this router and it's neighboring routers. >> + Neighboring routers are those that are either directly >> connected, >> + via Logical Router Port, or those that are connected via shared >> + Logical Switch. >> + </p> >> + >> <p> >> This value can be overwritten on a per LRP basis using >> <ref column="options" key="dynamic-routing-redistribute" >> @@ -3950,6 +3968,24 @@ or >> <ref table="Advertised_Route" db="OVN_Southbound"/> table. >> </p> >> >> + <p> >> + If <code>lb</code> is in the list then northd will create >> entries in >> + <ref table="Advertised_Route" db="OVN_Southbound"/> table for >> each >> + Load Balancer VIP on this port's router, and it's neighboring >> + routers. Neighboring routers are those that are either directly >> + connected to this Logical Router Port, or those that are >> connected >> + via shared Logical Switch. >> + </p> >> + >> + <p> >> + If <code>nat</code> is in the list then northd will create >> entries in >> + <ref table="Advertised_Route" db="OVN_Southbound"/> table for >> each >> + NAT's external IP on this port's router, and it's neighboring >> + routers. Neighboring routers are those that are either directly >> + connected to this Logical Router Port, or those that are >> connected >> + via shared Logical Switch. >> + </p> >> + >> <p> >> If not set the value from <ref column="options" >> key="dynamic-routing-redistribute" table="Logical_Router"/> on >> the >> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at >> index 64991ff75..a18321818 100644 >> --- a/tests/ovn-northd.at >> +++ b/tests/ovn-northd.at >> @@ -15542,3 +15542,631 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE >> >> AT_CLEANUP >> ]) >> + >> +OVN_FOR_EACH_NORTHD_NO_HV([ >> +AT_SETUP([dynamic-routing - nat sync to sb IPv4]) >> +AT_KEYWORDS([dynamic-routing]) >> +ovn_start >> + >> +# Start with GW router and a single LRP. >> +check ovn-nbctl lr-add lr0 >> +check ovn-nbctl set Logical_Router lr0 \ >> + options:dynamic-routing=true \ >> + options:chassis=hv1 >> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 >> +check ovn-nbctl --wait=sb sync >> + >> +check_row_count Advertised_Route 0 >> + >> +datapath=$(fetch_column Datapath_Binding _uuid external_ids:name=lr0) >> +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-sw0) >> + >> +# Adding LRP dynamic-routing-redistribute=nat option and NAT rule >> advertises a >> +# route entry. >> +check ovn-nbctl \ >> + -- lrp-set-options lr0-sw0 dynamic-routing-redistribute=nat \ >> + -- lr-nat-add lr0 dnat_and_snat 172.16.1.10 192.168.1.10 >> +check ovn-nbctl --wait=sb sync >> + >> +check_row_count Advertised_Route 1 >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="172.16.1.10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=[[]] >> + >> +# Add LR with distributed LRP connected to GW router through join LS >> +# and switch dynamic-routing-redistribute to lrp-guest-join so that >> +# it advertises NAT routes from the neighbors in the join LS. >> +check ovn-nbctl \ >> + -- remove Logical_Router_Port lr0-sw0 options >> dynamic-routing-redistribute \ >> + -- lrp-add lr0 lr0-join 00:00:00:00:ff:02 10.42.0.1/24 \ >> + -- lrp-set-options lr0-join dynamic-routing-redistribute=nat \ >> + -- ls-add ls-join \ >> + -- lsp-add ls-join lsp-join-to-lr0 \ >> + -- lsp-set-type lsp-join-to-lr0 router \ >> + -- lsp-set-options lsp-join-to-lr0 router-port=lr0-join \ >> + -- lsp-set-addresses lsp-join-to-lr0 router \ >> + -- lr-add lr-guest0 \ >> + -- lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 10.51.0.1/24 \ >> + -- lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 10.42.0.2/24 \ >> + -- lsp-add ls-join lsp-join-to-guest0 \ >> + -- lsp-set-type lsp-join-to-guest0 router \ >> + -- lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \ >> + -- lrp-set-gateway-chassis lrp-guest0-join hv1 >> +check ovn-nbctl --wait=sb sync >> + >> +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-join) >> +pb2=$(fetch_column Port_Binding _uuid logical_port=lrp-guest0-join) >> + >> +check ovn-nbctl \ >> + --add-route lr-nat-add lr-guest0 dnat_and_snat 172.16.2.10 >> 192.168.2.10 >> +check ovn-nbctl --wait=sb sync >> + >> +check_row_count Advertised_Route 2 >> + >> +# Routes to local NAT addresses are advertised without tracked port. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="172.16.1.10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=[[]] >> + >> +# Routes to NAT addresses of neighboring routers are advertised with >> +# tracked port pointing to the LRP of the neighboring router. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="172.16.2.10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=$pb2 >> + >> +# Add nonlocal LR with distributed LRP connected to GW router through >> join LS. >> +check ovn-nbctl \ >> + -- lr-add lr-guest1 \ >> + -- lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 10.51.1.1/24 \ >> + -- lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 10.42.0.3/24 \ >> + -- lsp-add ls-join lsp-join-to-guest1 \ >> + -- lsp-set-type lsp-join-to-guest1 router \ >> + -- lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \ >> + -- lrp-set-gateway-chassis lrp-guest1-join nonlocalhv >> +check ovn-nbctl --wait=sb sync >> + >> +pb3=$(fetch_column Port_Binding _uuid logical_port=lrp-guest1-join) >> + >> +check ovn-nbctl \ >> + -- --add-route lr-nat-add lr-guest1 dnat_and_snat 172.16.3.10 >> 192.168.3.10 >> +check ovn-nbctl --wait=sb sync >> + >> +# Add a distributed router with distributed NAT connected to a different >> +# public LS. >> +check ovn-nbctl \ >> + -- ls-add ls-pub \ >> + -- lsp-add ls-pub ls-pub-ln \ >> + -- lsp-set-type ls-pub-ln localnet \ >> + -- lsp-set-addresses ls-pub-ln unknown \ >> + -- lsp-set-options ls-pub-ln network_name=public \ >> + -- lr-add dr \ >> + -- set logical_router dr options:dynamic-routing=true \ >> + -- lrp-add dr lrp-dr-sw1 00:00:00:00:fd:01 10.52.1.1/24 \ >> + -- lrp-add dr lrp-dr-pub 00:00:00:00:fd:02 10.42.0.4/24 \ >> + -- lrp-set-options lrp-dr-pub dynamic-routing-redistribute=nat \ >> + -- lsp-add ls-pub lsp-pub-to-guest2 \ >> + -- lsp-set-type lsp-pub-to-guest2 router \ >> + -- lsp-set-options lsp-pub-to-guest2 router-port=lrp-dr-pub \ >> + -- lrp-set-gateway-chassis lrp-dr-pub nonlocalhv \ >> + -- ls-add sw1 \ >> + -- lsp-add sw1 sw1-lrp-dr \ >> + -- lsp-set-type sw1-lrp-dr router \ >> + -- lsp-set-addresses sw1-lrp-dr router \ >> + -- lsp-set-options sw1-lrp-dr router-port=lrp-dr-sw1 \ >> + -- lsp-add sw1 vm >> +check ovn-nbctl --wait=sb sync >> + >> +datapath_dr=$(fetch_column Datapath_Binding _uuid external_ids:name=dr) >> +pb_dr=$(fetch_column Port_Binding _uuid logical_port=lrp-dr-pub) >> +pb_vm=$(fetch_column Port_Binding _uuid logical_port=vm) >> + >> +check ovn-nbctl \ >> + -- --add-route lr-nat-add dr dnat_and_snat 172.16.4.10 192.168.3.20 \ >> + vm 00:00:00:00:fd:03 >> +check ovn-nbctl --wait=sb sync >> + >> +check_row_count Advertised_Route 4 >> + >> +# Routes to local NAT addresses are advertised without tracked port. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="172.16.1.10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=[[]] >> + >> +# Routes to NAT addresses of neighboring routers are advertised with >> +# tracked port pointing to the LRP of the neighboring router. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="172.16.2.10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=$pb2 >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="172.16.3.10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=$pb3 >> + >> +# Routes to distributed NAT addresses are advertised with tracked port set >> +# to the logical_port of the NAT entry. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="172.16.4.10" \ >> + datapath=$datapath_dr \ >> + logical_port=$pb_dr \ >> + tracked_port=$pb_vm >> + >> +# Removing the option:dynamic-routing removes all routes. >> +check ovn-nbctl --wait=sb remove Logical_Router lr0 option dynamic-routing >> +check_row_count Advertised_Route 1 >> + >> +check ovn-nbctl --wait=sb remove Logical_Router dr option dynamic-routing >> +check_row_count Advertised_Route 0 >> + >> +# And setting it again adds them again. >> +check ovn-nbctl --wait=sb set Logical_Router lr0 >> option:dynamic-routing=true >> +check_row_count Advertised_Route 3 >> + >> +check ovn-nbctl --wait=sb set Logical_Router dr >> option:dynamic-routing=true >> +check_row_count Advertised_Route 4 >> + >> +# Removing the lr will remove all routes. >> +check ovn-nbctl lr-del lr0 -- lr-del dr >> +check ovn-nbctl --wait=sb sync >> +check_row_count Advertised_Route 0 >> + >> +AT_CLEANUP >> +]) >> + >> +OVN_FOR_EACH_NORTHD_NO_HV([ >> +AT_SETUP([dynamic-routing - nat sync to sb IPv6]) >> +AT_KEYWORDS([dynamic-routing]) >> +ovn_start >> + >> +# Start with GW router and a single LRP. >> +check ovn-nbctl lr-add lr0 >> +check ovn-nbctl set Logical_Router lr0 \ >> + options:dynamic-routing=true \ >> + options:chassis=hv1 >> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 2001:db0::1/64 >> +check ovn-nbctl --wait=sb sync >> + >> +check_row_count Advertised_Route 0 >> + >> +datapath=$(fetch_column Datapath_Binding _uuid external_ids:name=lr0) >> +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-sw0) >> + >> +# Adding LRP dynamic-routing-redistribute=nat option and NAT rule >> advertises >> +# a route entry. >> +check ovn-nbctl \ >> + -- lrp-set-options lr0-sw0 dynamic-routing-redistribute=nat \ >> + -- lr-nat-add lr0 dnat_and_snat 2001:db1::10 2001:db2::10 >> +check ovn-nbctl --wait=sb sync >> + >> +check_row_count Advertised_Route 1 >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="2001\:db1\:\:10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=[[]] >> + >> +# Add LR with distributed LRP connected to GW router through join LS >> +# and switch dynamic-routing-redistribute to lrp-guest-join so that >> +# it advertises NAT routes from the neighbors in the join LS. >> +check ovn-nbctl \ >> + -- remove Logical_Router_Port lr0-sw0 options >> dynamic-routing-redistribute \ >> + -- lrp-add lr0 lr0-join 00:00:00:00:ff:02 2001:db42::1/64 \ >> + -- lrp-set-options lr0-join dynamic-routing-redistribute=nat \ >> + -- ls-add ls-join \ >> + -- lsp-add ls-join lsp-join-to-lr0 \ >> + -- lsp-set-type lsp-join-to-lr0 router \ >> + -- lsp-set-options lsp-join-to-lr0 router-port=lr0-join \ >> + -- lsp-set-addresses lsp-join-to-lr0 router \ >> + -- lr-add lr-guest0 \ >> + -- lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 2001:db51::1/64 >> \ >> + -- lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 >> 2001:db42::2/64 \ >> + -- lsp-add ls-join lsp-join-to-guest0 \ >> + -- lsp-set-type lsp-join-to-guest0 router \ >> + -- lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \ >> + -- lrp-set-gateway-chassis lrp-guest0-join hv1 >> +check ovn-nbctl --wait=sb sync >> + >> +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-join) >> +pb2=$(fetch_column Port_Binding _uuid logical_port=lrp-guest0-join) >> + >> +check ovn-nbctl \ >> + --add-route lr-nat-add lr-guest0 dnat_and_snat 2001:db3::10 >> 2001:db4::10 >> +check ovn-nbctl --wait=sb sync >> + >> +check_row_count Advertised_Route 2 >> + >> +# Routes to local NAT addresses are advertised without tracked port. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="2001\:db1\:\:10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=[[]] >> + >> +# Routes to NAT addresses of neighboring routers are advertised with >> +# tracked port pointing to the LRP of the neighboring router. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="2001\:db3\:\:10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=$pb2 >> + >> +# Add nonlocal LR with distributed LRP connected to GW router through >> join LS. >> +check ovn-nbctl \ >> + -- lr-add lr-guest1 \ >> + -- lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 2001:db52::1/64 >> \ >> + -- lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 >> 2001:db42::3/64 \ >> + -- lsp-add ls-join lsp-join-to-guest1 \ >> + -- lsp-set-type lsp-join-to-guest1 router \ >> + -- lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \ >> + -- lrp-set-gateway-chassis lrp-guest1-join nonlocalhv >> +check ovn-nbctl --wait=sb sync >> + >> +pb3=$(fetch_column Port_Binding _uuid logical_port=lrp-guest1-join) >> + >> +check ovn-nbctl \ >> + --add-route lr-nat-add lr-guest1 dnat_and_snat 2001:db5::10 >> 2001:db6::10 >> +check ovn-nbctl --wait=sb sync >> + >> +# Add a distributed router with distributed NAT connected to a different >> +# public LS. >> +check ovn-nbctl \ >> + -- ls-add ls-pub \ >> + -- lsp-add ls-pub ls-pub-ln \ >> + -- lsp-set-type ls-pub-ln localnet \ >> + -- lsp-set-addresses ls-pub-ln unknown \ >> + -- lsp-set-options ls-pub-ln network_name=public \ >> + -- lr-add dr \ >> + -- set logical_router dr options:dynamic-routing=true \ >> + -- lrp-add dr lrp-dr-sw1 00:00:00:00:fd:01 2001:db53::1/64 \ >> + -- lrp-add dr lrp-dr-pub 00:00:00:00:fd:02 2001:db42::4/64 \ >> + -- lrp-set-options lrp-dr-pub dynamic-routing-redistribute=nat \ >> + -- lsp-add ls-pub lsp-pub-to-guest2 \ >> + -- lsp-set-type lsp-pub-to-guest2 router \ >> + -- lsp-set-options lsp-pub-to-guest2 router-port=lrp-dr-pub \ >> + -- lrp-set-gateway-chassis lrp-dr-pub nonlocalhv \ >> + -- ls-add sw1 \ >> + -- lsp-add sw1 sw1-lrp-dr \ >> + -- lsp-set-type sw1-lrp-dr router \ >> + -- lsp-set-addresses sw1-lrp-dr router \ >> + -- lsp-set-options sw1-lrp-dr router-port=lrp-dr-sw1 \ >> + -- lsp-add sw1 vm >> +check ovn-nbctl --wait=sb sync >> + >> +datapath_dr=$(fetch_column Datapath_Binding _uuid external_ids:name=dr) >> +pb_dr=$(fetch_column Port_Binding _uuid logical_port=lrp-dr-pub) >> +pb_vm=$(fetch_column Port_Binding _uuid logical_port=vm) >> + >> +check ovn-nbctl \ >> + -- --add-route lr-nat-add dr dnat_and_snat 2001:db6::10 2001:db6::20 \ >> + vm 00:00:00:00:fd:03 >> +check ovn-nbctl --wait=sb sync >> + >> +check_row_count Advertised_Route 4 >> + >> +# Routes to local NAT addresses are advertised without tracked port. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="2001\:db1\:\:10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=[[]] >> + >> +# Routes to NAT addresses of neighboring routers are advertised with >> +# tracked port pointing to the LRP of the neighboring router. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="2001\:db3\:\:10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=$pb2 >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="2001\:db5\:\:10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=$pb3 >> + >> +# Routes to distributed NAT addresses are advertised with tracked port set >> +# to the logical_port of the NAT entry. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="2001\:db6\:\:10" \ >> + datapath=$datapath_dr \ >> + logical_port=$pb_dr \ >> + tracked_port=$pb_vm >> + >> +# Removing the option:dynamic-routing removes all routes. >> +check ovn-nbctl --wait=sb remove Logical_Router lr0 option dynamic-routing >> +check_row_count Advertised_Route 1 >> + >> +check ovn-nbctl --wait=sb remove Logical_Router dr option dynamic-routing >> +check_row_count Advertised_Route 0 >> + >> +# And setting it again adds them again. >> +check ovn-nbctl --wait=sb set Logical_Router lr0 >> option:dynamic-routing=true >> +check_row_count Advertised_Route 3 >> + >> +check ovn-nbctl --wait=sb set Logical_Router dr >> option:dynamic-routing=true >> +check_row_count Advertised_Route 4 >> + >> +# Removing the lr will remove all routes. >> +check ovn-nbctl lr-del lr0 -- lr-del dr >> +check ovn-nbctl --wait=sb sync >> +check_row_count Advertised_Route 0 >> + >> +AT_CLEANUP >> +]) >> + >> +OVN_FOR_EACH_NORTHD_NO_HV([ >> +AT_SETUP([dynamic-routing - LB sync to sb IPv4]) >> +AT_KEYWORDS([dynamic-routing]) >> +ovn_start >> + >> +# Start with GW router and a single LRP, >> +check ovn-nbctl lr-add lr0 >> +check ovn-nbctl set Logical_Router lr0 \ >> + options:dynamic-routing=true \ >> + options:chassis=hv1 >> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 >> +check ovn-nbctl --wait=sb sync >> + >> +check_row_count Advertised_Route 0 >> + >> +datapath=$(fetch_column Datapath_Binding _uuid external_ids:name=lr0) >> +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-sw0) >> + >> +# Adding LRP dynamic-routing-lb-vips option and LB VIP rule advertises >> +# a route entry. >> +check ovn-nbctl lrp-set-options lr0-sw0 dynamic-routing-redistribute=lb \ >> + -- lb-add lb0 172.16.1.10:80 192.168.1.10:80,192.168.1.11:80 \ >> + -- lr-lb-add lr0 lb0 >> +check ovn-nbctl --wait=sb sync >> + >> +check_row_count Advertised_Route 1 >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="172.16.1.10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=[[]] >> + >> +# Add LR with distributed LRP connected to GW router through join LS >> +# and switch dynamic-routing-redistribute to lrp-guest-join so that >> +# it advertises LB VIP routes from the neighbors in the join LS. >> +check ovn-nbctl \ >> + -- remove Logical_Router_Port lr0-sw0 options >> dynamic-routing-redistribute \ >> + -- lrp-add lr0 lr0-join 00:00:00:00:ff:02 10.42.0.1/24 \ >> + -- lrp-set-options lr0-join dynamic-routing-redistribute=lb \ >> + -- ls-add ls-join \ >> + -- lsp-add ls-join lsp-join-to-lr0 \ >> + -- lsp-set-type lsp-join-to-lr0 router \ >> + -- lsp-set-options lsp-join-to-lr0 router-port=lr0-join \ >> + -- lsp-set-addresses lsp-join-to-lr0 router \ >> + -- lr-add lr-guest0 \ >> + -- lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 10.51.0.1/24 \ >> + -- lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 10.42.0.2/24 \ >> + -- lrp-set-options lrp-guest0-join dynamic-routing-lb-vips=true \ >> + -- lsp-add ls-join lsp-join-to-guest0 \ >> + -- lsp-set-type lsp-join-to-guest0 router \ >> + -- lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \ >> + -- lrp-set-gateway-chassis lrp-guest0-join hv1 >> +check ovn-nbctl --wait=sb sync >> + >> +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-join) >> +pb2=$(fetch_column Port_Binding _uuid logical_port=lrp-guest0-join) >> + >> +check ovn-nbctl \ >> + --add-route lb-add lb1 172.16.2.10:80 192.168.2.10:80,192.168.2.11:80 >> \ >> + -- lr-lb-add lr-guest0 lb1 >> +check ovn-nbctl --wait=sb sync >> + >> +check_row_count Advertised_Route 2 >> + >> +# Routes to local LB VIPs are advertised without tracked port. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="172.16.1.10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=[[]] >> + >> +# Routes to LB VIPs of neighboring routers are advertised with >> +# tracked port pointing to the LRP of the neighboring router. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="172.16.2.10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=$pb2 >> + >> +# Add nonlocal LR with distributed LRP connected to GW router through >> join LS. >> +check ovn-nbctl \ >> + -- lr-add lr-guest1 \ >> + -- lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 10.51.1.1/24 \ >> + -- lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 10.42.0.3/24 \ >> + -- lsp-add ls-join lsp-join-to-guest1 \ >> + -- lsp-set-type lsp-join-to-guest1 router \ >> + -- lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \ >> + -- lrp-set-gateway-chassis lrp-guest1-join nonlocalhv >> +check ovn-nbctl --wait=sb sync >> + >> +pb3=$(fetch_column Port_Binding _uuid logical_port=lrp-guest1-join) >> + >> +check ovn-nbctl \ >> + -- --add-route lb-add lb2 172.16.3.10:80 192.168.3.10:80, >> 192.168.3.11:80 \ >> + -- lr-lb-add lr-guest1 lb2 >> +check ovn-nbctl --wait=sb sync >> + >> +check_row_count Advertised_Route 3 >> + >> +# Routes to local LB VIPs are advertised without tracked port. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="172.16.1.10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=[[]] >> + >> +# Routes to LB VIPs of neighboring routers are advertised with >> +# tracked port pointing to the LRP of the neighboring router. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="172.16.2.10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=$pb2 >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="172.16.3.10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=$pb3 >> + >> +# Removing the option:dynamic-routing removes all routes. >> +check ovn-nbctl --wait=sb remove Logical_Router lr0 option dynamic-routing >> +check_row_count Advertised_Route 0 >> + >> +# And setting it again adds them again. >> +check ovn-nbctl --wait=sb set Logical_Router lr0 >> option:dynamic-routing=true >> +check_row_count Advertised_Route 3 >> + >> +# Removing the lr will remove all routes. >> +check ovn-nbctl --wait=sb lr-del lr0 >> +check_row_count Advertised_Route 0 >> + >> +AT_CLEANUP >> +]) >> + >> +OVN_FOR_EACH_NORTHD_NO_HV([ >> +AT_SETUP([dynamic-routing - LB sync to sb IPv6]) >> +AT_KEYWORDS([dynamic-routing]) >> +ovn_start >> + >> +# Start with GW router and a single LRP. >> +check ovn-nbctl lr-add lr0 >> +check ovn-nbctl set Logical_Router lr0 \ >> + options:dynamic-routing=true \ >> + options:chassis=hv1 >> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 2001:db0::1/64 >> +check ovn-nbctl --wait=sb sync >> + >> +check_row_count Advertised_Route 0 >> + >> +datapath=$(fetch_column Datapath_Binding _uuid external_ids:name=lr0) >> +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-sw0) >> + >> +# Adding LRP dynamic-routing-lb-vips option and LB VIP rule advertises >> +# a route entry. >> +check ovn-nbctl lrp-set-options lr0-sw0 dynamic-routing-redistribute=lb >> \ >> + -- lb-add lb0 [[2001:db1::10]]:80 >> [[2001:db2::10]]:80,[[2001:db2::11]]:80 \ >> + -- lr-lb-add lr0 lb0 >> +check ovn-nbctl --wait=sb sync >> + >> +check_row_count Advertised_Route 1 >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="2001\:db1\:\:10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=[[]] >> + >> +# Add LR with distributed LRP connected to GW router through join LS >> +# and switch dynamic-routing-redistribute to lrp-guest-join so that >> +# it advertises LB VIP routes from the neighbors in the join LS. >> +check ovn-nbctl \ >> + -- remove Logical_Router_Port lr0-sw0 options >> dynamic-routing-redistribute \ >> + -- lrp-add lr0 lr0-join 00:00:00:00:ff:02 2001:db42::1/64 \ >> + -- lrp-set-options lr0-join dynamic-routing-redistribute=lb \ >> + -- ls-add ls-join \ >> + -- lsp-add ls-join lsp-join-to-lr0 \ >> + -- lsp-set-type lsp-join-to-lr0 router \ >> + -- lsp-set-options lsp-join-to-lr0 router-port=lr0-join \ >> + -- lsp-set-addresses lsp-join-to-lr0 router \ >> + -- lr-add lr-guest0 \ >> + -- lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 2001:db52::1/64 >> \ >> + -- lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 >> 2001:db42::2/64 \ >> + -- lrp-set-options lrp-guest0-join dynamic-routing-lb-vips=true \ >> + -- lsp-add ls-join lsp-join-to-guest0 \ >> + -- lsp-set-type lsp-join-to-guest0 router \ >> + -- lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \ >> + -- lrp-set-gateway-chassis lrp-guest0-join hv1 >> +check ovn-nbctl --wait=sb sync >> + >> +pb=$(fetch_column Port_Binding _uuid logical_port=lr0-join) >> +pb2=$(fetch_column Port_Binding _uuid logical_port=lrp-guest0-join) >> + >> +check ovn-nbctl \ >> + -- --add-route lb-add lb1 [[2001:db3::10]]:80 >> [[2001:db4::10]]:80,[[2001:db4::11]]:80 \ >> + -- lr-lb-add lr-guest0 lb1 >> +check ovn-nbctl --wait=sb sync >> + >> +check_row_count Advertised_Route 2 >> + >> +# Routes to local LB VIPs are advertised without tracked port. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="2001\:db1\:\:10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=[[]] >> + >> +# Routes to LB VIPs of neighboring routers are advertised with >> +# tracked port pointing to the LRP of the neighboring router. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="2001\:db3\:\:10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=$pb2 >> + >> +# Add nonlocal LR with distributed LRP connected to GW router through >> join LS. >> +check ovn-nbctl \ >> + -- lr-add lr-guest1 \ >> + -- lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 2001:db2::1/64 \ >> + -- lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 >> 2001:db42::3/64 \ >> + -- lsp-add ls-join lsp-join-to-guest1 \ >> + -- lsp-set-type lsp-join-to-guest1 router \ >> + -- lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \ >> + -- lrp-set-gateway-chassis lrp-guest1-join nonlocalhv >> +check ovn-nbctl --wait=sb sync >> + >> +pb3=$(fetch_column Port_Binding _uuid logical_port=lrp-guest1-join) >> + >> +check ovn-nbctl \ >> + -- --add-route lb-add lb2 [[2001:db5::10]]:80 >> [[2001:db4::10]]:80,[[2001:db4::11]]:80 \ >> + -- lr-lb-add lr-guest1 lb2 >> +check ovn-nbctl --wait=sb sync >> + >> +check_row_count Advertised_Route 3 >> + >> +# Routes to local LB VIPs are advertised without tracked port. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="2001\:db1\:\:10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=[[]] >> + >> +# Routes to LB VIPs of neighboring routers are advertised with >> +# tracked port pointing to the LRP of the neighboring router. >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="2001\:db3\:\:10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=$pb2 >> +check_row_count Advertised_Route 1 \ >> + ip_prefix="2001\:db5\:\:10" \ >> + datapath=$datapath \ >> + logical_port=$pb \ >> + tracked_port=$pb3 >> + >> +# Removing the option:dynamic-routing removes all routes. >> +check ovn-nbctl --wait=sb remove Logical_Router lr0 option dynamic-routing >> +check_row_count Advertised_Route 0 >> + >> +# And setting it again adds them again. >> +check ovn-nbctl --wait=sb set Logical_Router lr0 >> option:dynamic-routing=true >> +check_row_count Advertised_Route 3 >> + >> +# Removing the lr will remove all routes. >> +check ovn-nbctl --wait=sb lr-del lr0 >> +check_row_count Advertised_Route 0 >> + >> +AT_CLEANUP >> +]) >> diff --git a/tests/system-ovn.at b/tests/system-ovn.at >> index d24e50974..d7b16deda 100644 >> --- a/tests/system-ovn.at >> +++ b/tests/system-ovn.at >> @@ -16561,3 +16561,741 @@ OVS_TRAFFIC_VSWITCHD_STOP(["/.*error >> receiving.*/d >> /.*terminating with signal 15.*/d"]) >> AT_CLEANUP >> ]) >> + >> +OVN_FOR_EACH_NORTHD([ >> +AT_SETUP([dynamic-routing - LB VIPs IPv4]) >> +AT_KEYWORDS([dynamic-routing]) >> + >> +CHECK_VRF() >> +CHECK_CONNTRACK() >> +CHECK_CONNTRACK_NAT() >> + >> +vrf=1000 >> +VRF_RESERVE([$vrf]) >> +ovn_start >> +OVS_TRAFFIC_VSWITCHD_START() >> +ADD_BR([br-int]) >> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone]) >> + >> +# Set external-ids in br-int needed for ovn-controller >> +ovs-vsctl \ >> + -- set Open_vSwitch . external-ids:system-id=hv1 \ >> + -- set Open_vSwitch . >> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ >> + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ >> + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ >> + -- set bridge br-int fail-mode=secure >> other-config:disable-in-band=true >> + >> +# Start ovn-controller >> +start_daemon ovn-controller >> + >> +ovn-appctl vlog/set route_exchange >> +check ovn-nbctl -- lr-add R1 \ >> + -- set Logical_Router R1 options:requested-tnl-key=$vrf >> options:dynamic-routing=true >> + >> +check ovn-nbctl ls-add sw0 >> +check ovn-nbctl ls-add public >> + >> +check ovn-nbctl --wait=hv sync >> > > Missing in all tests: > > OVN_POPULATE_ARP > wait_for_ports_up > Good point, adding that. > >> + >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) >> > > Has this potential to flake? Wouldn't OVS_WAIT_UNTIL be better here? > Hmm, it should be OVS_WAIT_WHILE I think: OVS_WAIT_WHILE([ip link | grep -q ovnvrf$vrf:.*UP]) > >> + >> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 192.168.1.1/24 >> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 172.16.1.1/24 \ >> + -- lrp-set-options rp-public \ >> + dynamic-routing-maintain-vrf=true \ >> + dynamic-routing-redistribute=lb >> + >> +check ovn-nbctl set logical_router R1 options:chassis=hv1 >> + >> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \ >> + type=router options:router-port=rp-sw0 \ >> + -- lsp-set-addresses sw0-rp router >> + >> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port >> public-rp \ >> + type=router options:router-port=rp-public \ >> + -- lsp-set-addresses public-rp router >> + >> +check ovs-vsctl set Open_vSwitch . >> external-ids:ovn-bridge-mappings=phynet:br-ext >> + >> +check ovn-nbctl lsp-add public public1 \ >> + -- lsp-set-addresses public1 unknown \ >> + -- lsp-set-type public1 localnet \ >> + -- lsp-set-options public1 network_name=phynet >> + >> +check ovn-nbctl --wait=hv sync >> + >> +AT_CHECK([test `ip route show table $vrf | wc -l` -eq 1], [1]) >> > + >> + >> +# Create a load balancer and associate to R1 >> +check ovn-nbctl lb-add lb1 172.16.1.150:80 172.16.1.100:80 >> +check ovn-nbctl lr-lb-add R1 lb1 >> + >> +check ovn-nbctl --wait=hv sync >> + >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) >> + >> +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], >> [dnl >> +blackhole 172.16.1.150 proto 84 metric 1000]) >> + >> +# Add distributed router connected through "join" LS and ensure >> +# that its LB VIPs are redistributed by R1. LS join has no >> +# IP config, routers reach each other over IPv6 LLAs >> +check ovn-nbctl --wait=sb \ >> + -- \ >> + remove Logical_Router_Port rp-public options >> dynamic-routing-redistribute \ >> + -- \ >> + lrp-add R1 r1-join 00:00:00:00:ff:02 \ >> + -- \ >> + lrp-set-options r1-join dynamic-routing-redistribute=lb \ >> + -- \ >> + ls-add ls-join \ >> + -- \ >> + lsp-add ls-join lsp-join-to-r1 \ >> + -- \ >> + lsp-set-type lsp-join-to-r1 router \ >> + -- \ >> + lsp-set-options lsp-join-to-r1 router-port=r1-join \ >> + -- \ >> + lsp-set-addresses lsp-join-to-r1 router \ >> + -- \ >> + lr-add R2 \ >> + -- \ >> + lrp-add R2 r2-join 00:00:00:00:fe:02 \ >> + -- \ >> + lsp-add ls-join lsp-join-to-r2 \ >> + -- \ >> + lsp-set-type lsp-join-to-r2 router \ >> + -- \ >> + lsp-set-options lsp-join-to-r2 router-port=r2-join \ >> + -- \ >> + lrp-set-gateway-chassis r2-join hv1 >> + >> +# Create a load balancer and associate to R2 >> +check ovn-nbctl lb-add lb2 10.42.10.10:80 192.168.123.10:80 >> +check ovn-nbctl lr-lb-add R2 lb2 >> + >> +check ovn-nbctl --wait=hv sync >> + >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) >> + >> +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], >> [dnl >> +blackhole 10.42.10.10 proto 84 metric 100 >> +blackhole 172.16.1.150 proto 84 metric 1000]) >> + >> +# Move DGW of R2 to another chassis to verify that route priority will >> decrease >> +check ovn-nbctl --wait=hv \ >> + -- \ >> + lrp-del-gateway-chassis r2-join hv1 >> + -- \ >> + lrp-set-gateway-chassis r2-join hv123 >> + >> +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], >> [dnl >> +blackhole 10.42.10.10 proto 84 metric 1000 >> +blackhole 172.16.1.150 proto 84 metric 1000]) >> + >> +OVS_APP_EXIT_AND_WAIT([ovn-controller]) >> + >> +# Ensure system resources are cleaned up >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) >> +AT_CHECK([test `ip route show table $vrf | wc -l` -eq 1], [1]) >> + >> +as ovn-sb >> +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) >> + >> +as ovn-nb >> +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) >> + >> +as northd >> +OVS_APP_EXIT_AND_WAIT([ovn-northd]) >> + >> +as >> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d >> +/Failed to acquire.*/d >> +/connection dropped.*/d"]) >> +AT_CLEANUP >> +]) >> + >> +OVN_FOR_EACH_NORTHD([ >> +AT_SETUP([dynamic-routing - LB VIPs on gateway router IPv6]) >> +AT_KEYWORDS([dynamic-routing]) >> + >> +CHECK_VRF() >> +CHECK_CONNTRACK() >> +CHECK_CONNTRACK_NAT() >> + >> +vrf=1001 >> +VRF_RESERVE([$vrf]) >> +ovn_start >> +OVS_TRAFFIC_VSWITCHD_START() >> +ADD_BR([br-int]) >> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone]) >> + >> +# Set external-ids in br-int needed for ovn-controller >> +ovs-vsctl \ >> + -- set Open_vSwitch . external-ids:system-id=hv1 \ >> + -- set Open_vSwitch . >> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ >> + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ >> + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ >> + -- set bridge br-int fail-mode=secure >> other-config:disable-in-band=true >> + >> +# Start ovn-controller >> +start_daemon ovn-controller >> + >> +ovn-appctl vlog/set route_exchange >> +check ovn-nbctl -- lr-add R1 \ >> + -- set Logical_Router R1 options:requested-tnl-key=$vrf >> options:dynamic-routing=true >> + >> +check ovn-nbctl ls-add sw0 >> +check ovn-nbctl ls-add public >> + >> +check ovn-nbctl --wait=hv sync >> + >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) >> + >> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 2001:db8:100::1/64 >> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 >> 2001:db8:1001::1/64 \ >> + -- lrp-set-options rp-public \ >> + dynamic-routing-maintain-vrf=true \ >> + dynamic-routing-redistribute=lb >> + >> +check ovn-nbctl set logical_router R1 options:chassis=hv1 >> + >> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \ >> + type=router options:router-port=rp-sw0 \ >> + -- lsp-set-addresses sw0-rp router >> + >> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port >> public-rp \ >> + type=router options:router-port=rp-public \ >> + -- lsp-set-addresses public-rp router >> + >> +check ovs-vsctl set Open_vSwitch . >> external-ids:ovn-bridge-mappings=phynet:br-ext >> + >> +check ovn-nbctl lsp-add public public1 \ >> + -- lsp-set-addresses public1 unknown \ >> + -- lsp-set-type public1 localnet \ >> + -- lsp-set-options public1 network_name=phynet >> + >> +check ovn-nbctl --wait=hv sync >> + >> +AT_CHECK([test `ip -6 route show table $vrf | wc -l` -eq 1], [1]) >> + >> +# Create a load balancer and associate to R1 >> +check ovn-nbctl lb-add lb1 [[2001:db8:1001::150]]:80 >> [[2001:db8:2001::100]]:80 >> +check ovn-nbctl lr-lb-add R1 lb1 >> + >> +check ovn-nbctl --wait=hv sync >> + >> +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP]) >> +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1]) >> + >> +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk '{$1=$1};1'], >> [dnl >> +blackhole 2001:db8:1001::150 dev lo proto 84 metric 1000 pref medium]) >> + >> +# Add distributed router connected through "join" LS and ensure >> +# that its LB VIPs are redistributed by R1. LS join has no >> +# IP config, routers reach each other over IPv6 LLAs >> +check ovn-nbctl --wait=sb \ >> + -- \ >> + remove Logical_Router_Port rp-public options >> dynamic-routing-redistribute \ >> + -- \ >> + lrp-add R1 r1-join 00:00:00:00:ff:02 \ >> + -- \ >> + lrp-set-options r1-join dynamic-routing-redistribute=lb \ >> + -- \ >> + ls-add ls-join \ >> + -- \ >> + lsp-add ls-join lsp-join-to-r1 \ >> + -- \ >> + lsp-set-type lsp-join-to-r1 router \ >> + -- \ >> + lsp-set-options lsp-join-to-r1 router-port=r1-join \ >> + -- \ >> + lsp-set-addresses lsp-join-to-r1 router \ >> + -- \ >> + lr-add R2 \ >> + -- \ >> + lrp-add R2 r2-join 00:00:00:00:fe:02 \ >> + -- \ >> + lsp-add ls-join lsp-join-to-r2 \ >> + -- \ >> + lsp-set-type lsp-join-to-r2 router \ >> + -- \ >> + lsp-set-options lsp-join-to-r2 router-port=r2-join \ >> + -- \ >> + lrp-set-gateway-chassis r2-join hv1 >> + >> +# Create a load balancer and associate to R2 >> +check ovn-nbctl lb-add lb2 [[2001:db8:3001::150]]:80 >> [[2001:db8:4001::100]]:80 >> +check ovn-nbctl lr-lb-add R2 lb2 >> + >> +check ovn-nbctl --wait=hv sync >> + >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) >> + >> +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk '{$1=$1};1'], >> [dnl >> +blackhole 2001:db8:1001::150 dev lo proto 84 metric 1000 pref medium >> +blackhole 2001:db8:3001::150 dev lo proto 84 metric 100 pref medium]) >> + >> +# Move DGW of R2 to another chassis to verify that route priority will >> decrease >> +check ovn-nbctl --wait=hv \ >> + -- \ >> + lrp-del-gateway-chassis r2-join hv1 >> + -- \ >> + lrp-set-gateway-chassis r2-join hv123 >> + >> +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk '{$1=$1};1'], >> [dnl >> +blackhole 2001:db8:1001::150 dev lo proto 84 metric 1000 pref medium >> +blackhole 2001:db8:3001::150 dev lo proto 84 metric 1000 pref medium]) >> + >> +OVS_APP_EXIT_AND_WAIT([ovn-controller]) >> + >> +# Ensure system resources are cleaned up >> +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP], [1]) >> +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1], [1]) >> + >> +as ovn-sb >> +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) >> + >> +as ovn-nb >> +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) >> + >> +as northd >> +OVS_APP_EXIT_AND_WAIT([ovn-northd]) >> + >> +as >> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d >> +/Failed to acquire.*/d >> +/connection dropped.*/d"]) >> +AT_CLEANUP >> +]) >> + >> +OVN_FOR_EACH_NORTHD([ >> +AT_SETUP([dynamic-routing - DNAT and DNAT_AND_SNAT on gateway router >> IPv4]) >> +AT_KEYWORDS([dynamic-routing]) >> + >> +CHECK_VRF() >> +CHECK_CONNTRACK() >> +CHECK_CONNTRACK_NAT() >> + >> +vrf=1002 >> +VRF_RESERVE([$vrf]) >> +ovn_start >> +OVS_TRAFFIC_VSWITCHD_START() >> +ADD_BR([br-int]) >> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone]) >> + >> +# Set external-ids in br-int needed for ovn-controller >> +ovs-vsctl \ >> + -- set Open_vSwitch . external-ids:system-id=hv1 \ >> + -- set Open_vSwitch . >> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ >> + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ >> + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ >> + -- set bridge br-int fail-mode=secure >> other-config:disable-in-band=true >> + >> +# Start ovn-controller >> +start_daemon ovn-controller >> + >> +ovn-appctl vlog/set route_exchange >> +check ovn-nbctl -- lr-add R1 \ >> + -- set Logical_Router R1 options:requested-tnl-key=$vrf >> options:dynamic-routing=true >> + >> +check ovn-nbctl ls-add sw0 >> +check ovn-nbctl ls-add public >> + >> +check ovn-nbctl --wait=hv sync >> + >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) >> + >> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 192.168.1.1/24 >> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 172.16.1.1/24 \ >> + -- lrp-set-options rp-public \ >> + dynamic-routing-maintain-vrf=true \ >> + dynamic-routing-redistribute=nat >> + >> +check ovn-nbctl set logical_router R1 options:chassis=hv1 >> + >> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \ >> + type=router options:router-port=rp-sw0 \ >> + -- lsp-set-addresses sw0-rp router >> + >> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port >> public-rp \ >> + type=router options:router-port=rp-public \ >> + -- lsp-set-addresses public-rp router >> + >> +check ovs-vsctl set Open_vSwitch . >> external-ids:ovn-bridge-mappings=phynet:br-ext >> + >> +check ovn-nbctl lsp-add public public1 \ >> + -- lsp-set-addresses public1 unknown \ >> + -- lsp-set-type public1 localnet \ >> + -- lsp-set-options public1 network_name=phynet >> + >> +check ovn-nbctl --wait=hv sync >> + >> +AT_CHECK([test `ip route show table $vrf | wc -l` -eq 2], [1]) >> + >> +# Create dnat_and_snat, dnat rules in R1 >> +check ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.10 192.168.1.10 >> +check ovn-nbctl lr-nat-add R1 dnat 172.16.1.11 192.168.1.11 >> + >> +check ovn-nbctl --wait=hv sync >> + >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) >> + >> +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], >> [dnl >> +blackhole 172.16.1.10 proto 84 metric 1000 >> +blackhole 172.16.1.11 proto 84 metric 1000]) >> + >> +# Add distributed router connected through "join" LS and ensure >> +# that its external NAT IPs are redistributed by R1. LS join has no >> +# IP config, routers reach each other over IPv6 LLAs >> +check ovn-nbctl --wait=sb \ >> + -- \ >> + remove Logical_Router_Port rp-public options >> dynamic-routing-redistribute \ >> + -- \ >> + lrp-add R1 r1-join 00:00:00:00:ff:02 \ >> + -- \ >> + lrp-set-options r1-join dynamic-routing-redistribute=nat \ >> + -- \ >> + ls-add ls-join \ >> + -- \ >> + lsp-add ls-join lsp-join-to-r1 \ >> + -- \ >> + lsp-set-type lsp-join-to-r1 router \ >> + -- \ >> + lsp-set-options lsp-join-to-r1 router-port=r1-join \ >> + -- \ >> + lsp-set-addresses lsp-join-to-r1 router \ >> + -- \ >> + lr-add R2 \ >> + -- \ >> + lrp-add R2 r2-join 00:00:00:00:fe:02 \ >> + -- \ >> + lsp-add ls-join lsp-join-to-r2 \ >> + -- \ >> + lsp-set-type lsp-join-to-r2 router \ >> + -- \ >> + lsp-set-options lsp-join-to-r2 router-port=r2-join \ >> + -- \ >> + ha-chassis-group-add g1 \ >> + -- \ >> + ha-chassis-group-add-chassis g1 hv1 10 >> + >> +group_uuid=$(ovn-nbctl get ha_chassis_group g1 _uuid) >> +check ovn-nbctl set logical_router_port r2-join >> ha_chassis_group="$group_uuid" >> + >> +# Create NAT on R2 >> +check ovn-nbctl lr-nat-add R2 dnat_and_snat 10.42.10.10 192.168.1.10 >> +check ovn-nbctl lr-nat-add R2 dnat_and_snat 10.42.10.11 192.168.1.11 >> + >> +check ovn-nbctl --wait=hv sync >> + >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) >> + >> +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], >> [dnl >> +blackhole 10.42.10.10 proto 84 metric 100 >> +blackhole 10.42.10.11 proto 84 metric 100 >> +blackhole 172.16.1.10 proto 84 metric 1000 >> +blackhole 172.16.1.11 proto 84 metric 1000]) >> + >> +# Move DGW of R2 to another chassis to verify that route priority will >> decrease >> +check ovn-nbctl --wait=hv \ >> + -- \ >> + ha-chassis-group-add-chassis g1 hv2 20 \ >> + -- \ >> + ha-chassis-group-remove-chassis g1 hv1 >> + >> +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], >> [dnl >> +blackhole 10.42.10.10 proto 84 metric 1000 >> +blackhole 10.42.10.11 proto 84 metric 1000 >> +blackhole 172.16.1.10 proto 84 metric 1000 >> +blackhole 172.16.1.11 proto 84 metric 1000]) >> + >> +# Add "guest" LS connected the distributed router R2 and one "VM" called >> +# guest1. >> +# Also, connect R2 to ls-join via nother DGW >> +check ovn-nbctl --wait=sb \ >> + -- \ >> + lrp-add R2 r2-guest 00:00:00:00:fd:02 192.168.2.1/24 \ >> + -- \ >> + lrp-add R2 r2-join-dgw2 00:00:00:00:fd:02 \ >> + -- \ >> + lsp-add ls-join lsp-join-to-r2-dgw2 \ >> + -- \ >> + lsp-set-type lsp-join-to-r2-dgw2 router \ >> + -- \ >> + lsp-set-options lsp-join-to-r2-dgw2 router-port=r2-join-dgw2 \ >> + -- \ >> + ls-add ls-guest \ >> + -- \ >> + lsp-add ls-guest lsp-guest-to-r2 \ >> + -- \ >> + lsp-set-type lsp-guest-to-r2 router \ >> + -- \ >> + lsp-set-options lsp-guest-to-r2 router-port=r2-guest \ >> + -- \ >> + lsp-set-addresses lsp-guest-to-r2 router \ >> + -- \ >> + lsp-add ls-guest guest1 \ >> + -- \ >> + lsp-set-addresses guest1 '00:00:00:00:fc:03 192.168.2.10' >> + >> +check ovn-nbctl set logical_router_port r2-join-dgw2 \ >> + ha_chassis_group="$group_uuid" >> +# Bind guest1 on hv1 >> +ADD_NAMESPACES(guest1) >> +ADD_VETH(guest1, guest1, br-int, "192.168.2.10/24", "00:00:00:00:fd:03", >> \ >> + "192.168.2.1") >> + >> +# Add distributed NAT on R2 for guest1 and ensure that it's advertised >> +# with better metric on chassis hv1 even when DGP for R2 is elsewhere. >> +check ovn-nbctl --gateway-port \ >> + r2-join-dgw2 --add-route lr-nat-add R2 dnat_and_snat 10.42.20.11 \ >> + 192.168.2.10 guest1 00:00:00:00:fc:04 >> +check ovn-nbctl --wait=hv sync >> + >> +ovn-sbctl list advertised_route >> +OVS_WAIT_UNTIL_EQUAL([ip route list vrf ovnvrf$vrf | awk '{$1=$1};1'], >> [dnl >> +blackhole 10.42.10.10 proto 84 metric 1000 >> +blackhole 10.42.10.11 proto 84 metric 1000 >> +blackhole 10.42.20.11 proto 84 metric 100 >> +blackhole 172.16.1.10 proto 84 metric 1000 >> +blackhole 172.16.1.11 proto 84 metric 1000]) >> + >> +OVS_APP_EXIT_AND_WAIT([ovn-controller]) >> + >> +# Ensure system resources are cleaned up >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) >> +AT_CHECK([test `ip route show table $vrf | wc -l` -eq 1], [1]) >> + >> +as ovn-sb >> +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) >> + >> +as ovn-nb >> +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) >> + >> +as northd >> +OVS_APP_EXIT_AND_WAIT([ovn-northd]) >> + >> +as >> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d >> +/Failed to acquire.*/d >> +/connection dropped.*/d"]) >> +AT_CLEANUP >> +]) >> + >> +OVN_FOR_EACH_NORTHD([ >> +AT_SETUP([dynamic-routing - DNAT and DNAT_AND_SNAT on gateway router >> IPv6]) >> +AT_KEYWORDS([dynamic-routing]) >> + >> +CHECK_VRF() >> +CHECK_CONNTRACK() >> +CHECK_CONNTRACK_NAT() >> + >> +vrf=1003 >> +VRF_RESERVE([$vrf]) >> +ovn_start >> +OVS_TRAFFIC_VSWITCHD_START() >> +ADD_BR([br-int]) >> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone]) >> + >> +# Set external-ids in br-int needed for ovn-controller >> +ovs-vsctl \ >> + -- set Open_vSwitch . external-ids:system-id=hv1 \ >> + -- set Open_vSwitch . >> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ >> + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ >> + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \ >> + -- set bridge br-int fail-mode=secure >> other-config:disable-in-band=true >> + >> +# Start ovn-controller >> +start_daemon ovn-controller >> + >> +ovn-appctl vlog/set route_exchange >> +check ovn-nbctl -- lr-add R1 \ >> + -- set Logical_Router R1 options:requested-tnl-key=$vrf >> options:dynamic-routing=true >> + >> +check ovn-nbctl ls-add sw0 >> +check ovn-nbctl ls-add public >> + >> +check ovn-nbctl --wait=hv sync >> + >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP], [1]) >> + >> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 2001:db8:100::1/64 >> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 >> 2001:db8:1003::1/64 \ >> + -- lrp-set-options rp-public \ >> + dynamic-routing-maintain-vrf=true \ >> + dynamic-routing-redistribute=nat >> + >> +check ovn-nbctl set logical_router R1 options:chassis=hv1 >> + >> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \ >> + type=router options:router-port=rp-sw0 \ >> + -- lsp-set-addresses sw0-rp router >> + >> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port >> public-rp \ >> + type=router options:router-port=rp-public \ >> + -- lsp-set-addresses public-rp router >> + >> +check ovs-vsctl set Open_vSwitch . >> external-ids:ovn-bridge-mappings=phynet:br-ext >> + >> +check ovn-nbctl lsp-add public public1 \ >> + -- lsp-set-addresses public1 unknown \ >> + -- lsp-set-type public1 localnet \ >> + -- lsp-set-options public1 network_name=phynet >> + >> +check ovn-nbctl --wait=hv sync >> + >> +AT_CHECK([test `ip -6 route show table $vrf | wc -l` -eq 2], [1]) >> + >> +# Create dnat_and_snat, dnat rules in R1 >> +check ovn-nbctl lr-nat-add R1 \ >> + dnat_and_snat 2001:db8:1003::150 2001:db8:100::100 >> +check ovn-nbctl lr-nat-add R1 \ >> + dnat 2001:db8:1003::151 2001:db8:100::100 >> + >> +check ovn-nbctl --wait=hv sync >> + >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) >> +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk '{$1=$1};1'], >> [dnl >> +blackhole 2001:db8:1003::150 dev lo proto 84 metric 1000 pref medium >> +blackhole 2001:db8:1003::151 dev lo proto 84 metric 1000 pref medium]) >> + >> +# Add distributed router connected through "join" LS and ensure >> +# that its external NAT IPs are redistributed by R1. LS join has no >> +# IP config, routers reach each other over IPv6 LLAs >> +check ovn-nbctl --wait=sb \ >> + -- \ >> + remove Logical_Router_Port rp-public options >> dynamic-routing-redistribute \ >> + -- \ >> + lrp-add R1 r1-join 00:00:00:00:ff:02 \ >> + -- \ >> + lrp-set-options r1-join dynamic-routing-redistribute=nat \ >> + -- \ >> + ls-add ls-join \ >> + -- \ >> + lsp-add ls-join lsp-join-to-r1 \ >> + -- \ >> + lsp-set-type lsp-join-to-r1 router \ >> + -- \ >> + lsp-set-options lsp-join-to-r1 router-port=r1-join \ >> + -- \ >> + lsp-set-addresses lsp-join-to-r1 router \ >> + -- \ >> + lr-add R2 \ >> + -- \ >> + lrp-add R2 r2-join 00:00:00:00:fe:02 \ >> + -- \ >> + lsp-add ls-join lsp-join-to-r2 \ >> + -- \ >> + lsp-set-type lsp-join-to-r2 router \ >> + -- \ >> + lsp-set-options lsp-join-to-r2 router-port=r2-join \ >> + -- \ >> + ha-chassis-group-add g1 \ >> + -- \ >> + ha-chassis-group-add-chassis g1 hv1 10 >> + >> +group_uuid=$(ovn-nbctl get ha_chassis_group g1 _uuid) >> +check ovn-nbctl set logical_router_port r2-join >> ha_chassis_group="$group_uuid" >> + >> +# Create NAT on R2 >> +check ovn-nbctl lr-nat-add R2 \ >> + dnat_and_snat 2001:db8:1004::150 2001:db8:201::100 >> +check ovn-nbctl lr-nat-add R2 \ >> + dnat 2001:db8:1004::151 2001:db8:201::101 >> + >> +check ovn-nbctl --wait=hv sync >> + >> +AT_CHECK([ip link | grep -q ovnvrf$vrf:.*UP]) >> + >> +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk '{$1=$1};1'], >> [dnl >> +blackhole 2001:db8:1003::150 dev lo proto 84 metric 1000 pref medium >> +blackhole 2001:db8:1003::151 dev lo proto 84 metric 1000 pref medium >> +blackhole 2001:db8:1004::150 dev lo proto 84 metric 100 pref medium >> +blackhole 2001:db8:1004::151 dev lo proto 84 metric 100 pref medium]) >> + >> +# Move DGW of R2 to another chassis to verify that route priority will >> decrease >> +check ovn-nbctl --wait=hv \ >> + -- \ >> + ha-chassis-group-add-chassis g1 hv2 20 \ >> + -- \ >> + ha-chassis-group-remove-chassis g1 hv1 >> + >> +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk '{$1=$1};1'], >> [dnl >> +blackhole 2001:db8:1003::150 dev lo proto 84 metric 1000 pref medium >> +blackhole 2001:db8:1003::151 dev lo proto 84 metric 1000 pref medium >> +blackhole 2001:db8:1004::150 dev lo proto 84 metric 1000 pref medium >> +blackhole 2001:db8:1004::151 dev lo proto 84 metric 1000 pref medium]) >> + >> +# Add "guest" LS connected the distributed router R2 and one "VM" called >> +# guest1. >> +# Also, connect R2 to ls-join via nother DGW >> +check ovn-nbctl --wait=sb \ >> + -- \ >> + lrp-add R2 r2-guest 00:00:00:00:fd:02 2001:db9:2000::1/64 \ >> + -- \ >> + lrp-add R2 r2-join-dgw2 00:00:00:00:fd:02 \ >> + -- \ >> + lsp-add ls-join lsp-join-to-r2-dgw2 \ >> + -- \ >> + lsp-set-type lsp-join-to-r2-dgw2 router \ >> + -- \ >> + lsp-set-options lsp-join-to-r2-dgw2 router-port=r2-join-dgw2 \ >> + -- \ >> + ls-add ls-guest \ >> + -- \ >> + lsp-add ls-guest lsp-guest-to-r2 \ >> + -- \ >> + lsp-set-type lsp-guest-to-r2 router \ >> + -- \ >> + lsp-set-options lsp-guest-to-r2 router-port=r2-guest \ >> + -- \ >> + lsp-set-addresses lsp-guest-to-r2 router \ >> + -- \ >> + lsp-add ls-guest guest1 \ >> + -- \ >> + lsp-set-addresses guest1 '00:00:00:00:fc:03 2001:db9:2000::10' >> + >> +check ovn-nbctl set logical_router_port r2-join-dgw2 >> ha_chassis_group="$group_uuid" >> +# Bind guest1 on hv1 >> +ADD_NAMESPACES(guest1) >> +ADD_VETH(guest1, guest1, br-int, "2001:db9:2000::10/64", >> "00:00:00:00:fd:03", \ >> + "2001:db9:2000::1", "nodad") >> + >> +# Add distributed NAT on R2 for guest1 and ensure that it's advertised >> +# with better metric on chassis hv1 even when DGP for R2 is elsewhere. >> +check ovn-nbctl --gateway-port \ >> + r2-join-dgw2 --add-route lr-nat-add R2 dnat_and_snat \ >> + 2001:db8:1005::150 2001:db9:2000::10 guest1 00:00:00:00:fc:04 >> +check ovn-nbctl --wait=hv sync >> + >> +OVS_WAIT_UNTIL_EQUAL([ip -6 route list vrf ovnvrf$vrf | awk '{$1=$1};1'], >> [dnl >> +blackhole 2001:db8:1003::150 dev lo proto 84 metric 1000 pref medium >> +blackhole 2001:db8:1003::151 dev lo proto 84 metric 1000 pref medium >> +blackhole 2001:db8:1004::150 dev lo proto 84 metric 1000 pref medium >> +blackhole 2001:db8:1004::151 dev lo proto 84 metric 1000 pref medium >> +blackhole 2001:db8:1005::150 dev lo proto 84 metric 100 pref medium]) >> + >> +OVS_APP_EXIT_AND_WAIT([ovn-controller]) >> +# Ensure system resources are cleaned up >> +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP], [1]) >> +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2], [1]) >> + >> +as ovn-sb >> +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) >> + >> +as ovn-nb >> +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) >> + >> +as northd >> +OVS_APP_EXIT_AND_WAIT([ovn-northd]) >> + >> +as >> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d >> +/Failed to acquire.*/d >> +/connection dropped.*/d"]) >> +AT_CLEANUP >> +]) >> + >> + >> -- >> 2.43.0 >> >> > Thanks, > Ales > I tried to address Ales' review comments here: https://github.com/dceara/ovn/commit/3655aa3 But I'm also going to reindent and do some minor style changes in the system tests. In the meantime, Ales, Martin, please let me know if the incremental changes look OK to you, I can squash them in when applying the series. Thanks, Dumitru _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
