Hi Dumitru, Frode Thanks for the detailed feedback. I'll add my thoughts inline below.
On Fri, 2025-02-07 at 08:34 +0100, Frode Nordahl wrote: > On Fri, Feb 7, 2025 at 1:35 AM Dumitru Ceara <[email protected]> > wrote: > > > > On 2/6/25 10:45 PM, Dumitru Ceara wrote: > > > On 2/6/25 10:38 PM, Dumitru Ceara wrote: > > > > On 2/5/25 10:21 AM, Martin Kalcok wrote: > > > > > This change builds on top of the new "dynamic routing" OVN > > > > > feature > > > > > that allows advertising routes to the fabric network. When LR > > > > > option > > > > > "dynamic-routing" is set on the router, following two new LRP > > > > > options > > > > > become available: > > > > > > > > > > * dynamic-routing-nat - When set to "true", ovn-controller > > > > > will advertise > > > > > routes for external NAT IPs valid for > > > > > the LRP. > > > > > * dynamic-routing-lb-vips - When set to "true", ovn- > > > > > controller will advertise > > > > > host routes to LB VIPs via the > > > > > LRP. > > > > > > > > > > Co-authored-by: Frode Nordahl <[email protected]> > > > > > Signed-off-by: Frode Nordahl <[email protected]> > > > > > Signed-off-by: Martin Kalcok <[email protected]> > > > > > --- > > > > > > > > Hi Martin, Frode, > > > > > > > > Martin, I know you had this patchset up for review for quite a > > > > long time > > > > and I apologize for not finding time before to closely look at > > > > the > > > > implementation. > > > > > > > > I now have some concerns related to both desing and also > > > > scalability. > > > > > > > > I know the incremental processing engine in northd (or OVN in > > > > general) > > > > is not that developer friendly but we do already have the > > > > information > > > > that's needed for the changes you're making in the en_routes > > > > node: > > > > > > > > - load balancers applied on each router (directly or through lb > > > > groups) > > > > are parsed in the lb_data I-P node > > > > - nats are parsed in the lr_nat I-P node > > > > > > > > This also gets propagated in the lr_stateful I-P node data: > > > > > > > > struct lr_stateful_record { > > > > ... > > > > /* UUID of the NB Logical Router. */ > > > > struct uuid nbr_uuid; > > > > ... > > > > /* This lrnat_rec comes from the en_lrnat engine node data. > > > > */ > > > > const struct lr_nat_record *lrnat_rec; > > > > > > > > bool has_lb_vip; > > > > > > > > /* Load Balancer vIPs relevant for this datapath. */ > > > > struct ovn_lb_ip_set *lb_ips; > > > > ... > > > > } > > > > > > > > It's true that we don't have per LB IP information that tells > > > > us which > > > > LB that IP came from but if needed we could try to add that > > > > too. > > > > > > > > So we shouldn't have to re-parse NB.NAT/NB.Load_Balancer > > > > records. I'm > > > > detailing this concern lower in publish_lb_route() but it also > > > > applies > > > > to publish_nat_route(). > > > > > > > > Looking at the tests I realized that we don't cover any of the > > "tracked_port" related cases. So I have another > > question/suggestion below. Good catch, I thought that we re checking route metrics in the system tests, but that was for the "connected host routes". > > > > > > > NEWS | 17 + > > > > > northd/en-advertised-route-sync.c | 198 +++++++++++ > > > > > northd/en-northd.c | 5 +- > > > > > northd/inc-proc-northd.c | 5 + > > > > > northd/northd.c | 134 ++++++- > > > > > northd/northd.h | 8 +- > > > > > ovn-nb.xml | 32 ++ > > > > > ovs | 2 +- > > > > > tests/ovn-northd.at | 556 > > > > > ++++++++++++++++++++++++++++++ > > > > > tests/system-ovn.at | 379 ++++++++++++++++++++ > > > > > 10 files changed, 1332 insertions(+), 4 deletions(-) > > > > > > > > > > diff --git a/NEWS b/NEWS > > > > > index eca17b728..137ff48fd 100644 > > > > > --- a/NEWS > > > > > +++ b/NEWS > > > > > @@ -61,6 +61,23 @@ Post v24.09.0 > > > > > * Add the option "dynamic-routing-ifname" to LRPs. If > > > > > set only routes > > > > > learned from a linux iterfaces with that name are > > > > > treated as relevant > > > > > routes for this LRP. > > > > > + - Add the option "dynamic-routing" to Logical Routers. If > > > > > set to true all > > > > > + static and connected routes attached to the router are > > > > > shared to the > > > > > + southbound "Route" table for sharing outside of OVN. > > > > > + The routes can furthe be filtered by setting `dynamic- > > > > > routing-connected` > > > > > + and `dynamic-routing-static` on the LR or LRP. The LRP > > > > > settings overwrite > > > > > + the LR settings for all routes using this interface as > > > > > an exit. > > > > > + - Allow Logical Routers to dynamically learn routes from > > > > > outside the fabric. > > > > > + Routes entered into the "Route" table in the southbound > > > > > database will be > > > > > + learned by the respective LR. They are included in the > > > > > route table with > > > > > + a lower priority than static routes. > > > > > + - Add the option "dynamic-routing-connected-as-host- > > > > > routes" to LRPs. If set > > > > > + to true then connected routes are announced as > > > > > individual host routes. > > > > > + - Add 'dynamic-routing-lb-vips' LRP option. If set to > > > > > true, the LRP can be > > > > > + used to advertise host paths to the Load Balancer VIPs > > > > > associated with the > > > > > + LR. > > > > > + - Add 'dynamic-routing-nat' LRP option. If set to true, > > > > > the LRP can be used > > > > > + to advertise external NAT IPs associated with it. > > > > > > > > > > OVN v24.09.0 - 13 Sep 2024 > > > > > -------------------------- > > > > > diff --git a/northd/en-advertised-route-sync.c b/northd/en- > > > > > advertised-route-sync.c > > > > > index 02b3aba7a..b6b759201 100644 > > > > > --- a/northd/en-advertised-route-sync.c > > > > > +++ b/northd/en-advertised-route-sync.c > > > > > @@ -16,6 +16,7 @@ > > > > > > > > > > #include <config.h> > > > > > > > > > > +#include "openvswitch/vlog.h" > > > > > #include "smap.h" > > > > > #include "stopwatch.h" > > > > > #include "northd.h" > > > > > @@ -26,6 +27,8 @@ > > > > > #include "openvswitch/hmap.h" > > > > > #include "ovn-util.h" > > > > > > > > > > +VLOG_DEFINE_THIS_MODULE(en_advertised_route_sync); > > > > > + > > > > > static void > > > > > advertised_route_table_sync( > > > > > struct ovsdb_idl_txn *ovnsb_txn, > > > > > @@ -312,6 +315,185 @@ publish_host_routes(struct hmap > > > > > *sync_routes, > > > > > } > > > > > } > > > > > > > > > > +/* This function searches for an ovn_port with specific > > > > > "op_ip" > > > > > + * IP address in all LS datapaths directly connected to the > > > > > + * LR datapath "od". > > > > > + * If no match is found, this function returns NULL.*/ > > > > > +static struct ovn_port * > > > > > +find_port_in_connected_ls(struct ovn_datapath *od, char > > > > > *op_ip) > > > > > +{ > > > > > + struct in6_addr *ip_addr = xmalloc(sizeof *ip_addr); > > > > > + unsigned int plen; > > > > > + bool is_ipv4 = true; > > > > > + > > > > > + /* Return immediately if op_ip isn't a host route.*/ > > > > > > > > We shouldn't have to check this, it should be the caller doing > > > > it. We > > > > shouldn't parse values coming directly from the database here. > > > > We > > > > should receive an already correct struct in6_addr * instead as > > > > argument. Hmm, the ultimate source of the "op_ip" string is the NAT's internal IP/LB backend IP. I don't see immediate way to have this information available in the in6_addr* format. Am I overlooking something? > > > > > > > > > + if (!ip46_parse_cidr(op_ip, ip_addr, &plen)) { > > > > > + static struct vlog_rate_limit rl = > > > > > VLOG_RATE_LIMIT_INIT(5, 1); > > > > > + VLOG_WARN_RL(&rl, "bad IP for OVN port: %s", op_ip); > > > > > + free(ip_addr); > > > > > + return NULL; > > > > > + } > > > > > + if (IN6_IS_ADDR_V4MAPPED(ip_addr) && plen != 32) { > > > > > + if (plen != 32) { > > > > > + free(ip_addr); > > > > > + return NULL; > > > > > + } > > > > > + } else { > > > > > + is_ipv4 = false; > > > > > + if (plen != 128) { > > > > > + free(ip_addr); > > > > > + return NULL; > > > > > + } > > > > > + } > > > > > + free(ip_addr); > > > > > + > > > > > + for (int i = 0; i < od->n_ls_peers; i++) { > > > > > > > > Nit: size_t (in a few places) > > > > > > > > > + struct ovn_datapath *peer_od = od->ls_peers[i]; > > > > > + struct ovn_port *op; > > > > > + HMAP_FOR_EACH (op, dp_node, &peer_od->ports) { > > > > > + for (int j = 0; j < op->n_lsp_addrs; j++) { > > > > > + struct lport_addresses *addrs = &op- > > > > > >lsp_addrs[j]; > > > > > + if (is_ipv4) { > > > > > + for (int k = 0; k < addrs->n_ipv4_addrs; > > > > > k++) { > > > > > + if (!strcmp(op_ip, addrs- > > > > > >ipv4_addrs[k].addr_s)) { > > > > > + return op; > > > > > + } > > > > > + } > > > > > + } else { > > > > > + for (int k = 0; k < addrs->n_ipv6_addrs; > > > > > k++) { > > > > > + if (!strcmp(op_ip, addrs- > > > > > >ipv6_addrs[k].addr_s)) { > > > > > + return op; > > > > > + } > > > > > + } > > > > > + } > > > > > + } > > > > > + } > > > > > + } > > > > > > > > 'struct ovn_datapath' has the following field: > > > > > > > > struct sset router_ips; /* Router port IPs except the IPv6 > > > > LLAs. */ > > > > > > > > Instead of walking all ports of all connected switches of 'od' > > > > maybe we > > > > should also store all switch port IPs inside 'struct > > > > ovn_datapath'. > > > > Then we'd just have to do a sset_find() for each connected > > > > switch. This would only tell us if LS has LSP with the given IP, right? Or are you suggesting this as an optimization step that would tell us if it's even worth looking through the LSPs on the given LS? i.e. we would go from: for each LS -> for each LSP -> compare op_ip with lsp_ip to: for each LS -> if sset_find(host_ips, op_ip) -> for each LSP -> compare op_ip with lsp_ip If so, I agree this would be spare us a lot of unnecessary lookups. Though proposition to alter ovn_datapath structure is bit scary. > > > > > > > > We're calling this function for each IP of a NAT/LB we're > > > > advertising (i > > > > expect quite a few of those) so all these linear lookups > > > > quickly become > > > > very costly. > > > > > > > > > + return NULL; > > > > > +} > > > > > + > > > > > +/* Publish parsed_route as a Advertised_Route record. The > > > > > main purpose > > > > > + * of this function is to find LSP with IP address that > > > > > matches the > > > > > + * logical_ip of a NAT rule on the route's out_port > > > > > datapath. This port is > > > > > + * then set as a tracked_port for the route. > > > > > + * Search is performed in all LS datapaths directly > > > > > connected to the > > > > > + * route's out_port datapath. > > > > > + * If no suitable tracked_port is found, route is still > > > > > published, but > > > > > + * without the tracked_port set.*/ > > > > > +static void > > > > > +publish_nat_route(struct hmap *route_map, > > > > > + const struct parsed_route *route) > > > > > +{ > > > > > + const struct ovn_datapath *advertised_dp = route->od; > > > > > + const struct ovn_port *ext_nat_port = route->out_port; > > > > > + > > > > > + if (!advertised_dp->nbr || !ext_nat_port->od) { > > > > > + return; > > > > > + } > > > > > + > > > > > + char *ip_prefix = normalize_v46_prefix(&route->prefix, > > > > > + route->plen); > > > > > + char *logical_ip = NULL; > > > > > + > > > > > + /* Find NAT rule with external IP that matches the > > > > > route's > > > > > + * ip_prefix.*/ > > > > > + for (int i = 0; i < ext_nat_port->od->nbr->n_nat; i++) { > > > > > > > > Nit: size_t > > > > > > > > > + struct nbrec_nat *nat = ext_nat_port->od->nbr- > > > > > >nat[i]; > > > > > + if (!strcmp(nat->external_ip, ip_prefix)) { > > > > > + logical_ip = nat->logical_ip; > > > > > + break; > > > > > + } > > > > > + } > > > > > + > > > > > + if (!logical_ip) { > > > > > + return; > > > > > + } > > > > > + > > > > > + const struct sbrec_port_binding *tracked_pb = NULL; > > > > > + struct ovn_port *tracked_port = > > > > > find_port_in_connected_ls(ext_nat_port->od, > > > > > + > > > > > logical_ip); > > > > > + if (tracked_port) { > > > > > + tracked_pb = tracked_port->sb; > > > > > + } > > > > If I comment out the part that tries to find the tracked_port all > > the > > new system and unit tests still pass. ack, I will improve tests. > > > > > > > + > > > > > + char *ip_with_mask = xasprintf("%s/%d", ip_prefix, > > > > > route->plen); > > > > > + ar_alloc_entry(route_map, > > > > > + route->od->sb, > > > > > + route->out_port->sb, > > > > > + ip_with_mask, > > > > > + tracked_pb); > > > > > + free(ip_prefix); > > > > > +} > > > > > + > > > > > +/* Publish parsed_route as a Advertised_Route record. The > > > > > main purpose > > > > > + * of this function is to find LSP with IP address that > > > > > matches the > > > > > + * endpoint IP of a LB on the route's out_port datapath. > > > > > This port is then > > > > > + * set as a tracked_port for the route. > > > > > + * Search is performed in all LS datapaths directly > > > > > connected to the > > > > > + * route's out_port datapath. > > > > > + * If no suitable tracked_port is found, route is still > > > > > published, but > > > > > + * without the tracked_port set.*/ > > > > > +static void > > > > > +publish_lb_route(struct hmap *route_map, > > > > > + const struct parsed_route *route) > > > > > +{ > > > > > + const struct ovn_datapath *advertised_dp = route->od; > > > > > + const struct ovn_port *ext_lb_port = route->out_port; > > > > > + > > > > > + if (!advertised_dp->nbr || !ext_lb_port->od) { > > > > > + return; > > > > > + } > > > > > + > > > > > + char *ip_prefix = normalize_v46_prefix(&route->prefix, > > > > > + route->plen); > > > > > + > > > > > + for (int i = 0; i < ext_lb_port->od->nbr- > > > > > >n_load_balancer; i++) { > > > > > > > > Nit: size_t > > > > > > > > Also, this is not correct/complete. Load balancers can also be > > > > applied > > > > through load balancer groups. So you'd have to also iterate on > > > > od->nbr->load_balancer_groups. > > > > > > > > However, the publish_lb_route() receives an argument a > > > > parsed_route that > > > > has been built based on a load balancer VIP address. > > > > > > > > Now we're iterating through all load balancers configured on > > > > the route's > > > > datapath to try to find one that has the same VIP as what was > > > > used when > > > > the route was built. We actually do more than that, we > > > > actually reparse > > > > each and every load balancer attached to that router. > > > > > > > > That seems very inefficient. We should find a way to store > > > > this > > > > information in the route structure itself at creation time > > > > because it > > > > gets built from a load balancer. > > > > > > > > > > Re-reading this sentence I realized it might not be that clear. > > > I meant: > > > > > > 1. based on information computed when parsing load balancer > > > configuration (lb_data -> lr_stateful_rec) > > > 2. we create 'struct parsed_route' records - the calls to > > > parsed_routes_add_lb() > > > 3. then later, if the route was created from a load balancer > > > (type == > > > ROUTE_SOURCE_LB) we try to go back and find the load balancer > > > information that triggered the route creation (at step "2"). > > > > > > We should, when doing step "2", store a pointer to the parsed LB > > > information we'll be needing at step "3". No, I've got you, it was clear the first time :) I think we could go even further and just add (optional) "struct ovn_port *tracked_port" to the "parsed_route" and populate it in the "parsed_routes_add_{nat,lb}". We would still need the functionality that currently in "find_port_in_connected_ls", so those concerns still stand and need to be addressed. However, it would allow us to toss out "publish_{nat,lb}_route" completely and thus avoid re-parsing NAT/LB records. > > > > > > > This comment also applies to the publish_nat_route() function. > > > > > > > > > + struct nbrec_load_balancer *lb = > > > > > + ext_lb_port->od->nbr->load_balancer[i]; > > > > > + struct smap lb_vips = SMAP_INITIALIZER(&lb_vips); > > > > > + struct smap_node *node; > > > > > + SMAP_FOR_EACH (node, &lb->vips) { > > > > > + struct ovn_lb_vip *lb_vip = > > > > > xmalloc(sizeof(*lb_vip)); > > > > > + char *error = ovn_lb_vip_init(lb_vip, node->key, > > > > > + node->value, > > > > > false, AF_INET6); > > > > > + if (error) { > > > > > + static struct vlog_rate_limit rl = > > > > > VLOG_RATE_LIMIT_INIT(5, 1); > > > > > + VLOG_WARN_RL(&rl, "Failed to parse LB VIP: > > > > > %s", error); > > > > > + ovn_lb_vip_destroy(lb_vip); > > > > > + free(error); > > > > > + continue; > > > > > + } > > > > > + > > > > > + /* Continue only for LBs whose VIP matches > > > > > route's ip_prefix.*/ > > > > > + if (strcmp(lb_vip->vip_str, ip_prefix)) { > > > > > + ovn_lb_vip_destroy(lb_vip); > > > > > + continue; > > > > > + } > > > > > + > > > > > + /* Find LSP that matches at least one endpoint > > > > > IP of the LB.*/ > > > > > + const struct sbrec_port_binding *tracked_pb = > > > > > NULL; > > > > > + for (int j = 0; j < lb_vip->n_backends; j++) { > > > > > > > > Nit: size_t > > > > > > > > > + char *backend = lb_vip->backends[j].ip_str; > > > > > + struct ovn_port *tracked_port = > > > > > + find_port_in_connected_ls(ext_lb_port- > > > > > >od, backend); > > > > > > > > This is a function call but it hides a LOT of complexity (see > > > > the > > > > comment above on the function definition). > > > > > > > > > + if (tracked_port) { > > > > > + tracked_pb = tracked_port->sb; > > > > > + break; > > > > > > > > This makes me wonder why we even use tracked_port for load > > > > balancers. > > > > Load balancer backends could be distributed across different > > > > physical nodes. > > > > > > > > Why advertise the route only with one port? > > > > > > > > OR, why not advertise the route with all ports who own backend > > > > IPs? > > > > > > > > > + } > > > > > + } > > > > Here too, if I comment out the part that tries to find the > > tracked_port, > > all new tests still pass. > > I'm a bit of an early bird, so I'll give my high level view on the > below question, and allow others to look at your responses in more > detail when their day starts before participating in any details > myself. > > First, thanks alot for your thorough review, and sharing of insights, > the incremental processing engine is indeed one of the darker arts, > so > I think everyone on this thread have things to learn from you. > > > Finding the tracked_port (especially for load balancers but for > > NATs > > too) seems to be the very costly operation here. However we don't > > seem > > to have any tests that cover that. > > MJ is working on a multi-node test which will cover some of this, the > current plan is to add it as a separate patch once ready. > > > This makes me wonder, is it really a requirement for this feature > > that > > the tracked port is set for these routes? Not having to implement > > it > > now seems to simplify the logic and would remove the need to trace > > back > > from lb/nat parsed_routes to their NB configurations. > > The motivation for us with this is to provide a potential low barrier > entry to consuming native BGP support from OVN for CMSes such as > OpenStack in brown field deployments. > > Some of the constraints we work in are: > * clouds with hundreds of tenants, each of which have their own > routers, so configuring dynamic routing directly on all of those is > not an option. > * l3-only underlay with no VLANs spanned across racks or EVPN support > in networking equipment. > > To meet those constraints, a potential solution is to pin a gateway > router on every chassis, connect it up to a join-ls that spans all or > groups of chassis, and then connect existing logical OVN topology to > the join-ls. > > In our tests, this appears to work quite well, and for it to be truly > useful the tracked_port needs to be populated so that we can tell the > fabric where to send packets to NAT/LB addresses through the use of > the lower metric. Without it, all chassis with reachability to the > join-ls would announce the same prefixes with the same metric, > leading > to suboptimal path for traffic (tromboning). > > > Would that be an acceptable compromise? > > I believe Martin discovered that we have missed planning work on > properly distributing the dnat flows, so they will still be on the > chassis with the active DGP in the above topology. So since we need > followup work on that anyway, I guess pin-pointing the location of > tracked_port to backend also could be delayed. > > But if we at the very least can get tracked_port pointed at the > active > DGP, the feature would be useful and consumable in this topology, > with > truly distributed support perhaps added later. > > With the branching delayed we may or may not get to more of this now > though, I'll let Martin chime in on his thoughts on how much time > would be needed. > > -- > Frode Nordahl Frode summarized it very well and I hear your concerns as well Dumitru. However if we manage to remove the need for "publish_{nat,lb}_route", as I suggested above, and if we improve the "find_port_in_connected_ls", do you think that it would reduce complexity enough? > > > > > > > > + > > > > > + char *ip_with_mask = xasprintf("%s/%d", > > > > > ip_prefix, route->plen); > > > > > + ar_alloc_entry(route_map, > > > > > + route->od->sb, > > > > > + route->out_port->sb, > > > > > + ip_with_mask, > > > > > + tracked_pb); > > > > > + ovn_lb_vip_destroy(lb_vip); > > > > > + free(ip_prefix); > > > > Don't we need to break here in any case? Yes, we should. > > > > > > > + } > > > > > + } > > > > > +} > > > > > + > > > > > static void > > > > > advertised_route_table_sync( > > > > > struct ovsdb_idl_txn *ovnsb_txn, > > > > > @@ -359,6 +541,22 @@ advertised_route_table_sync( > > > > > !route->out_port->dynamic_routing_static) { > > > > > continue; > > > > > } > > > > > + if (route->source == ROUTE_SOURCE_NAT) { > > > > > + if (!smap_get_bool(&route->out_port->nbrp- > > > > > >options, > > > > > + "dynamic-routing-nat", > > > > > false)) { > > > > > + continue; > > > > > + } > > > > > + publish_nat_route(&sync_routes, route); > > > > > + continue; > > > > > + } > > > > > + if (route->source == ROUTE_SOURCE_LB) { > > > > > + if (!smap_get_bool(&route->out_port->nbrp- > > > > > >options, > > > > > + "dynamic-routing-lb-vips", > > > > > false)) { > > > > > + continue; > > > > > + } > > > > > + publish_lb_route(&sync_routes, route); > > > > > + continue; > > > > > + } > > > > > > > > > > char *ip_prefix = normalize_v46_prefix(&route- > > > > > >prefix, > > > > > route->plen); > > > > > diff --git a/northd/en-northd.c b/northd/en-northd.c > > > > > index c7d1ebcb3..60e295628 100644 > > > > > --- a/northd/en-northd.c > > > > > +++ b/northd/en-northd.c > > > > > @@ -316,6 +316,8 @@ en_routes_run(struct engine_node *node, > > > > > void *data) > > > > > { > > > > > struct northd_data *northd_data = > > > > > engine_get_input_data("northd", node); > > > > > struct bfd_data *bfd_data = engine_get_input_data("bfd", > > > > > node); > > > > > + struct ed_type_lr_stateful *lr_stateful_data = > > > > > + engine_get_input_data("lr_stateful", node); > > > > > struct routes_data *routes_data = data; > > > > > > > > > > routes_destroy(data); > > > > > @@ -330,7 +332,8 @@ en_routes_run(struct engine_node *node, > > > > > void *data) > > > > > route_table_name); > > > > > } > > > > > > > > > > - build_parsed_routes(od, &northd_data->lr_ports, > > > > > + build_parsed_routes(od, &lr_stateful_data->table, > > > > > + &northd_data->lr_ports, > > > > > &bfd_data->bfd_connections, > > > > > &routes_data->parsed_routes, > > > > > &routes_data->route_tables, > > > > > diff --git a/northd/inc-proc-northd.c b/northd/inc-proc- > > > > > northd.c > > > > > index 86b7b999e..d74c2e3dc 100644 > > > > > --- a/northd/inc-proc-northd.c > > > > > +++ b/northd/inc-proc-northd.c > > > > > @@ -267,6 +267,11 @@ void inc_proc_northd_init(struct > > > > > ovsdb_idl_loop *nb, > > > > > engine_add_input(&en_routes, &en_bfd, NULL); > > > > > engine_add_input(&en_routes, &en_northd, > > > > > routes_northd_change_handler); > > > > > + engine_add_input(&en_routes, &en_lr_nat, > > > > > + NULL); > > > > > + engine_add_input(&en_routes, &en_lb_data, > > > > > + NULL); > > > > > + engine_add_input(&en_routes, &en_lr_stateful, > > > > > engine_noop_handler); > > > > > > > > As mentioned in my other reply, this is a performance > > > > regression and I > > > > don't really think we need the en_lr_nat and en_lb_data nodes > > > > as direct > > > > inputs for en_routes: > > > > > > > > https://mail.openvswitch.org/pipermail/ovs-dev/2025-February/420765.html ack > > > > > > > > > > > > > > engine_add_input(&en_bfd_sync, &en_bfd, NULL); > > > > > engine_add_input(&en_bfd_sync, &en_nb_bfd, NULL); > > > > > diff --git a/northd/northd.c b/northd/northd.c > > > > > index 9f8d3118c..bff69e545 100644 > > > > > --- a/northd/northd.c > > > > > +++ b/northd/northd.c > > > > > @@ -11076,8 +11076,111 @@ parsed_routes_add_connected(const > > > > > struct ovn_datapath *od, > > > > > } > > > > > } > > > > > > > > > > +/* Add "parsed_route"s to the "routes" for each IP address > > > > > in laddrs.*/ > > > > > +static void > > > > > +lport_addrs_to_parsed_routes(const struct ovn_datapath *od, > > > > > + const struct ovn_port *op, > > > > > + const struct lport_addresses > > > > > *laddrs, > > > > > + enum route_source route_type, > > > > > + struct hmap *routes, > > > > > + bool install_lflow) > > > > > +{ > > > > > + for (int i = 0; i < laddrs->n_ipv4_addrs; i++) { > > > > > > > > Nit: size_t > > > > > > > > > + struct ipv4_netaddr *addr = &laddrs->ipv4_addrs[i]; > > > > > + struct in6_addr prefix; > > > > > + ip46_parse(addr->network_s, &prefix); > > > > > + parsed_route_add(od, NULL, &prefix, addr->plen, > > > > > + false, addr->addr_s, op, > > > > > + 0, false, > > > > > + false, NULL, route_type, > > > > > + &op->nbrp->header_, routes, > > > > > install_lflow); > > > > > + } > > > > > + for (int i = 0; i < laddrs->n_ipv6_addrs; i++) { > > > > > > > > Nit: size_t > > > > > > > > > + struct ipv6_netaddr *addr = &laddrs->ipv6_addrs[i]; > > > > > + parsed_route_add(od, NULL, &addr->addr, addr->plen, > > > > > + false, addr->addr_s, op, > > > > > + 0, false, > > > > > + false, NULL, route_type, > > > > > + &op->nbrp->header_, routes, > > > > > install_lflow); > > > > > + } > > > > > +} > > > > > + > > > > > +/* Add routes of all NAT external_addresses on the ovn_port > > > > > "op", to be > > > > > + * advertised by the datapath "od". > > > > > + * If "routeable_only" is true, only NATs with option > > > > > "add_route=true" will > > > > > + * be considered. > > > > > + * Note that the "op" does not necessarily need to be in the > > > > > "od" datapath. > > > > > + * Such use-case is for advertising external NAT IPs of LRPs > > > > > in neighboring > > > > > + * routers and should be paired with routable_only=true. > > > > > + */ > > > > > +static void > > > > > +parsed_routes_add_nat(const struct ovn_datapath *od, > > > > > + const struct ovn_port *op, > > > > > + struct hmap *routes, > > > > > + bool routable_only) > > > > > +{ > > > > > + if (!op->nbrp || !smap_get_bool(&op->nbrp->options, > > > > > + "dynamic-routing-nat", > > > > > false)) { > > > > > + return; > > > > > + } > > > > > + size_t n_nats = 0; > > > > > + char **nats = NULL; > > > > > + nats = get_nat_addresses(op, &n_nats, routable_only, > > > > > false, NULL, true); > > > > > + > > > > > + for (size_t i = 0; i < n_nats; i++) { > > > > > + struct lport_addresses *laddrs = xzalloc(sizeof > > > > > *laddrs); > > > > > > > > We leak this struct. Actually, we don't need to xalloc() this, > > > > it could > > > > just be a variable on the stack AFAICT. You are right. "laddrs" doesn't need to be a pointer here. > > > > > > > > > + int ofs = 0; > > > > > + extract_addresses(nats[i], laddrs, &ofs); > > > > > + lport_addrs_to_parsed_routes(od, op, laddrs, > > > > > ROUTE_SOURCE_NAT, > > > > > + routes, false); > > > > > + destroy_lport_addresses(laddrs); > > > > > + free(nats[i]); > > > > > + } > > > > > + free(nats); > > > > > +} > > > > > + > > > > > +/* Add routes of all LB VIPs on the ovn_port "op", to be > > > > > advertised by the > > > > > + * datapath "od". > > > > > + * If "routeable_only" is true, only LBs with option > > > > > "add_route=true" will > > > > > + * be considered. > > > > > + * Note that the "op" does not necessarily need to be in the > > > > > "od" datapath. > > > > > + * Such use-case is for advertising LB VIPs of LRPs in > > > > > neighboring routers > > > > > + * and should be paired with routable_only=true. > > > > > + */ > > > > > +static void > > > > > +parsed_routes_add_lb(const struct ovn_datapath *od, > > > > > + const struct ovn_port *op, > > > > > + struct hmap *routes, > > > > > + const struct lr_stateful_record > > > > > *lr_stateful_rec, > > > > > + bool routable_only) > > > > > +{ > > > > > + if (!op->nbrp || !smap_get_bool(&op->nbrp->options, > > > > > + "dynamic-routing-lb- > > > > > vips", false)) { > > > > > + return; > > > > > + } > > > > > + > > > > > + if (!lr_stateful_rec) { > > > > > + return; > > > > > + } > > > > > + > > > > > + struct ds addresses = DS_EMPTY_INITIALIZER; > > > > > + get_lb_addresses(lr_stateful_rec, &addresses, > > > > > routable_only); > > > > > + > > > > > + struct lport_addresses *laddrs = xzalloc(sizeof > > > > > *laddrs); > > > > > > > > Same here, we leak laddrs. ack > > > > > > > > > + extract_ip_addresses(ds_cstr(&addresses), laddrs); > > > > > + > > > > > + lport_addrs_to_parsed_routes(od, op, laddrs, > > > > > ROUTE_SOURCE_LB, > > > > > + routes, false); > > > > > + > > > > > + destroy_lport_addresses(laddrs); > > > > > + ds_destroy(&addresses); > > > > > + > > > > > +} > > > > > + > > > > > void > > > > > -build_parsed_routes(const struct ovn_datapath *od, const > > > > > struct hmap *lr_ports, > > > > > +build_parsed_routes(const struct ovn_datapath *od, > > > > > + const struct lr_stateful_table > > > > > *lr_stateful_table, > > > > > + const struct hmap *lr_ports, > > > > > const struct hmap *bfd_connections, > > > > > struct hmap *routes, > > > > > struct simap *route_tables, > > > > > struct hmap *bfd_active_connections) > > > > > @@ -11097,7 +11200,34 @@ build_parsed_routes(const struct > > > > > ovn_datapath *od, const struct hmap *lr_ports, > > > > > > > > > > const struct ovn_port *op; > > > > > HMAP_FOR_EACH (op, dp_node, &od->ports) { > > > > > + const struct lr_stateful_record *lr_stateful_rec = > > > > > NULL; > > > > > + lr_stateful_rec = lr_stateful_table_find_by_index( > > > > > + lr_stateful_table, op->od->index); > > > > > + > > > > > parsed_routes_add_connected(od, op, routes); > > > > > + parsed_routes_add_nat(od, op, routes, false); > > > > > + parsed_routes_add_lb(od, op, routes, > > > > > lr_stateful_rec, false); > > > > > + } > > > > > + > > > > > + /* For GW routers we traverse also neighboring LR in > > > > > search for > > > > > + * LBs and NATs with "add_route" option set to "true". > > > > > Such NATs/LBs > > > > > + * install logical flows for their external IP addresses > > > > > to their > > > > > + * neighboring routers, and as such, we can advertise > > > > > them as routes. > > > > > + * The LRP on the neighboring router needs to have > > > > > appropriate > > > > > + * "dynamic-routing-{nat,lb-vips}" option set for this > > > > > to take effect.*/ > > > > I guess we don't have any system tests covering this? As far as I > > can > > tell the tests this patch adds only use a single GW router and no > > other > > neighbor LRs. > > > > If I comment this part out all the new system tests still pass. This is covered by unit tests "dynamic-routing - {nat,lb} sync to sb". They will fail if you comment this out. > > > > > > > + for (size_t i = 0; od->is_gw_router && i < od- > > > > > >n_ls_peers; i++) { > > > > > > > > What about direct router peers? Two logical routers can be > > > > directly > > > > attached. As discussed in the Felix's v6 series. NAT/LB VIPs on the neighboring routers will be handled exclusively by "connected-as-host" option. This section will go away in next version and the "dynamic-routing- redistribute=nat,lb-vips" will handle only "local" NATs/LBs . ref: https://mail.openvswitch.org/pipermail/ovs-dev/2025-February/420732.html > > > > > > > > > + for (size_t j = 0; j < od->ls_peers[i]- > > > > > >n_router_ports; j++) { > > > > > + struct ovn_port *router_port; > > > > > + router_port = od->ls_peers[i]->router_ports[j]- > > > > > >peer; > > > > > + > > > > > + const struct lr_stateful_record *lr_stateful_rec > > > > > = NULL; > > > > > + lr_stateful_rec = > > > > > lr_stateful_table_find_by_index( > > > > > + lr_stateful_table, router_port->od->index); > > > > > + > > > > > + parsed_routes_add_nat(od, router_port, routes, > > > > > true); > > > > > + parsed_routes_add_lb(od, router_port, routes, > > > > > + lr_stateful_rec, true); > > > > > + } > > > > > } > > > > > > > > > > HMAP_FOR_EACH_SAFE (pr, key_node, routes) { > > > > > @@ -11279,6 +11409,8 @@ route_source_to_offset(enum > > > > > route_source source) > > > > > { > > > > > switch (source) { > > > > > case ROUTE_SOURCE_CONNECTED: > > > > > + case ROUTE_SOURCE_NAT: > > > > > + case ROUTE_SOURCE_LB: > > > > > return ROUTE_PRIO_OFFSET_CONNECTED; > > > > > case ROUTE_SOURCE_STATIC: > > > > > return ROUTE_PRIO_OFFSET_STATIC; > > > > > diff --git a/northd/northd.h b/northd/northd.h > > > > > index fffaab3a0..d55909f76 100644 > > > > > --- a/northd/northd.h > > > > > +++ b/northd/northd.h > > > > > @@ -708,6 +708,10 @@ enum route_source { > > > > > ROUTE_SOURCE_CONNECTED, > > > > > /* The route is derived from a northbound static route > > > > > entry. */ > > > > > ROUTE_SOURCE_STATIC, > > > > > + /* Host route generated from NAT's external IP. */ > > > > > + ROUTE_SOURCE_NAT, > > > > > + /* Host route generated from LB's external IP. */ > > > > > + ROUTE_SOURCE_LB, > > > > > /* The route is dynamically learned by an ovn- > > > > > controller. */ > > > > > ROUTE_SOURCE_LEARNED, > > > > > }; > > > > > @@ -780,7 +784,9 @@ void northd_indices_create(struct > > > > > northd_data *data, > > > > > > > > > > void route_policies_init(struct route_policies_data *); > > > > > void route_policies_destroy(struct route_policies_data *); > > > > > -void build_parsed_routes(const struct ovn_datapath *, const > > > > > struct hmap *, > > > > > +void build_parsed_routes(const struct ovn_datapath *, > > > > > + const struct lr_stateful_table > > > > > *lr_stateful_table, > > > > > + const struct hmap *, > > > > > const struct hmap *, struct hmap *, > > > > > struct simap *, > > > > > struct hmap *); > > > > > uint32_t get_route_table_id(struct simap *, const char *); > > > > > diff --git a/ovn-nb.xml b/ovn-nb.xml > > > > > index 734fcf2bd..e8d9d5491 100644 > > > > > --- a/ovn-nb.xml > > > > > +++ b/ovn-nb.xml > > > > > @@ -2971,6 +2971,10 @@ or > > > > > table="Logical_Router_Port"/></li> > > > > > <li><ref column="options" key="dynamic-routing- > > > > > static" > > > > > table="Logical_Router_Port"/></li> > > > > > + <li><ref column="options" key="dynamic-routing-lb- > > > > > vips" > > > > > + table="Logical_Router_Port"/></li> > > > > > + <li><ref column="options" key="dynamic-routing-nat" > > > > > + table="Logical_Router_Port"/></li> > > > > > </ul> > > > > > </column> > > > > > > > > > > @@ -3828,6 +3832,34 @@ or > > > > > This option allows to have a 1:1 mapping between a > > > > > single LRP and an > > > > > individual link. > > > > > </column> > > > > > + > > > > > + <column name="options" key="dynamic-routing-lb-vips" > > > > > + type='{"type": "boolean"}'> > > > > > + <p> > > > > > + Only relevant if <ref column="options" > > > > > key="dynamic-routing" > > > > > + table="Logical_Router"/> on the respective > > > > > Logical_Router is set > > > > > + to <code>true</code>. > > > > > + > > > > > + If this option is <code>true</code>, northd will > > > > > create host route > > > > > + entries in the southbound <ref > > > > > table="Advertised_Route" > > > > > + db="OVN_Southbound"/> table, associated with this > > > > > LRP, for each LB > > > > > + VIP. > > > > > + </p> > > > > > + </column> > > > > > + > > > > > + <column name="options" key="dynamic-routing-nat" > > > > > + type='{"type": "boolean"}'> > > > > > + <p> > > > > > + Only relevant if <ref column="options" > > > > > key="dynamic-routing" > > > > > + table="Logical_Router"/> on the respective > > > > > Logical_Router is set > > > > > + to <code>true</code>. > > > > > + > > > > > + If this option is <code>true</code>, northd will > > > > > create host route > > > > > + entries in the southbound <ref > > > > > table="Advertised_Route" > > > > > + db="OVN_Southbound"/> table, for external IP > > > > > addresses of NAT rules > > > > > + associated with this LRP. > > > > > + </p> > > > > > + </column> > > > > > </group> > > > > > > > > > > <group title="Attachment"> > > > > > diff --git a/ovs b/ovs > > > > > index a3c06c309..c648ee626 160000 > > > > > --- a/ovs > > > > > +++ b/ovs > > > > > @@ -1 +1 @@ > > > > > -Subproject commit a3c06c309ab9f21fd9ce4e8dca542119d46be9c5 > > > > > +Subproject commit c648ee626efdc959ed0cff37c2b9128816450f15 > > > > > > > > This ovs submodule change is accidental I guess. > > > > > > > > > diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at > > > > > index 44dc92869..bad9ca0b5 100644 > > > > > --- a/tests/ovn-northd.at > > > > > +++ b/tests/ovn-northd.at > > > > > @@ -15134,3 +15134,559 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE > > > > > AT_CLEANUP > > > > > ]) > > > > > > > > > > +OVN_FOR_EACH_NORTHD_NO_HV([ > > > > > +AT_SETUP([dynamic-routing - nat sync to sb IPv4]) > > > > > +AT_KEYWORDS([dynamic-routing]) > > > > > +ovn_start > > > > > + > > > > > +# Start with GW router and a single LRP > > > > > +check ovn-nbctl lr-add lr0 > > > > > +check ovn-nbctl \ > > > > > + -- \ > > > > > + set Logical_Router lr0 options:dynamic-routing=true \ > > > > > + options:chassis=hv1 > > > > > +check ovn-nbctl --wait=sb \ > > > > > + -- \ > > > > > + lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 > > > > > + > > > > > +check_row_count Advertised_Route 0 > > > > > + > > > > > +datapath=$(ovn-sbctl --bare --columns _uuid list > > > > > datapath_binding lr0) > > > > > +pb=$(ovn-sbctl --bare --columns _uuid list port_binding lr0- > > > > > sw0) > > > > > + > > > > > +# Adding LRP dynamic-routing-nat option and NAT rule > > > > > advertises a route entry > > > > > +check ovn-nbctl --wait=sb \ > > > > > + -- \ > > > > > + lrp-set-options lr0-sw0 dynamic-routing-nat=true \ > > > > > + -- \ > > > > > + lr-nat-add lr0 dnat_and_snat 172.16.1.10 192.168.1.10 > > > > > + > > > > > +ovn-nbctl list NAT > > > > > +ovn-sbctl list Advertised_Route > > > > > +ovn-sbctl lflow-list > > > > > + > > > > > +check_row_count Advertised_Route 1 > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb], [0], [dnl > > > > > +172.16.1.10/32 > > > > > +]) > > > > > + > > > > > +# Add LR with distributed LRP connected to GW router through > > > > > join LS > > > > > +check ovn-nbctl --wait=sb \ > > > > > + -- \ > > > > > + lrp-add lr0 lr0-join 00:00:00:00:ff:02 10.42.0.1/24 \ > > > > > + -- \ > > > > > + ls-add ls-join \ > > > > > + -- \ > > > > > + lsp-add ls-join lsp-join-to-lr0 \ > > > > > + -- \ > > > > > + lsp-set-type lsp-join-to-lr0 router \ > > > > > + -- \ > > > > > + lsp-set-options lsp-join-to-lr0 router-port=lr0-join \ > > > > > + -- \ > > > > > + lsp-set-addresses lsp-join-to-lr0 router \ > > > > > + -- \ > > > > > + lr-add lr-guest0 \ > > > > > + -- \ > > > > > + lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 > > > > > 10.51.0.1/24 \ > > > > > + -- \ > > > > > + lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 > > > > > 10.42.0.2/24 \ > > > > > + -- \ > > > > > + lrp-set-options lrp-guest0-join dynamic-routing-nat=true > > > > > \ > > > > > + -- \ > > > > > + lsp-add ls-join lsp-join-to-guest0 \ > > > > > + -- \ > > > > > + lsp-set-type lsp-join-to-guest0 router \ > > > > > + -- \ > > > > > + lsp-set-options lsp-join-to-guest0 router-port=lrp- > > > > > guest0-join \ > > > > > + -- \ > > > > > + lrp-set-gateway-chassis lrp-guest0-join hv1 > > > > > + > > > > > +pb2=$(ovn-sbctl --bare --columns _uuid list port_binding > > > > > lrp-guest0-join) > > > > > + > > > > > +check ovn-nbctl --wait=sb \ > > > > > + --add-route lr-nat-add lr-guest0 dnat_and_snat > > > > > 172.16.2.10 192.168.2.10 > > > > > + > > > > > +check_row_count Advertised_Route 2 > > > > > +ovn-sbctl list advertised_route > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port="$pb"], [0], [dnl > > > > > +172.16.1.10/32 > > > > > +]) > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb2], [0], [dnl > > > > > +172.16.2.10/32 > > > > > +]) > > > > > + > > > > > +# Add nonlocal LR with distributed LRP connected to GW > > > > > router through join LS > > > > > +check ovn-nbctl --wait=sb \ > > > > > + -- \ > > > > > + lr-add lr-guest1 \ > > > > > + -- \ > > > > > + lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 > > > > > 10.51.1.1/24 \ > > > > > + -- \ > > > > > + lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 > > > > > 10.42.0.3/24 \ > > > > > + -- \ > > > > > + lrp-set-options lrp-guest1-join dynamic-routing-nat=true > > > > > \ > > > > > + -- \ > > > > > + lsp-add ls-join lsp-join-to-guest1 \ > > > > > + -- \ > > > > > + lsp-set-type lsp-join-to-guest1 router \ > > > > > + -- \ > > > > > + lsp-set-options lsp-join-to-guest1 router-port=lrp- > > > > > guest1-join \ > > > > > + -- \ > > > > > + lrp-set-gateway-chassis lrp-guest1-join nonlocalhv > > > > > + > > > > > +pb3=$(ovn-sbctl --bare --columns _uuid list port_binding > > > > > lrp-guest1-join) > > > > > + > > > > > +check ovn-nbctl --wait=sb \ > > > > > + --add-route lr-nat-add lr-guest1 dnat_and_snat > > > > > 172.16.3.10 192.168.3.10 > > > > > +check_row_count Advertised_Route 3 > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb], [0], [dnl > > > > > +172.16.1.10/32 > > > > > +]) > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb2], [0], [dnl > > > > > +172.16.2.10/32 > > > > > +]) > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb3], [0], [dnl > > > > > +172.16.3.10/32 > > > > > +]) > > > > > + > > > > > +# removing the option:dynamic-routing removes all routes > > > > > +check ovn-nbctl --wait=sb remove Logical_Router lr0 option > > > > > dynamic-routing > > > > > +check_row_count Advertised_Route 0 > > > > > + > > > > > +# and setting it again adds them again > > > > > +check ovn-nbctl --wait=sb set Logical_Router lr0 > > > > > option:dynamic-routing=true > > > > > +check_row_count Advertised_Route 3 > > > > > + > > > > > +# removing the lr will remove all routes > > > > > +check ovn-nbctl --wait=sb lr-del lr0 > > > > > +check_row_count Advertised_Route 0 > > > > > + > > > > > +AT_CLEANUP > > > > > +]) > > > > > + > > > > > +OVN_FOR_EACH_NORTHD_NO_HV([ > > > > > +AT_SETUP([dynamic-routing - nat sync to sb IPv6]) > > > > > +AT_KEYWORDS([dynamic-routing]) > > > > > +ovn_start > > > > > + > > > > > +# Start with GW router and a single LRP > > > > > +check ovn-nbctl lr-add lr0 > > > > > +check ovn-nbctl \ > > > > > + -- \ > > > > > + set Logical_Router lr0 options:dynamic-routing=true \ > > > > > + options:chassis=hv1 > > > > > +check ovn-nbctl --wait=sb \ > > > > > + -- \ > > > > > + lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 2001:db0::1/64 > > > > > + > > > > > +check_row_count Advertised_Route 0 > > > > > + > > > > > +datapath=$(ovn-sbctl --bare --columns _uuid list > > > > > datapath_binding lr0) > > > > > +pb=$(ovn-sbctl --bare --columns _uuid list port_binding lr0- > > > > > sw0) > > > > > + > > > > > +# Adding LRP dynamic-routing-nat option and NAT rule > > > > > advertises a route entry > > > > > +check ovn-nbctl --wait=sb \ > > > > > + -- \ > > > > > + lrp-set-options lr0-sw0 dynamic-routing-nat=true \ > > > > > + -- \ > > > > > + lr-nat-add lr0 dnat_and_snat 2001:db1::10 2001:db2::10 > > > > > + > > > > > +ovn-nbctl list NAT > > > > > +ovn-sbctl list Advertised_Route > > > > > +ovn-sbctl lflow-list > > > > > + > > > > > +check_row_count Advertised_Route 1 > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb], [0], [dnl > > > > > +2001:db1::10/128 > > > > > +]) > > > > > + > > > > > +# Add LR with distributed LRP connected to GW router through > > > > > join LS > > > > > +check ovn-nbctl --wait=sb \ > > > > > + -- \ > > > > > + lrp-add lr0 lr0-join 00:00:00:00:ff:02 2001:db42::1/64 \ > > > > > + -- \ > > > > > + ls-add ls-join \ > > > > > + -- \ > > > > > + lsp-add ls-join lsp-join-to-lr0 \ > > > > > + -- \ > > > > > + lsp-set-type lsp-join-to-lr0 router \ > > > > > + -- \ > > > > > + lsp-set-options lsp-join-to-lr0 router-port=lr0-join \ > > > > > + -- \ > > > > > + lsp-set-addresses lsp-join-to-lr0 router \ > > > > > + -- \ > > > > > + lr-add lr-guest0 \ > > > > > + -- \ > > > > > + lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 > > > > > 2001:db51::1/64 \ > > > > > + -- \ > > > > > + lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 > > > > > 2001:db42::2/64 \ > > > > > + -- \ > > > > > + lrp-set-options lrp-guest0-join dynamic-routing-nat=true > > > > > \ > > > > > + -- \ > > > > > + lsp-add ls-join lsp-join-to-guest0 \ > > > > > + -- \ > > > > > + lsp-set-type lsp-join-to-guest0 router \ > > > > > + -- \ > > > > > + lsp-set-options lsp-join-to-guest0 router-port=lrp- > > > > > guest0-join \ > > > > > + -- \ > > > > > + lrp-set-gateway-chassis lrp-guest0-join hv1 > > > > > + > > > > > +pb2=$(ovn-sbctl --bare --columns _uuid list port_binding > > > > > lrp-guest0-join) > > > > > + > > > > > +check ovn-nbctl --wait=sb \ > > > > > + --add-route lr-nat-add lr-guest0 dnat_and_snat > > > > > 2001:db3::10 2001:db4::10 > > > > > + > > > > > +check_row_count Advertised_Route 2 > > > > > +ovn-sbctl list advertised_route > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port="$pb"], [0], [dnl > > > > > +2001:db1::10/128 > > > > > +]) > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb2], [0], [dnl > > > > > +2001:db3::10/128 > > > > > +]) > > > > > + > > > > > +# Add nonlocal LR with distributed LRP connected to GW > > > > > router through join LS > > > > > +check ovn-nbctl --wait=sb \ > > > > > + -- \ > > > > > + lr-add lr-guest1 \ > > > > > + -- \ > > > > > + lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 > > > > > 2001:db52::1/64 \ > > > > > + -- \ > > > > > + lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 > > > > > 2001:db42::3/64 \ > > > > > + -- \ > > > > > + lrp-set-options lrp-guest1-join dynamic-routing-nat=true > > > > > \ > > > > > + -- \ > > > > > + lsp-add ls-join lsp-join-to-guest1 \ > > > > > + -- \ > > > > > + lsp-set-type lsp-join-to-guest1 router \ > > > > > + -- \ > > > > > + lsp-set-options lsp-join-to-guest1 router-port=lrp- > > > > > guest1-join \ > > > > > + -- \ > > > > > + lrp-set-gateway-chassis lrp-guest1-join nonlocalhv > > > > > + > > > > > +pb3=$(ovn-sbctl --bare --columns _uuid list port_binding > > > > > lrp-guest1-join) > > > > > + > > > > > +check ovn-nbctl --wait=sb \ > > > > > + --add-route lr-nat-add lr-guest1 dnat_and_snat > > > > > 2001:db5::10 2001:db6::10 > > > > > +check_row_count Advertised_Route 3 > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb], [0], [dnl > > > > > +2001:db1::10/128 > > > > > +]) > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb2], [0], [dnl > > > > > +2001:db3::10/128 > > > > > +]) > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb3], [0], [dnl > > > > > +2001:db5::10/128 > > > > > +]) > > > > > + > > > > > +# removing the option:dynamic-routing removes all routes > > > > > +check ovn-nbctl --wait=sb remove Logical_Router lr0 option > > > > > dynamic-routing > > > > > +check_row_count Advertised_Route 0 > > > > > + > > > > > +# and setting it again adds them again > > > > > +check ovn-nbctl --wait=sb set Logical_Router lr0 > > > > > option:dynamic-routing=true > > > > > +check_row_count Advertised_Route 3 > > > > > + > > > > > +# removing the lr will remove all routes > > > > > +check ovn-nbctl --wait=sb lr-del lr0 > > > > > +check_row_count Advertised_Route 0 > > > > > + > > > > > +AT_CLEANUP > > > > > +]) > > > > > + > > > > > +OVN_FOR_EACH_NORTHD_NO_HV([ > > > > > +AT_SETUP([dynamic-routing - LB sync to sb IPv4]) > > > > > +AT_KEYWORDS([dynamic-routing]) > > > > > +ovn_start > > > > > + > > > > > +# Start with GW router and a single LRP > > > > > +check ovn-nbctl lr-add lr0 > > > > > +check ovn-nbctl \ > > > > > + -- \ > > > > > + set Logical_Router lr0 options:dynamic-routing=true \ > > > > > + options:chassis=hv1 > > > > > +check ovn-nbctl --wait=sb \ > > > > > + -- \ > > > > > + lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 > > > > > + > > > > > +check_row_count Advertised_Route 0 > > > > > + > > > > > +datapath=$(ovn-sbctl --bare --columns _uuid list > > > > > datapath_binding lr0) > > > > > +pb=$(ovn-sbctl --bare --columns _uuid list port_binding lr0- > > > > > sw0) > > > > > + > > > > > +# Adding LRP dynamic-routing-lb-vips option and LB VIP rule > > > > > advertises a route entry > > > > > +check ovn-nbctl --wait=sb \ > > > > > + -- \ > > > > > + lrp-set-options lr0-sw0 dynamic-routing-lb-vips=true \ > > > > > + -- \ > > > > > + lb-add lb0 172.16.1.10:80 > > > > > 192.168.1.10:80,192.168.1.11:80 \ > > > > > + -- \ > > > > > + lr-lb-add lr0 lb0 > > > > > + > > > > > +ovn-nbctl list Load_Balancer > > > > > +ovn-sbctl list Advertised_Route > > > > > +ovn-sbctl lflow-list > > > > > + > > > > > +check_row_count Advertised_Route 1 > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb], [0], [dnl > > > > > +172.16.1.10/32 > > > > > +]) > > > > > + > > > > > +# Add LR with distributed LRP connected to GW router through > > > > > join LS > > > > > +check ovn-nbctl --wait=sb \ > > > > > + -- \ > > > > > + lrp-add lr0 lr0-join 00:00:00:00:ff:02 10.42.0.1/24 \ > > > > > + -- \ > > > > > + ls-add ls-join \ > > > > > + -- \ > > > > > + lsp-add ls-join lsp-join-to-lr0 \ > > > > > + -- \ > > > > > + lsp-set-type lsp-join-to-lr0 router \ > > > > > + -- \ > > > > > + lsp-set-options lsp-join-to-lr0 router-port=lr0-join \ > > > > > + -- \ > > > > > + lsp-set-addresses lsp-join-to-lr0 router \ > > > > > + -- \ > > > > > + lr-add lr-guest0 \ > > > > > + -- \ > > > > > + lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 > > > > > 10.51.0.1/24 \ > > > > > + -- \ > > > > > + lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 > > > > > 10.42.0.2/24 \ > > > > > + -- \ > > > > > + lrp-set-options lrp-guest0-join dynamic-routing-lb- > > > > > vips=true \ > > > > > + -- \ > > > > > + lsp-add ls-join lsp-join-to-guest0 \ > > > > > + -- \ > > > > > + lsp-set-type lsp-join-to-guest0 router \ > > > > > + -- \ > > > > > + lsp-set-options lsp-join-to-guest0 router-port=lrp- > > > > > guest0-join \ > > > > > + -- \ > > > > > + lrp-set-gateway-chassis lrp-guest0-join hv1 > > > > > + > > > > > +pb2=$(ovn-sbctl --bare --columns _uuid list port_binding > > > > > lrp-guest0-join) > > > > > + > > > > > +check ovn-nbctl --wait=sb \ > > > > > + --add-route lb-add lb1 172.16.2.10:80 > > > > > 192.168.2.10:80,192.168.2.11:80 \ > > > > > + -- \ > > > > > + lr-lb-add lr-guest0 lb1 > > > > > + > > > > > +check_row_count Advertised_Route 2 > > > > > +ovn-sbctl list advertised_route > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port="$pb"], [0], [dnl > > > > > +172.16.1.10/32 > > > > > +]) > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb2], [0], [dnl > > > > > +172.16.2.10/32 > > > > > +]) > > > > > + > > > > > +# Add nonlocal LR with distributed LRP connected to GW > > > > > router through join LS > > > > > +check ovn-nbctl --wait=sb \ > > > > > + -- \ > > > > > + lr-add lr-guest1 \ > > > > > + -- \ > > > > > + lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 > > > > > 10.51.1.1/24 \ > > > > > + -- \ > > > > > + lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 > > > > > 10.42.0.3/24 \ > > > > > + -- \ > > > > > + lrp-set-options lrp-guest1-join dynamic-routing-lb- > > > > > vips=true \ > > > > > + -- \ > > > > > + lsp-add ls-join lsp-join-to-guest1 \ > > > > > + -- \ > > > > > + lsp-set-type lsp-join-to-guest1 router \ > > > > > + -- \ > > > > > + lsp-set-options lsp-join-to-guest1 router-port=lrp- > > > > > guest1-join \ > > > > > + -- \ > > > > > + lrp-set-gateway-chassis lrp-guest1-join nonlocalhv > > > > > + > > > > > +pb3=$(ovn-sbctl --bare --columns _uuid list port_binding > > > > > lrp-guest1-join) > > > > > + > > > > > +check ovn-nbctl --wait=sb \ > > > > > + --add-route lb-add lb2 172.16.3.10:80 > > > > > 192.168.3.10:80,192.168.3.11:80 \ > > > > > + -- \ > > > > > + lr-lb-add lr-guest1 lb2 > > > > > +check_row_count Advertised_Route 3 > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb], [0], [dnl > > > > > +172.16.1.10/32 > > > > > +]) > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb2], [0], [dnl > > > > > +172.16.2.10/32 > > > > > +]) > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb3], [0], [dnl > > > > > +172.16.3.10/32 > > > > > +]) > > > > > + > > > > > +# removing the option:dynamic-routing removes all routes > > > > > +check ovn-nbctl --wait=sb remove Logical_Router lr0 option > > > > > dynamic-routing > > > > > +check_row_count Advertised_Route 0 > > > > > + > > > > > +# and setting it again adds them again > > > > > +check ovn-nbctl --wait=sb set Logical_Router lr0 > > > > > option:dynamic-routing=true > > > > > +check_row_count Advertised_Route 3 > > > > > + > > > > > +# removing the lr will remove all routes > > > > > +check ovn-nbctl --wait=sb lr-del lr0 > > > > > +check_row_count Advertised_Route 0 > > > > > + > > > > > +AT_CLEANUP > > > > > +]) > > > > > + > > > > > +OVN_FOR_EACH_NORTHD_NO_HV([ > > > > > +AT_SETUP([dynamic-routing - LB sync to sb IPv6]) > > > > > +AT_KEYWORDS([dynamic-routing]) > > > > > +ovn_start > > > > > + > > > > > +# Start with GW router and a single LRP > > > > > +check ovn-nbctl lr-add lr0 > > > > > +check ovn-nbctl \ > > > > > + -- \ > > > > > + set Logical_Router lr0 options:dynamic-routing=true \ > > > > > + options:chassis=hv1 > > > > > +check ovn-nbctl --wait=sb \ > > > > > + -- \ > > > > > + lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 2001:db0::1/64 > > > > > + > > > > > +check_row_count Advertised_Route 0 > > > > > + > > > > > +datapath=$(ovn-sbctl --bare --columns _uuid list > > > > > datapath_binding lr0) > > > > > +pb=$(ovn-sbctl --bare --columns _uuid list port_binding lr0- > > > > > sw0) > > > > > + > > > > > +# Adding LRP dynamic-routing-lb-vips option and LB VIP rule > > > > > advertises a route entry > > > > > +check ovn-nbctl --wait=sb \ > > > > > + -- \ > > > > > + lrp-set-options lr0-sw0 dynamic-routing-lb-vips=true \ > > > > > + -- \ > > > > > + lb-add lb0 [[2001:db1::10]]:80 > > > > > [[2001:db2::10]]:80,[[2001:db2::11]]:80 \ > > > > > + -- \ > > > > > + lr-lb-add lr0 lb0 > > > > > + > > > > > +ovn-nbctl list Load_Balancer > > > > > +ovn-sbctl list Advertised_Route > > > > > +ovn-sbctl lflow-list > > > > > + > > > > > +check_row_count Advertised_Route 1 > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb], [0], [dnl > > > > > +2001:db1::10/128 > > > > > +]) > > > > > + > > > > > +# Add LR with distributed LRP connected to GW router through > > > > > join LS > > > > > +check ovn-nbctl --wait=sb \ > > > > > + -- \ > > > > > + lrp-add lr0 lr0-join 00:00:00:00:ff:02 2001:db42::1/64 \ > > > > > + -- \ > > > > > + ls-add ls-join \ > > > > > + -- \ > > > > > + lsp-add ls-join lsp-join-to-lr0 \ > > > > > + -- \ > > > > > + lsp-set-type lsp-join-to-lr0 router \ > > > > > + -- \ > > > > > + lsp-set-options lsp-join-to-lr0 router-port=lr0-join \ > > > > > + -- \ > > > > > + lsp-set-addresses lsp-join-to-lr0 router \ > > > > > + -- \ > > > > > + lr-add lr-guest0 \ > > > > > + -- \ > > > > > + lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 > > > > > 2001:db52::1/64 \ > > > > > + -- \ > > > > > + lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 > > > > > 2001:db42::2/64 \ > > > > > + -- \ > > > > > + lrp-set-options lrp-guest0-join dynamic-routing-lb- > > > > > vips=true \ > > > > > + -- \ > > > > > + lsp-add ls-join lsp-join-to-guest0 \ > > > > > + -- \ > > > > > + lsp-set-type lsp-join-to-guest0 router \ > > > > > + -- \ > > > > > + lsp-set-options lsp-join-to-guest0 router-port=lrp- > > > > > guest0-join \ > > > > > + -- \ > > > > > + lrp-set-gateway-chassis lrp-guest0-join hv1 > > > > > + > > > > > +pb2=$(ovn-sbctl --bare --columns _uuid list port_binding > > > > > lrp-guest0-join) > > > > > + > > > > > +check ovn-nbctl --wait=sb \ > > > > > + --add-route lb-add lb1 [[2001:db3::10]]:80 > > > > > [[2001:db4::10]]:80,[[2001:db4::11]]:80 \ > > > > > + -- \ > > > > > + lr-lb-add lr-guest0 lb1 > > > > > + > > > > > +check_row_count Advertised_Route 2 > > > > > +ovn-sbctl list advertised_route > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port="$pb"], [0], [dnl > > > > > +2001:db1::10/128 > > > > > +]) > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb2], [0], [dnl > > > > > +2001:db3::10/128 > > > > > +]) > > > > > + > > > > > +# Add nonlocal LR with distributed LRP connected to GW > > > > > router through join LS > > > > > +check ovn-nbctl --wait=sb \ > > > > > + -- \ > > > > > + lr-add lr-guest1 \ > > > > > + -- \ > > > > > + lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 > > > > > 2001:db2::1/64 \ > > > > > + -- \ > > > > > + lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 > > > > > 2001:db42::3/64 \ > > > > > + -- \ > > > > > + lrp-set-options lrp-guest1-join dynamic-routing-lb- > > > > > vips=true \ > > > > > + -- \ > > > > > + lsp-add ls-join lsp-join-to-guest1 \ > > > > > + -- \ > > > > > + lsp-set-type lsp-join-to-guest1 router \ > > > > > + -- \ > > > > > + lsp-set-options lsp-join-to-guest1 router-port=lrp- > > > > > guest1-join \ > > > > > + -- \ > > > > > + lrp-set-gateway-chassis lrp-guest1-join nonlocalhv > > > > > + > > > > > +pb3=$(ovn-sbctl --bare --columns _uuid list port_binding > > > > > lrp-guest1-join) > > > > > + > > > > > +check ovn-nbctl --wait=sb \ > > > > > + --add-route lb-add lb2 [[2001:db5::10]]:80 > > > > > [[2001:db4::10]]:80,[[2001:db4::11]]:80 \ > > > > > + -- \ > > > > > + lr-lb-add lr-guest1 lb2 > > > > > +check_row_count Advertised_Route 3 > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb], [0], [dnl > > > > > +2001:db1::10/128 > > > > > +]) > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb2], [0], [dnl > > > > > +2001:db3::10/128 > > > > > +]) > > > > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find > > > > > Advertised_Route \ > > > > > +datapath=$datapath logical_port=$pb3], [0], [dnl > > > > > +2001:db5::10/128 > > > > > +]) > > > > > + > > > > > +# removing the option:dynamic-routing removes all routes > > > > > +check ovn-nbctl --wait=sb remove Logical_Router lr0 option > > > > > dynamic-routing > > > > > +check_row_count Advertised_Route 0 > > > > > + > > > > > +# and setting it again adds them again > > > > > +check ovn-nbctl --wait=sb set Logical_Router lr0 > > > > > option:dynamic-routing=true > > > > > +check_row_count Advertised_Route 3 > > > > > + > > > > > +# removing the lr will remove all routes > > > > > +check ovn-nbctl --wait=sb lr-del lr0 > > > > > +check_row_count Advertised_Route 0 > > > > > + > > > > > +AT_CLEANUP > > > > > +]) > > > > > + > > > > > diff --git a/tests/system-ovn.at b/tests/system-ovn.at > > > > > index 5c4364663..918266ad2 100644 > > > > > --- a/tests/system-ovn.at > > > > > +++ b/tests/system-ovn.at > > > > > @@ -16033,3 +16033,382 @@ > > > > > OVS_TRAFFIC_VSWITCHD_STOP(["/.*error receiving.*/d > > > > > AT_CLEANUP > > > > > ]) > > > > > > > > > > +OVN_FOR_EACH_NORTHD([ > > > > > +AT_SETUP([route-exchange for LB VIPs with gateway router > > > > > IPv4]) > > > > > +AT_KEYWORDS([route-exchange]) > > > > > + > > > > > +CHECK_VRF() > > > > > +CHECK_CONNTRACK() > > > > > +CHECK_CONNTRACK_NAT() > > > > > +ovn_start > > > > > +OVS_TRAFFIC_VSWITCHD_START() > > > > > +ADD_BR([br-int]) > > > > > +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone]) > > > > > + > > > > > +# Set external-ids in br-int needed for ovn-controller > > > > > +ovs-vsctl \ > > > > > + -- set Open_vSwitch . external-ids:system-id=hv1 \ > > > > > + -- set Open_vSwitch . external-ids:ovn- > > > > > remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ > > > > > + -- set Open_vSwitch . external-ids:ovn-encap- > > > > > type=geneve \ > > > > > + -- set Open_vSwitch . external-ids:ovn-encap- > > > > > ip=169.0.0.1 \ > > > > > + -- set bridge br-int fail-mode=secure other- > > > > > config:disable-in-band=true > > > > > + > > > > > +# Start ovn-controller > > > > > +start_daemon ovn-controller > > > > > + > > > > > +ovn-appctl vlog/set route_exchange > > > > > +check ovn-nbctl -- lr-add R1 \ > > > > > + -- set Logical_Router R1 options:requested- > > > > > tnl-key=1000 options:dynamic-routing=true > > > > > + > > > > > +check ovn-nbctl ls-add sw0 > > > > > +check ovn-nbctl ls-add public > > > > > + > > > > > +check ovn-nbctl --wait=hv sync > > > > > + > > > > > +AT_CHECK([ip link | grep -q ovnvrf1000:.*UP], [1]) > > > > > + > > > > > +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 > > > > > 192.168.1.1/24 > > > > > +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 > > > > > 172.16.1.1/24 \ > > > > > + -- lrp-set-options rp-public \ > > > > > + dynamic-routing-maintain-vrf=true \ > > > > > + dynamic-routing-lb-vips=true > > > > > + > > > > > +check ovn-nbctl set logical_router R1 options:chassis=hv1 > > > > > + > > > > > +check ovn-nbctl lsp-add sw0 sw0-rp -- set > > > > > Logical_Switch_Port sw0-rp \ > > > > > + type=router options:router-port=rp-sw0 \ > > > > > + -- lsp-set-addresses sw0-rp router > > > > > + > > > > > +check ovn-nbctl lsp-add public public-rp -- set > > > > > Logical_Switch_Port public-rp \ > > > > > + type=router options:router-port=rp-public \ > > > > > + -- lsp-set-addresses public-rp router > > > > > + > > > > > +check ovs-vsctl set Open_vSwitch . external-ids:ovn-bridge- > > > > > mappings=phynet:br-ext > > > > > + > > > > > +check ovn-nbctl lsp-add public public1 \ > > > > > + -- lsp-set-addresses public1 unknown \ > > > > > + -- lsp-set-type public1 localnet \ > > > > > + -- lsp-set-options public1 network_name=phynet > > > > > + > > > > > +check ovn-nbctl --wait=hv sync > > > > > + > > > > > +AT_CHECK([test `ip route show table 1000 | wc -l` -eq 1], > > > > > [1]) > > > > > + > > > > > + > > > > > +# Create a load balancer and associate to R1 > > > > > +check ovn-nbctl lb-add lb1 172.16.1.150:80 172.16.1.100:80 > > > > > +check ovn-nbctl lr-lb-add R1 lb1 > > > > > + > > > > > +check ovn-nbctl --wait=hv sync > > > > > + > > > > > +AT_CHECK([ip link | grep -q ovnvrf1000:.*UP]) > > > > > +AT_CHECK([test `ip route show table 1000 | wc -l` -eq 1]) > > > > > +AT_CHECK([ip route show table 1000 | grep -q 172.16.1.150]) > > > > > + > > > > > +OVS_APP_EXIT_AND_WAIT([ovn-controller]) > > > > > + > > > > > +# Ensure system resources are cleaned up > > > > > +AT_CHECK([ip link | grep -q ovnvrf1000:.*UP], [1]) > > > > > +AT_CHECK([test `ip route show table 1000 | wc -l` -eq 1], > > > > > [1]) > > > > > + > > > > > +as ovn-sb > > > > > +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > > > > > + > > > > > +as ovn-nb > > > > > +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > > > > > + > > > > > +as northd > > > > > +OVS_APP_EXIT_AND_WAIT([ovn-northd]) > > > > > + > > > > > +as > > > > > +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d > > > > > +/Failed to acquire.*/d > > > > > +/connection dropped.*/d"]) > > > > > +AT_CLEANUP > > > > > +]) > > > > > + > > > > > +OVN_FOR_EACH_NORTHD([ > > > > > +AT_SETUP([route-exchange for LB VIPs with gateway router > > > > > IPv6]) > > > > > +AT_KEYWORDS([route-exchange]) > > > > > + > > > > > +CHECK_VRF() > > > > > +CHECK_CONNTRACK() > > > > > +CHECK_CONNTRACK_NAT() > > > > > +ovn_start > > > > > +OVS_TRAFFIC_VSWITCHD_START() > > > > > +ADD_BR([br-int]) > > > > > +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone]) > > > > > + > > > > > +# Set external-ids in br-int needed for ovn-controller > > > > > +ovs-vsctl \ > > > > > + -- set Open_vSwitch . external-ids:system-id=hv1 \ > > > > > + -- set Open_vSwitch . external-ids:ovn- > > > > > remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ > > > > > + -- set Open_vSwitch . external-ids:ovn-encap- > > > > > type=geneve \ > > > > > + -- set Open_vSwitch . external-ids:ovn-encap- > > > > > ip=169.0.0.1 \ > > > > > + -- set bridge br-int fail-mode=secure other- > > > > > config:disable-in-band=true > > > > > + > > > > > +# Start ovn-controller > > > > > +start_daemon ovn-controller > > > > > + > > > > > +ovn-appctl vlog/set route_exchange > > > > > +check ovn-nbctl -- lr-add R1 \ > > > > > + -- set Logical_Router R1 options:requested- > > > > > tnl-key=1001 options:dynamic-routing=true > > > > > + > > > > > +check ovn-nbctl ls-add sw0 > > > > > +check ovn-nbctl ls-add public > > > > > + > > > > > +check ovn-nbctl --wait=hv sync > > > > > + > > > > > +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP], [1]) > > > > > + > > > > > +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 > > > > > 2001:db8:100::1/64 > > > > > +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 > > > > > 2001:db8:1001::1/64 \ > > > > > + -- lrp-set-options rp-public \ > > > > > + dynamic-routing-maintain-vrf=true \ > > > > > + dynamic-routing-lb-vips=true > > > > > + > > > > > +check ovn-nbctl set logical_router R1 options:chassis=hv1 > > > > > + > > > > > +check ovn-nbctl lsp-add sw0 sw0-rp -- set > > > > > Logical_Switch_Port sw0-rp \ > > > > > + type=router options:router-port=rp-sw0 \ > > > > > + -- lsp-set-addresses sw0-rp router > > > > > + > > > > > +check ovn-nbctl lsp-add public public-rp -- set > > > > > Logical_Switch_Port public-rp \ > > > > > + type=router options:router-port=rp-public \ > > > > > + -- lsp-set-addresses public-rp router > > > > > + > > > > > +check ovs-vsctl set Open_vSwitch . external-ids:ovn-bridge- > > > > > mappings=phynet:br-ext > > > > > + > > > > > +check ovn-nbctl lsp-add public public1 \ > > > > > + -- lsp-set-addresses public1 unknown \ > > > > > + -- lsp-set-type public1 localnet \ > > > > > + -- lsp-set-options public1 network_name=phynet > > > > > + > > > > > +check ovn-nbctl --wait=hv sync > > > > > + > > > > > +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1], > > > > > [1]) > > > > > + > > > > > +# Create a load balancer and associate to R1 > > > > > +check ovn-nbctl lb-add lb1 [[2001:db8:1001::150]]:80 > > > > > [[2001:db8:1001::100]]:80 > > > > > +check ovn-nbctl lr-lb-add R1 lb1 > > > > > + > > > > > +check ovn-nbctl --wait=hv sync > > > > > + > > > > > +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP]) > > > > > +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1]) > > > > > +AT_CHECK([ip -6 route show table 1001 | grep -q > > > > > 2001:db8:1001::150]) > > > > > + > > > > > + > > > > > +OVS_APP_EXIT_AND_WAIT([ovn-controller]) > > > > > + > > > > > +# Ensure system resources are cleaned up > > > > > +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP], [1]) > > > > > +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1], > > > > > [1]) > > > > > + > > > > > +as ovn-sb > > > > > +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > > > > > + > > > > > +as ovn-nb > > > > > +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > > > > > + > > > > > +as northd > > > > > +OVS_APP_EXIT_AND_WAIT([ovn-northd]) > > > > > + > > > > > +as > > > > > +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d > > > > > +/Failed to acquire.*/d > > > > > +/connection dropped.*/d"]) > > > > > +AT_CLEANUP > > > > > +]) > > > > > + > > > > > +OVN_FOR_EACH_NORTHD([ > > > > > +AT_SETUP([route-exchange for DNAT and DNAT_AND_SNAT with > > > > > gateway router IPv4]) > > > > > +AT_KEYWORDS([route-exchange]) > > > > > + > > > > > +CHECK_VRF() > > > > > +CHECK_CONNTRACK() > > > > > +CHECK_CONNTRACK_NAT() > > > > > +ovn_start > > > > > +OVS_TRAFFIC_VSWITCHD_START() > > > > > +ADD_BR([br-int]) > > > > > +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone]) > > > > > + > > > > > +# Set external-ids in br-int needed for ovn-controller > > > > > +ovs-vsctl \ > > > > > + -- set Open_vSwitch . external-ids:system-id=hv1 \ > > > > > + -- set Open_vSwitch . external-ids:ovn- > > > > > remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ > > > > > + -- set Open_vSwitch . external-ids:ovn-encap- > > > > > type=geneve \ > > > > > + -- set Open_vSwitch . external-ids:ovn-encap- > > > > > ip=169.0.0.1 \ > > > > > + -- set bridge br-int fail-mode=secure other- > > > > > config:disable-in-band=true > > > > > + > > > > > +# Start ovn-controller > > > > > +start_daemon ovn-controller > > > > > + > > > > > +ovn-appctl vlog/set route_exchange > > > > > +check ovn-nbctl -- lr-add R1 \ > > > > > + -- set Logical_Router R1 options:requested- > > > > > tnl-key=1002 options:dynamic-routing=true > > > > > + > > > > > +check ovn-nbctl ls-add sw0 > > > > > +check ovn-nbctl ls-add public > > > > > + > > > > > +check ovn-nbctl --wait=hv sync > > > > > + > > > > > +AT_CHECK([ip link | grep -q ovnvrf1002:.*UP], [1]) > > > > > + > > > > > +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 > > > > > 192.168.1.1/24 > > > > > +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 > > > > > 172.16.1.1/24 \ > > > > > + -- lrp-set-options rp-public \ > > > > > + dynamic-routing-maintain-vrf=true \ > > > > > + dynamic-routing-nat=true > > > > > + > > > > > +check ovn-nbctl set logical_router R1 options:chassis=hv1 > > > > > + > > > > > +check ovn-nbctl lsp-add sw0 sw0-rp -- set > > > > > Logical_Switch_Port sw0-rp \ > > > > > + type=router options:router-port=rp-sw0 \ > > > > > + -- lsp-set-addresses sw0-rp router > > > > > + > > > > > +check ovn-nbctl lsp-add public public-rp -- set > > > > > Logical_Switch_Port public-rp \ > > > > > + type=router options:router-port=rp-public \ > > > > > + -- lsp-set-addresses public-rp router > > > > > + > > > > > +check ovs-vsctl set Open_vSwitch . external-ids:ovn-bridge- > > > > > mappings=phynet:br-ext > > > > > + > > > > > +check ovn-nbctl lsp-add public public1 \ > > > > > + -- lsp-set-addresses public1 unknown \ > > > > > + -- lsp-set-type public1 localnet \ > > > > > + -- lsp-set-options public1 network_name=phynet > > > > > + > > > > > +check ovn-nbctl --wait=hv sync > > > > > + > > > > > +AT_CHECK([test `ip route show table 1002 | wc -l` -eq 2], > > > > > [1]) > > > > > + > > > > > +# Create dnat_and_snat, dnat rules in R1 > > > > > +check ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.10 > > > > > 192.168.1.10 > > > > > +check ovn-nbctl lr-nat-add R1 dnat 172.16.1.11 192.168.1.11 > > > > > + > > > > > +check ovn-nbctl --wait=hv sync > > > > > + > > > > > +AT_CHECK([ip link | grep -q ovnvrf1002:.*UP]) > > > > > +AT_CHECK([test `ip route show table 1002 | wc -l` -eq 2]) > > > > > +AT_CHECK([ip route show table 1002 | grep -q 172.16.1.10]) > > > > > +AT_CHECK([ip route show table 1002 | grep -q 172.16.1.11]) > > > > > + > > > > > + > > > > > +OVS_APP_EXIT_AND_WAIT([ovn-controller]) > > > > > + > > > > > +# Ensure system resources are cleaned up > > > > > +AT_CHECK([ip link | grep -q ovnvrf1002:.*UP], [1]) > > > > > +AT_CHECK([test `ip route show table 1002 | wc -l` -eq 1], > > > > > [1]) > > > > > + > > > > > +as ovn-sb > > > > > +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > > > > > + > > > > > +as ovn-nb > > > > > +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > > > > > + > > > > > +as northd > > > > > +OVS_APP_EXIT_AND_WAIT([ovn-northd]) > > > > > + > > > > > +as > > > > > +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d > > > > > +/Failed to acquire.*/d > > > > > +/connection dropped.*/d"]) > > > > > +AT_CLEANUP > > > > > +]) > > > > > + > > > > > +OVN_FOR_EACH_NORTHD([ > > > > > +AT_SETUP([route-exchange for DNAT and DNAT_AND_SNAT with > > > > > gateway router IPv6]) > > > > > +AT_KEYWORDS([route-exchange]) > > > > > + > > > > > +CHECK_VRF() > > > > > +CHECK_CONNTRACK() > > > > > +CHECK_CONNTRACK_NAT() > > > > > +ovn_start > > > > > +OVS_TRAFFIC_VSWITCHD_START() > > > > > +ADD_BR([br-int]) > > > > > +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone]) > > > > > + > > > > > +# Set external-ids in br-int needed for ovn-controller > > > > > +ovs-vsctl \ > > > > > + -- set Open_vSwitch . external-ids:system-id=hv1 \ > > > > > + -- set Open_vSwitch . external-ids:ovn- > > > > > remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \ > > > > > + -- set Open_vSwitch . external-ids:ovn-encap- > > > > > type=geneve \ > > > > > + -- set Open_vSwitch . external-ids:ovn-encap- > > > > > ip=169.0.0.1 \ > > > > > + -- set bridge br-int fail-mode=secure other- > > > > > config:disable-in-band=true > > > > > + > > > > > +# Start ovn-controller > > > > > +start_daemon ovn-controller > > > > > + > > > > > +ovn-appctl vlog/set route_exchange > > > > > +check ovn-nbctl -- lr-add R1 \ > > > > > + -- set Logical_Router R1 options:requested- > > > > > tnl-key=1003 options:dynamic-routing=true > > > > > + > > > > > +check ovn-nbctl ls-add sw0 > > > > > +check ovn-nbctl ls-add public > > > > > + > > > > > +check ovn-nbctl --wait=hv sync > > > > > + > > > > > +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP], [1]) > > > > > + > > > > > +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 > > > > > 2001:db8:100::1/64 > > > > > +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 > > > > > 2001:db8:1003::1/64 \ > > > > > + -- lrp-set-options rp-public \ > > > > > + dynamic-routing-maintain-vrf=true \ > > > > > + dynamic-routing-nat=true > > > > > + > > > > > +check ovn-nbctl set logical_router R1 options:chassis=hv1 > > > > > + > > > > > +check ovn-nbctl lsp-add sw0 sw0-rp -- set > > > > > Logical_Switch_Port sw0-rp \ > > > > > + type=router options:router-port=rp-sw0 \ > > > > > + -- lsp-set-addresses sw0-rp router > > > > > + > > > > > +check ovn-nbctl lsp-add public public-rp -- set > > > > > Logical_Switch_Port public-rp \ > > > > > + type=router options:router-port=rp-public \ > > > > > + -- lsp-set-addresses public-rp router > > > > > + > > > > > +check ovs-vsctl set Open_vSwitch . external-ids:ovn-bridge- > > > > > mappings=phynet:br-ext > > > > > + > > > > > +check ovn-nbctl lsp-add public public1 \ > > > > > + -- lsp-set-addresses public1 unknown \ > > > > > + -- lsp-set-type public1 localnet \ > > > > > + -- lsp-set-options public1 network_name=phynet > > > > > + > > > > > +check ovn-nbctl --wait=hv sync > > > > > + > > > > > +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2], > > > > > [1]) > > > > > + > > > > > +# Create dnat_and_snat, dnat rules in R1 > > > > > +check ovn-nbctl lr-nat-add R1 \ > > > > > + dnat_and_snat 2001:db8:1003::150 2001:db8:100::100 > > > > > +check ovn-nbctl lr-nat-add R1 \ > > > > > + dnat 2001:db8:1003::151 2001:db8:100::100 > > > > > + > > > > > +check ovn-nbctl --wait=hv sync > > > > > + > > > > > +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP]) > > > > > +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2]) > > > > > +AT_CHECK([ip -6 route show table 1003 | grep -q > > > > > 2001:db8:1003::150]) > > > > > +AT_CHECK([ip -6 route show table 1003 | grep -q > > > > > 2001:db8:1003::151]) > > > > > + > > > > > +OVS_APP_EXIT_AND_WAIT([ovn-controller]) > > > > > + > > > > > +# Ensure system resources are cleaned up > > > > > +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP], [1]) > > > > > +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2], > > > > > [1]) > > > > > + > > > > > +as ovn-sb > > > > > +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > > > > > + > > > > > +as ovn-nb > > > > > +OVS_APP_EXIT_AND_WAIT([ovsdb-server]) > > > > > + > > > > > +as northd > > > > > +OVS_APP_EXIT_AND_WAIT([ovn-northd]) > > > > > + > > > > > +as > > > > > +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d > > > > > +/Failed to acquire.*/d > > > > > +/connection dropped.*/d"]) > > > > > +AT_CLEANUP > > > > > +]) > > > > > + > > > > > > > > I didn't review the tests yet but I do plan to. I wanted to > > > > start the > > > > discussion about the rest of the implementation first though. > > > > I'd love > > > > it if we could get this feature in 25.03.0. Thanks for the thorough review. The review of tests could be probably postponed 'till the next version, as there will be some changes. Mainly regarding the NAT/LB IPs of the neighboring routers being announced only via "connected-as-host-routes" option Martin. > > > > > > > > Regards, > > > > Dumitru > > > > > > _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
