On 2/6/25 10:38 PM, Dumitru Ceara wrote:
> On 2/5/25 10:21 AM, Martin Kalcok wrote:
>> This change builds on top of the new "dynamic routing" OVN feature
>> that allows advertising routes to the fabric network. When LR option
>> "dynamic-routing" is set on the router, following two new LRP options
>> become available:
>>
>> * dynamic-routing-nat - When set to "true", ovn-controller will advertise
>>                         routes for external NAT IPs valid for the LRP.
>> * dynamic-routing-lb-vips - When set to "true", ovn-controller will advertise
>>                             host routes to LB VIPs via the LRP.
>>
>> Co-authored-by: Frode Nordahl <[email protected]>
>> Signed-off-by: Frode Nordahl <[email protected]>
>> Signed-off-by: Martin Kalcok <[email protected]>
>> ---
> 
> Hi Martin, Frode,
> 
> Martin, I know you had this patchset up for review for quite a long time
> and I apologize for not finding time before to closely look at the
> implementation.
> 
> I now have some concerns related to both desing and also scalability.
> 
> I know the incremental processing engine in northd (or OVN in general)
> is not that developer friendly but we do already have the information
> that's needed for the changes you're making in the en_routes node:
> 
> - load balancers applied on each router (directly or through lb groups)
> are parsed in the lb_data I-P node
> - nats are parsed in the lr_nat I-P node
> 
> This also gets propagated in the lr_stateful I-P node data:
> 
> struct lr_stateful_record {
> ...
>     /* UUID of the NB Logical Router. */
>     struct uuid nbr_uuid;
> ...
>     /* This lrnat_rec comes from the en_lrnat engine node data. */
>     const struct lr_nat_record *lrnat_rec;
> 
>     bool has_lb_vip;
> 
>     /* Load Balancer vIPs relevant for this datapath. */
>     struct ovn_lb_ip_set *lb_ips;
> ...
> }
> 
> It's true that we don't have per LB IP information that tells us which
> LB that IP came from but if needed we could try to add that too.
> 
> So we shouldn't have to re-parse NB.NAT/NB.Load_Balancer records.  I'm
> detailing this concern lower in publish_lb_route() but it also applies
> to publish_nat_route().
> 
>>  NEWS                              |  17 +
>>  northd/en-advertised-route-sync.c | 198 +++++++++++
>>  northd/en-northd.c                |   5 +-
>>  northd/inc-proc-northd.c          |   5 +
>>  northd/northd.c                   | 134 ++++++-
>>  northd/northd.h                   |   8 +-
>>  ovn-nb.xml                        |  32 ++
>>  ovs                               |   2 +-
>>  tests/ovn-northd.at               | 556 ++++++++++++++++++++++++++++++
>>  tests/system-ovn.at               | 379 ++++++++++++++++++++
>>  10 files changed, 1332 insertions(+), 4 deletions(-)
>>
>> diff --git a/NEWS b/NEWS
>> index eca17b728..137ff48fd 100644
>> --- a/NEWS
>> +++ b/NEWS
>> @@ -61,6 +61,23 @@ Post v24.09.0
>>       * Add the option "dynamic-routing-ifname" to LRPs. If set only routes
>>         learned from a linux iterfaces with that name are treated as relevant
>>         routes for this LRP.
>> +   - Add the option "dynamic-routing" to Logical Routers. If set to true all
>> +     static and connected routes attached to the router are shared to the
>> +     southbound "Route" table for sharing outside of OVN.
>> +     The routes can furthe be filtered by setting 
>> `dynamic-routing-connected`
>> +     and `dynamic-routing-static` on the LR or LRP. The LRP settings 
>> overwrite
>> +     the LR settings for all routes using this interface as an exit.
>> +   - Allow Logical Routers to dynamically learn routes from outside the 
>> fabric.
>> +     Routes entered into the "Route" table in the southbound database will 
>> be
>> +     learned by the respective LR. They are included in the route table with
>> +     a lower priority than static routes.
>> +   - Add the option "dynamic-routing-connected-as-host-routes" to LRPs. If 
>> set
>> +     to true then connected routes are announced as individual host routes.
>> +   - Add 'dynamic-routing-lb-vips' LRP option. If set to true, the LRP can 
>> be
>> +     used to advertise host paths to the Load Balancer VIPs associated with 
>> the
>> +     LR.
>> +   - Add 'dynamic-routing-nat' LRP option. If set to true, the LRP can be 
>> used
>> +     to advertise external NAT IPs associated with it.
>>  
>>  OVN v24.09.0 - 13 Sep 2024
>>  --------------------------
>> diff --git a/northd/en-advertised-route-sync.c 
>> b/northd/en-advertised-route-sync.c
>> index 02b3aba7a..b6b759201 100644
>> --- a/northd/en-advertised-route-sync.c
>> +++ b/northd/en-advertised-route-sync.c
>> @@ -16,6 +16,7 @@
>>  
>>  #include <config.h>
>>  
>> +#include "openvswitch/vlog.h"
>>  #include "smap.h"
>>  #include "stopwatch.h"
>>  #include "northd.h"
>> @@ -26,6 +27,8 @@
>>  #include "openvswitch/hmap.h"
>>  #include "ovn-util.h"
>>  
>> +VLOG_DEFINE_THIS_MODULE(en_advertised_route_sync);
>> +
>>  static void
>>  advertised_route_table_sync(
>>      struct ovsdb_idl_txn *ovnsb_txn,
>> @@ -312,6 +315,185 @@ publish_host_routes(struct hmap *sync_routes,
>>      }
>>  }
>>  
>> +/* This function searches for an ovn_port with specific "op_ip"
>> + * IP address in all LS datapaths directly connected to the
>> + * LR datapath "od".
>> + * If no match is found, this function returns NULL.*/
>> +static struct ovn_port *
>> +find_port_in_connected_ls(struct ovn_datapath *od, char *op_ip)
>> +{
>> +    struct in6_addr *ip_addr = xmalloc(sizeof *ip_addr);
>> +    unsigned int plen;
>> +    bool is_ipv4 = true;
>> +
>> +    /* Return immediately if op_ip isn't a host route.*/
> 
> We shouldn't have to check this, it should be the caller doing it.  We
> shouldn't parse values coming directly from the database here.  We
> should receive an already correct struct in6_addr * instead as argument.
> 
>> +    if (!ip46_parse_cidr(op_ip, ip_addr, &plen)) {
>> +        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
>> +        VLOG_WARN_RL(&rl, "bad IP for OVN port: %s", op_ip);
>> +        free(ip_addr);
>> +        return NULL;
>> +    }
>> +    if (IN6_IS_ADDR_V4MAPPED(ip_addr) && plen != 32) {
>> +        if (plen != 32) {
>> +            free(ip_addr);
>> +            return NULL;
>> +        }
>> +    } else {
>> +        is_ipv4 = false;
>> +        if (plen != 128) {
>> +            free(ip_addr);
>> +            return NULL;
>> +        }
>> +    }
>> +    free(ip_addr);
>> +
>> +    for (int i = 0; i < od->n_ls_peers; i++) {
> 
> Nit: size_t (in a few places)
> 
>> +        struct ovn_datapath *peer_od = od->ls_peers[i];
>> +        struct ovn_port *op;
>> +        HMAP_FOR_EACH (op, dp_node, &peer_od->ports) {
>> +            for (int j = 0; j < op->n_lsp_addrs; j++) {
>> +                struct lport_addresses *addrs = &op->lsp_addrs[j];
>> +                if (is_ipv4) {
>> +                    for (int k = 0; k < addrs->n_ipv4_addrs; k++) {
>> +                        if (!strcmp(op_ip, addrs->ipv4_addrs[k].addr_s)) {
>> +                            return op;
>> +                        }
>> +                    }
>> +                } else {
>> +                    for (int k = 0; k < addrs->n_ipv6_addrs; k++) {
>> +                        if (!strcmp(op_ip, addrs->ipv6_addrs[k].addr_s)) {
>> +                            return op;
>> +                        }
>> +                    }
>> +                }
>> +            }
>> +        }
>> +    }
> 
> 'struct ovn_datapath' has the following field:
> 
> struct sset router_ips; /* Router port IPs except the IPv6 LLAs. */
> 
> Instead of walking all ports of all connected switches of 'od' maybe we
> should also store all switch port IPs inside 'struct ovn_datapath'.
> Then we'd just have to do a sset_find() for each connected switch.
> 
> We're calling this function for each IP of a NAT/LB we're advertising (i
> expect quite a few of those) so all these linear lookups quickly become
> very costly.
> 
>> +    return NULL;
>> +}
>> +
>> +/* Publish parsed_route as a Advertised_Route record. The main purpose
>> + * of this function is to find LSP with IP address that matches the
>> + * logical_ip of a NAT rule on the route's out_port datapath. This port is
>> + * then set as a tracked_port for the route.
>> + * Search is performed in all LS datapaths directly connected to the
>> + * route's out_port datapath.
>> + * If no suitable tracked_port is found, route is still published, but
>> + * without the tracked_port set.*/
>> +static void
>> +publish_nat_route(struct hmap *route_map,
>> +                  const struct parsed_route *route)
>> +{
>> +    const struct ovn_datapath *advertised_dp = route->od;
>> +    const struct ovn_port *ext_nat_port = route->out_port;
>> +
>> +    if (!advertised_dp->nbr || !ext_nat_port->od) {
>> +        return;
>> +    }
>> +
>> +    char *ip_prefix = normalize_v46_prefix(&route->prefix,
>> +                                           route->plen);
>> +    char *logical_ip = NULL;
>> +
>> +    /* Find NAT rule with external IP that matches the route's
>> +     * ip_prefix.*/
>> +    for (int i = 0; i < ext_nat_port->od->nbr->n_nat; i++) {
> 
> Nit: size_t
> 
>> +        struct nbrec_nat *nat = ext_nat_port->od->nbr->nat[i];
>> +        if (!strcmp(nat->external_ip, ip_prefix)) {
>> +            logical_ip = nat->logical_ip;
>> +            break;
>> +        }
>> +    }
>> +
>> +    if (!logical_ip) {
>> +        return;
>> +    }
>> +
>> +    const struct sbrec_port_binding *tracked_pb = NULL;
>> +    struct ovn_port *tracked_port = 
>> find_port_in_connected_ls(ext_nat_port->od,
>> +                                                              logical_ip);
>> +    if (tracked_port) {
>> +        tracked_pb = tracked_port->sb;
>> +    }
>> +
>> +    char *ip_with_mask = xasprintf("%s/%d", ip_prefix, route->plen);
>> +    ar_alloc_entry(route_map,
>> +                  route->od->sb,
>> +                  route->out_port->sb,
>> +                  ip_with_mask,
>> +                  tracked_pb);
>> +    free(ip_prefix);
>> +}
>> +
>> +/* Publish parsed_route as a Advertised_Route record. The main purpose
>> + * of this function is to find LSP with IP address that matches the
>> + * endpoint IP of a LB on the route's out_port datapath. This port is then
>> + * set as a tracked_port for the route.
>> + * Search is performed in all LS datapaths directly connected to the
>> + * route's out_port datapath.
>> + * If no suitable tracked_port is found, route is still published, but
>> + * without the tracked_port set.*/
>> +static void
>> +publish_lb_route(struct hmap *route_map,
>> +                 const struct parsed_route *route)
>> +{
>> +    const struct ovn_datapath *advertised_dp = route->od;
>> +    const struct ovn_port *ext_lb_port = route->out_port;
>> +
>> +    if (!advertised_dp->nbr || !ext_lb_port->od) {
>> +        return;
>> +    }
>> +
>> +    char *ip_prefix = normalize_v46_prefix(&route->prefix,
>> +                                           route->plen);
>> +
>> +    for (int i = 0; i < ext_lb_port->od->nbr->n_load_balancer; i++) {
> 
> Nit: size_t
> 
> Also, this is not correct/complete.  Load balancers can also be applied
> through load balancer groups.  So you'd have to also iterate on
> od->nbr->load_balancer_groups.
> 
> However, the publish_lb_route() receives an argument a parsed_route that
> has been built based on a load balancer VIP address.
> 
> Now we're iterating through all load balancers configured on the route's
> datapath to try to find one that has the same VIP as what was used when
> the route was built.  We actually do more than that, we actually reparse
> each and every load balancer attached to that router.
> 
> That seems very inefficient.  We should find a way to store this
> information in the route structure itself at creation time because it
> gets built from a load balancer.
> 

Re-reading this sentence I realized it might not be that clear.  I meant:

1. based on information computed when parsing load balancer
configuration (lb_data -> lr_stateful_rec)
2. we create 'struct parsed_route' records - the calls to
parsed_routes_add_lb()
3. then later, if the route was created from a load balancer (type ==
ROUTE_SOURCE_LB) we try to go back and find the load balancer
information that triggered the route creation (at step "2").

We should, when doing step "2", store a pointer to the parsed LB
information we'll be needing at step "3".

> This comment also applies to the publish_nat_route() function.
> 
>> +        struct nbrec_load_balancer *lb =
>> +            ext_lb_port->od->nbr->load_balancer[i];
>> +        struct smap lb_vips = SMAP_INITIALIZER(&lb_vips);
>> +        struct smap_node *node;
>> +        SMAP_FOR_EACH (node, &lb->vips) {
>> +            struct ovn_lb_vip *lb_vip = xmalloc(sizeof(*lb_vip));
>> +            char *error = ovn_lb_vip_init(lb_vip, node->key,
>> +                                          node->value, false, AF_INET6);
>> +            if (error) {
>> +                static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 
>> 1);
>> +                VLOG_WARN_RL(&rl, "Failed to parse LB VIP: %s", error);
>> +                ovn_lb_vip_destroy(lb_vip);
>> +                free(error);
>> +                continue;
>> +            }
>> +
>> +            /* Continue only for LBs whose VIP matches route's ip_prefix.*/
>> +            if (strcmp(lb_vip->vip_str, ip_prefix)) {
>> +                ovn_lb_vip_destroy(lb_vip);
>> +                continue;
>> +            }
>> +
>> +            /* Find LSP that matches at least one endpoint IP of the LB.*/
>> +            const struct sbrec_port_binding *tracked_pb = NULL;
>> +            for (int j = 0; j < lb_vip->n_backends; j++) {
> 
> Nit: size_t
> 
>> +                char *backend = lb_vip->backends[j].ip_str;
>> +                struct ovn_port *tracked_port =
>> +                    find_port_in_connected_ls(ext_lb_port->od, backend);
> 
> This is a function call but it hides a LOT of complexity (see the
> comment above on the function definition).
> 
>> +                if (tracked_port) {
>> +                    tracked_pb = tracked_port->sb;
>> +                    break;
> 
> This makes me wonder why we even use tracked_port for load balancers.
> Load balancer backends could be distributed across different physical nodes.
> 
> Why advertise the route only with one port?
> 
> OR, why not advertise the route with all ports who own backend IPs?
> 
>> +                }
>> +            }
>> +
>> +            char *ip_with_mask = xasprintf("%s/%d", ip_prefix, route->plen);
>> +            ar_alloc_entry(route_map,
>> +                           route->od->sb,
>> +                           route->out_port->sb,
>> +                           ip_with_mask,
>> +                           tracked_pb);
>> +            ovn_lb_vip_destroy(lb_vip);
>> +            free(ip_prefix);
>> +        }
>> +    }
>> +}
>> +
>>  static void
>>  advertised_route_table_sync(
>>      struct ovsdb_idl_txn *ovnsb_txn,
>> @@ -359,6 +541,22 @@ advertised_route_table_sync(
>>                  !route->out_port->dynamic_routing_static) {
>>              continue;
>>          }
>> +        if (route->source == ROUTE_SOURCE_NAT) {
>> +            if (!smap_get_bool(&route->out_port->nbrp->options,
>> +                               "dynamic-routing-nat", false)) {
>> +                continue;
>> +            }
>> +            publish_nat_route(&sync_routes, route);
>> +            continue;
>> +        }
>> +        if (route->source == ROUTE_SOURCE_LB) {
>> +            if (!smap_get_bool(&route->out_port->nbrp->options,
>> +                               "dynamic-routing-lb-vips", false)) {
>> +                continue;
>> +            }
>> +            publish_lb_route(&sync_routes, route);
>> +            continue;
>> +        }
>>  
>>          char *ip_prefix = normalize_v46_prefix(&route->prefix,
>>                                                 route->plen);
>> diff --git a/northd/en-northd.c b/northd/en-northd.c
>> index c7d1ebcb3..60e295628 100644
>> --- a/northd/en-northd.c
>> +++ b/northd/en-northd.c
>> @@ -316,6 +316,8 @@ en_routes_run(struct engine_node *node, void *data)
>>  {
>>      struct northd_data *northd_data = engine_get_input_data("northd", node);
>>      struct bfd_data *bfd_data = engine_get_input_data("bfd", node);
>> +    struct ed_type_lr_stateful *lr_stateful_data =
>> +        engine_get_input_data("lr_stateful", node);
>>      struct routes_data *routes_data = data;
>>  
>>      routes_destroy(data);
>> @@ -330,7 +332,8 @@ en_routes_run(struct engine_node *node, void *data)
>>                                 route_table_name);
>>          }
>>  
>> -        build_parsed_routes(od, &northd_data->lr_ports,
>> +        build_parsed_routes(od, &lr_stateful_data->table,
>> +                            &northd_data->lr_ports,
>>                              &bfd_data->bfd_connections,
>>                              &routes_data->parsed_routes,
>>                              &routes_data->route_tables,
>> diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c
>> index 86b7b999e..d74c2e3dc 100644
>> --- a/northd/inc-proc-northd.c
>> +++ b/northd/inc-proc-northd.c
>> @@ -267,6 +267,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb,
>>      engine_add_input(&en_routes, &en_bfd, NULL);
>>      engine_add_input(&en_routes, &en_northd,
>>                       routes_northd_change_handler);
>> +    engine_add_input(&en_routes, &en_lr_nat,
>> +                     NULL);
>> +    engine_add_input(&en_routes, &en_lb_data,
>> +                     NULL);
>> +    engine_add_input(&en_routes, &en_lr_stateful, engine_noop_handler);
> 
> As mentioned in my other reply, this is a performance regression and I
> don't really think we need the en_lr_nat and en_lb_data nodes as direct
> inputs for en_routes:
> 
> https://mail.openvswitch.org/pipermail/ovs-dev/2025-February/420765.html
> 
>>  
>>      engine_add_input(&en_bfd_sync, &en_bfd, NULL);
>>      engine_add_input(&en_bfd_sync, &en_nb_bfd, NULL);
>> diff --git a/northd/northd.c b/northd/northd.c
>> index 9f8d3118c..bff69e545 100644
>> --- a/northd/northd.c
>> +++ b/northd/northd.c
>> @@ -11076,8 +11076,111 @@ parsed_routes_add_connected(const struct 
>> ovn_datapath *od,
>>      }
>>  }
>>  
>> +/* Add "parsed_route"s to the "routes" for each IP address in laddrs.*/
>> +static void
>> +lport_addrs_to_parsed_routes(const struct ovn_datapath *od,
>> +                             const struct ovn_port *op,
>> +                             const struct lport_addresses *laddrs,
>> +                             enum route_source route_type,
>> +                             struct hmap *routes,
>> +                             bool install_lflow)
>> +{
>> +    for (int i = 0; i < laddrs->n_ipv4_addrs; i++) {
> 
> Nit: size_t
> 
>> +        struct ipv4_netaddr *addr = &laddrs->ipv4_addrs[i];
>> +        struct in6_addr prefix;
>> +        ip46_parse(addr->network_s, &prefix);
>> +        parsed_route_add(od, NULL, &prefix, addr->plen,
>> +                         false, addr->addr_s, op,
>> +                         0, false,
>> +                         false, NULL, route_type,
>> +                         &op->nbrp->header_, routes, install_lflow);
>> +    }
>> +    for (int i = 0; i < laddrs->n_ipv6_addrs; i++) {
> 
> Nit: size_t
> 
>> +        struct ipv6_netaddr *addr = &laddrs->ipv6_addrs[i];
>> +        parsed_route_add(od, NULL, &addr->addr, addr->plen,
>> +                         false, addr->addr_s, op,
>> +                         0, false,
>> +                         false, NULL, route_type,
>> +                         &op->nbrp->header_, routes, install_lflow);
>> +    }
>> +}
>> +
>> +/* Add routes of all NAT external_addresses on the ovn_port "op", to be
>> + * advertised by the datapath "od".
>> + * If "routeable_only" is true, only NATs with option "add_route=true" will
>> + * be considered.
>> + * Note that the "op" does not necessarily need to be in the "od" datapath.
>> + * Such use-case is for advertising external NAT IPs of LRPs in neighboring
>> + * routers and should be paired with routable_only=true.
>> + */
>> +static void
>> +parsed_routes_add_nat(const struct ovn_datapath *od,
>> +                      const struct ovn_port *op,
>> +                      struct hmap *routes,
>> +                      bool routable_only)
>> +{
>> +    if (!op->nbrp || !smap_get_bool(&op->nbrp->options,
>> +                                    "dynamic-routing-nat", false)) {
>> +        return;
>> +    }
>> +    size_t n_nats = 0;
>> +    char **nats = NULL;
>> +    nats = get_nat_addresses(op, &n_nats, routable_only, false, NULL, true);
>> +
>> +    for (size_t i = 0; i < n_nats; i++) {
>> +        struct lport_addresses *laddrs = xzalloc(sizeof *laddrs);
> 
> We leak this struct.  Actually, we don't need to xalloc() this, it could
> just be a variable on the stack AFAICT.
> 
>> +        int ofs = 0;
>> +        extract_addresses(nats[i], laddrs, &ofs);
>> +        lport_addrs_to_parsed_routes(od, op, laddrs, ROUTE_SOURCE_NAT,
>> +                                     routes, false);
>> +        destroy_lport_addresses(laddrs);
>> +        free(nats[i]);
>> +    }
>> +    free(nats);
>> +}
>> +
>> +/* Add routes of all LB VIPs on the ovn_port "op", to be advertised by the
>> + * datapath "od".
>> + * If "routeable_only" is true, only LBs with option "add_route=true" will
>> + * be considered.
>> + * Note that the "op" does not necessarily need to be in the "od" datapath.
>> + * Such use-case is for advertising LB VIPs of LRPs in neighboring routers
>> + * and should be paired with routable_only=true.
>> + */
>> +static void
>> +parsed_routes_add_lb(const struct ovn_datapath *od,
>> +                     const struct ovn_port *op,
>> +                     struct hmap *routes,
>> +                     const struct lr_stateful_record *lr_stateful_rec,
>> +                     bool routable_only)
>> +{
>> +    if (!op->nbrp || !smap_get_bool(&op->nbrp->options,
>> +                                    "dynamic-routing-lb-vips", false)) {
>> +        return;
>> +    }
>> +
>> +    if (!lr_stateful_rec) {
>> +        return;
>> +    }
>> +
>> +    struct ds addresses = DS_EMPTY_INITIALIZER;
>> +    get_lb_addresses(lr_stateful_rec, &addresses, routable_only);
>> +
>> +    struct lport_addresses *laddrs = xzalloc(sizeof *laddrs);
> 
> Same here, we leak laddrs.
> 
>> +    extract_ip_addresses(ds_cstr(&addresses), laddrs);
>> +
>> +    lport_addrs_to_parsed_routes(od, op, laddrs, ROUTE_SOURCE_LB,
>> +                                 routes, false);
>> +
>> +    destroy_lport_addresses(laddrs);
>> +    ds_destroy(&addresses);
>> +
>> +}
>> +
>>  void
>> -build_parsed_routes(const struct ovn_datapath *od, const struct hmap 
>> *lr_ports,
>> +build_parsed_routes(const struct ovn_datapath *od,
>> +                    const struct lr_stateful_table *lr_stateful_table,
>> +                    const struct hmap *lr_ports,
>>                      const struct hmap *bfd_connections, struct hmap *routes,
>>                      struct simap *route_tables,
>>                      struct hmap *bfd_active_connections)
>> @@ -11097,7 +11200,34 @@ build_parsed_routes(const struct ovn_datapath *od, 
>> const struct hmap *lr_ports,
>>  
>>      const struct ovn_port *op;
>>      HMAP_FOR_EACH (op, dp_node, &od->ports) {
>> +        const struct lr_stateful_record *lr_stateful_rec = NULL;
>> +        lr_stateful_rec = lr_stateful_table_find_by_index(
>> +            lr_stateful_table, op->od->index);
>> +
>>          parsed_routes_add_connected(od, op, routes);
>> +        parsed_routes_add_nat(od, op, routes, false);
>> +        parsed_routes_add_lb(od, op, routes, lr_stateful_rec, false);
>> +    }
>> +
>> +    /* For GW routers we traverse also neighboring LR in search for
>> +     * LBs and NATs with "add_route" option set to "true". Such NATs/LBs
>> +     * install logical flows for their external IP addresses to their
>> +     * neighboring routers, and as such, we can advertise them as routes.
>> +     * The LRP on the neighboring router needs to have appropriate
>> +     * "dynamic-routing-{nat,lb-vips}" option set for this to take effect.*/
>> +    for (size_t i = 0; od->is_gw_router && i < od->n_ls_peers; i++) {
> 
> What about direct router peers?  Two logical routers can be directly
> attached.
> 
>> +        for (size_t j = 0; j < od->ls_peers[i]->n_router_ports; j++) {
>> +            struct ovn_port *router_port;
>> +            router_port = od->ls_peers[i]->router_ports[j]->peer;
>> +
>> +            const struct lr_stateful_record *lr_stateful_rec = NULL;
>> +            lr_stateful_rec = lr_stateful_table_find_by_index(
>> +                lr_stateful_table, router_port->od->index);
>> +
>> +            parsed_routes_add_nat(od, router_port, routes, true);
>> +            parsed_routes_add_lb(od, router_port, routes,
>> +                                 lr_stateful_rec, true);
>> +        }
>>      }
>>  
>>      HMAP_FOR_EACH_SAFE (pr, key_node, routes) {
>> @@ -11279,6 +11409,8 @@ route_source_to_offset(enum route_source source)
>>  {
>>      switch (source) {
>>      case ROUTE_SOURCE_CONNECTED:
>> +    case ROUTE_SOURCE_NAT:
>> +    case ROUTE_SOURCE_LB:
>>          return ROUTE_PRIO_OFFSET_CONNECTED;
>>      case ROUTE_SOURCE_STATIC:
>>          return ROUTE_PRIO_OFFSET_STATIC;
>> diff --git a/northd/northd.h b/northd/northd.h
>> index fffaab3a0..d55909f76 100644
>> --- a/northd/northd.h
>> +++ b/northd/northd.h
>> @@ -708,6 +708,10 @@ enum route_source {
>>      ROUTE_SOURCE_CONNECTED,
>>      /* The route is derived from a northbound static route entry. */
>>      ROUTE_SOURCE_STATIC,
>> +    /* Host route generated from NAT's external IP. */
>> +    ROUTE_SOURCE_NAT,
>> +    /* Host route generated from LB's external IP. */
>> +    ROUTE_SOURCE_LB,
>>      /* The route is dynamically learned by an ovn-controller. */
>>      ROUTE_SOURCE_LEARNED,
>>  };
>> @@ -780,7 +784,9 @@ void northd_indices_create(struct northd_data *data,
>>  
>>  void route_policies_init(struct route_policies_data *);
>>  void route_policies_destroy(struct route_policies_data *);
>> -void build_parsed_routes(const struct ovn_datapath *, const struct hmap *,
>> +void build_parsed_routes(const struct ovn_datapath *,
>> +                         const struct lr_stateful_table *lr_stateful_table,
>> +                         const struct hmap *,
>>                           const struct hmap *, struct hmap *, struct simap *,
>>                           struct hmap *);
>>  uint32_t get_route_table_id(struct simap *, const char *);
>> diff --git a/ovn-nb.xml b/ovn-nb.xml
>> index 734fcf2bd..e8d9d5491 100644
>> --- a/ovn-nb.xml
>> +++ b/ovn-nb.xml
>> @@ -2971,6 +2971,10 @@ or
>>                   table="Logical_Router_Port"/></li>
>>          <li><ref column="options" key="dynamic-routing-static"
>>                   table="Logical_Router_Port"/></li>
>> +        <li><ref column="options" key="dynamic-routing-lb-vips"
>> +                 table="Logical_Router_Port"/></li>
>> +        <li><ref column="options" key="dynamic-routing-nat"
>> +                 table="Logical_Router_Port"/></li>
>>          </ul>
>>        </column>
>>  
>> @@ -3828,6 +3832,34 @@ or
>>          This option allows to have a 1:1 mapping between a single LRP and an
>>          individual link.
>>        </column>
>> +
>> +      <column name="options" key="dynamic-routing-lb-vips"
>> +              type='{"type": "boolean"}'>
>> +        <p>
>> +          Only relevant if <ref column="options" key="dynamic-routing"
>> +          table="Logical_Router"/> on the respective Logical_Router is set
>> +          to <code>true</code>.
>> +
>> +          If this option is <code>true</code>, northd will create host route
>> +          entries in the southbound <ref table="Advertised_Route"
>> +          db="OVN_Southbound"/> table, associated with this LRP, for each LB
>> +          VIP.
>> +        </p>
>> +      </column>
>> +
>> +      <column name="options" key="dynamic-routing-nat"
>> +              type='{"type": "boolean"}'>
>> +        <p>
>> +          Only relevant if <ref column="options" key="dynamic-routing"
>> +          table="Logical_Router"/> on the respective Logical_Router is set
>> +          to <code>true</code>.
>> +
>> +          If this option is <code>true</code>, northd will create host route
>> +          entries in the southbound <ref table="Advertised_Route"
>> +          db="OVN_Southbound"/> table, for external IP addresses of NAT 
>> rules
>> +          associated with this LRP.
>> +        </p>
>> +      </column>
>>      </group>
>>  
>>      <group title="Attachment">
>> diff --git a/ovs b/ovs
>> index a3c06c309..c648ee626 160000
>> --- a/ovs
>> +++ b/ovs
>> @@ -1 +1 @@
>> -Subproject commit a3c06c309ab9f21fd9ce4e8dca542119d46be9c5
>> +Subproject commit c648ee626efdc959ed0cff37c2b9128816450f15
> 
> This ovs submodule change is accidental I guess.
> 
>> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
>> index 44dc92869..bad9ca0b5 100644
>> --- a/tests/ovn-northd.at
>> +++ b/tests/ovn-northd.at
>> @@ -15134,3 +15134,559 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE
>>  AT_CLEANUP
>>  ])
>>  
>> +OVN_FOR_EACH_NORTHD_NO_HV([
>> +AT_SETUP([dynamic-routing - nat sync to sb IPv4])
>> +AT_KEYWORDS([dynamic-routing])
>> +ovn_start
>> +
>> +# Start with GW router and a single LRP
>> +check ovn-nbctl lr-add lr0
>> +check ovn-nbctl \
>> +    -- \
>> +    set Logical_Router lr0 options:dynamic-routing=true \
>> +                           options:chassis=hv1
>> +check ovn-nbctl --wait=sb \
>> +    -- \
>> +    lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
>> +
>> +check_row_count Advertised_Route 0
>> +
>> +datapath=$(ovn-sbctl --bare --columns _uuid list datapath_binding lr0)
>> +pb=$(ovn-sbctl --bare --columns _uuid list port_binding lr0-sw0)
>> +
>> +# Adding LRP dynamic-routing-nat option and NAT rule advertises a route 
>> entry
>> +check ovn-nbctl --wait=sb \
>> +    -- \
>> +    lrp-set-options lr0-sw0 dynamic-routing-nat=true \
>> +    -- \
>> +    lr-nat-add lr0 dnat_and_snat 172.16.1.10 192.168.1.10
>> +
>> +ovn-nbctl list NAT
>> +ovn-sbctl list Advertised_Route
>> +ovn-sbctl lflow-list
>> +
>> +check_row_count Advertised_Route 1
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb], [0], [dnl
>> +172.16.1.10/32
>> +])
>> +
>> +# Add LR with distributed LRP connected to GW router through join LS
>> +check ovn-nbctl --wait=sb \
>> +    -- \
>> +    lrp-add lr0 lr0-join 00:00:00:00:ff:02 10.42.0.1/24 \
>> +    -- \
>> +    ls-add ls-join \
>> +    -- \
>> +    lsp-add ls-join lsp-join-to-lr0 \
>> +    -- \
>> +    lsp-set-type lsp-join-to-lr0 router \
>> +    -- \
>> +    lsp-set-options lsp-join-to-lr0 router-port=lr0-join \
>> +    -- \
>> +    lsp-set-addresses lsp-join-to-lr0 router \
>> +    -- \
>> +    lr-add lr-guest0 \
>> +    -- \
>> +    lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 10.51.0.1/24 \
>> +    -- \
>> +    lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 10.42.0.2/24 \
>> +    -- \
>> +    lrp-set-options lrp-guest0-join dynamic-routing-nat=true \
>> +    -- \
>> +    lsp-add ls-join lsp-join-to-guest0 \
>> +    -- \
>> +    lsp-set-type lsp-join-to-guest0 router \
>> +    -- \
>> +    lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \
>> +    -- \
>> +    lrp-set-gateway-chassis lrp-guest0-join hv1
>> +
>> +pb2=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest0-join)
>> +
>> +check ovn-nbctl --wait=sb \
>> +    --add-route lr-nat-add lr-guest0 dnat_and_snat 172.16.2.10 192.168.2.10
>> +
>> +check_row_count Advertised_Route 2
>> +ovn-sbctl list advertised_route
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port="$pb"], [0], [dnl
>> +172.16.1.10/32
>> +])
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb2], [0], [dnl
>> +172.16.2.10/32
>> +])
>> +
>> +# Add nonlocal LR with distributed LRP connected to GW router through join 
>> LS
>> +check ovn-nbctl --wait=sb \
>> +    -- \
>> +    lr-add lr-guest1 \
>> +    -- \
>> +    lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 10.51.1.1/24 \
>> +    -- \
>> +    lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 10.42.0.3/24 \
>> +    -- \
>> +    lrp-set-options lrp-guest1-join dynamic-routing-nat=true \
>> +    -- \
>> +    lsp-add ls-join lsp-join-to-guest1 \
>> +    -- \
>> +    lsp-set-type lsp-join-to-guest1 router \
>> +    -- \
>> +    lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \
>> +    -- \
>> +    lrp-set-gateway-chassis lrp-guest1-join nonlocalhv
>> +
>> +pb3=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest1-join)
>> +
>> +check ovn-nbctl --wait=sb \
>> +    --add-route lr-nat-add lr-guest1 dnat_and_snat 172.16.3.10 192.168.3.10
>> +check_row_count Advertised_Route 3
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb], [0], [dnl
>> +172.16.1.10/32
>> +])
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb2], [0], [dnl
>> +172.16.2.10/32
>> +])
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb3], [0], [dnl
>> +172.16.3.10/32
>> +])
>> +
>> +# removing the option:dynamic-routing removes all routes
>> +check ovn-nbctl --wait=sb remove Logical_Router lr0 option dynamic-routing
>> +check_row_count Advertised_Route 0
>> +
>> +# and setting it again adds them again
>> +check ovn-nbctl --wait=sb set Logical_Router lr0 option:dynamic-routing=true
>> +check_row_count Advertised_Route 3
>> +
>> +# removing the lr will remove all routes
>> +check ovn-nbctl --wait=sb lr-del lr0
>> +check_row_count Advertised_Route 0
>> +
>> +AT_CLEANUP
>> +])
>> +
>> +OVN_FOR_EACH_NORTHD_NO_HV([
>> +AT_SETUP([dynamic-routing - nat sync to sb IPv6])
>> +AT_KEYWORDS([dynamic-routing])
>> +ovn_start
>> +
>> +# Start with GW router and a single LRP
>> +check ovn-nbctl lr-add lr0
>> +check ovn-nbctl \
>> +    -- \
>> +    set Logical_Router lr0 options:dynamic-routing=true \
>> +                           options:chassis=hv1
>> +check ovn-nbctl --wait=sb \
>> +    -- \
>> +    lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 2001:db0::1/64
>> +
>> +check_row_count Advertised_Route 0
>> +
>> +datapath=$(ovn-sbctl --bare --columns _uuid list datapath_binding lr0)
>> +pb=$(ovn-sbctl --bare --columns _uuid list port_binding lr0-sw0)
>> +
>> +# Adding LRP dynamic-routing-nat option and NAT rule advertises a route 
>> entry
>> +check ovn-nbctl --wait=sb \
>> +    -- \
>> +    lrp-set-options lr0-sw0 dynamic-routing-nat=true \
>> +    -- \
>> +    lr-nat-add lr0 dnat_and_snat 2001:db1::10 2001:db2::10
>> +
>> +ovn-nbctl list NAT
>> +ovn-sbctl list Advertised_Route
>> +ovn-sbctl lflow-list
>> +
>> +check_row_count Advertised_Route 1
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb], [0], [dnl
>> +2001:db1::10/128
>> +])
>> +
>> +# Add LR with distributed LRP connected to GW router through join LS
>> +check ovn-nbctl --wait=sb \
>> +    -- \
>> +    lrp-add lr0 lr0-join 00:00:00:00:ff:02 2001:db42::1/64 \
>> +    -- \
>> +    ls-add ls-join \
>> +    -- \
>> +    lsp-add ls-join lsp-join-to-lr0 \
>> +    -- \
>> +    lsp-set-type lsp-join-to-lr0 router \
>> +    -- \
>> +    lsp-set-options lsp-join-to-lr0 router-port=lr0-join \
>> +    -- \
>> +    lsp-set-addresses lsp-join-to-lr0 router \
>> +    -- \
>> +    lr-add lr-guest0 \
>> +    -- \
>> +    lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 2001:db51::1/64 \
>> +    -- \
>> +    lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 2001:db42::2/64 \
>> +    -- \
>> +    lrp-set-options lrp-guest0-join dynamic-routing-nat=true \
>> +    -- \
>> +    lsp-add ls-join lsp-join-to-guest0 \
>> +    -- \
>> +    lsp-set-type lsp-join-to-guest0 router \
>> +    -- \
>> +    lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \
>> +    -- \
>> +    lrp-set-gateway-chassis lrp-guest0-join hv1
>> +
>> +pb2=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest0-join)
>> +
>> +check ovn-nbctl --wait=sb \
>> +    --add-route lr-nat-add lr-guest0 dnat_and_snat 2001:db3::10 2001:db4::10
>> +
>> +check_row_count Advertised_Route 2
>> +ovn-sbctl list advertised_route
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port="$pb"], [0], [dnl
>> +2001:db1::10/128
>> +])
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb2], [0], [dnl
>> +2001:db3::10/128
>> +])
>> +
>> +# Add nonlocal LR with distributed LRP connected to GW router through join 
>> LS
>> +check ovn-nbctl --wait=sb \
>> +    -- \
>> +    lr-add lr-guest1 \
>> +    -- \
>> +    lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 2001:db52::1/64 \
>> +    -- \
>> +    lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 2001:db42::3/64 \
>> +    -- \
>> +    lrp-set-options lrp-guest1-join dynamic-routing-nat=true \
>> +    -- \
>> +    lsp-add ls-join lsp-join-to-guest1 \
>> +    -- \
>> +    lsp-set-type lsp-join-to-guest1 router \
>> +    -- \
>> +    lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \
>> +    -- \
>> +    lrp-set-gateway-chassis lrp-guest1-join nonlocalhv
>> +
>> +pb3=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest1-join)
>> +
>> +check ovn-nbctl --wait=sb \
>> +    --add-route lr-nat-add lr-guest1 dnat_and_snat 2001:db5::10 2001:db6::10
>> +check_row_count Advertised_Route 3
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb], [0], [dnl
>> +2001:db1::10/128
>> +])
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb2], [0], [dnl
>> +2001:db3::10/128
>> +])
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb3], [0], [dnl
>> +2001:db5::10/128
>> +])
>> +
>> +# removing the option:dynamic-routing removes all routes
>> +check ovn-nbctl --wait=sb remove Logical_Router lr0 option dynamic-routing
>> +check_row_count Advertised_Route 0
>> +
>> +# and setting it again adds them again
>> +check ovn-nbctl --wait=sb set Logical_Router lr0 option:dynamic-routing=true
>> +check_row_count Advertised_Route 3
>> +
>> +# removing the lr will remove all routes
>> +check ovn-nbctl --wait=sb lr-del lr0
>> +check_row_count Advertised_Route 0
>> +
>> +AT_CLEANUP
>> +])
>> +
>> +OVN_FOR_EACH_NORTHD_NO_HV([
>> +AT_SETUP([dynamic-routing - LB sync to sb IPv4])
>> +AT_KEYWORDS([dynamic-routing])
>> +ovn_start
>> +
>> +# Start with GW router and a single LRP
>> +check ovn-nbctl lr-add lr0
>> +check ovn-nbctl \
>> +    -- \
>> +    set Logical_Router lr0 options:dynamic-routing=true \
>> +                           options:chassis=hv1
>> +check ovn-nbctl --wait=sb \
>> +    -- \
>> +    lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
>> +
>> +check_row_count Advertised_Route 0
>> +
>> +datapath=$(ovn-sbctl --bare --columns _uuid list datapath_binding lr0)
>> +pb=$(ovn-sbctl --bare --columns _uuid list port_binding lr0-sw0)
>> +
>> +# Adding LRP dynamic-routing-lb-vips option and LB VIP rule advertises a 
>> route entry
>> +check ovn-nbctl --wait=sb \
>> +    -- \
>> +    lrp-set-options lr0-sw0 dynamic-routing-lb-vips=true \
>> +    -- \
>> +    lb-add lb0 172.16.1.10:80 192.168.1.10:80,192.168.1.11:80 \
>> +    -- \
>> +    lr-lb-add lr0 lb0
>> +
>> +ovn-nbctl list Load_Balancer
>> +ovn-sbctl list Advertised_Route
>> +ovn-sbctl lflow-list
>> +
>> +check_row_count Advertised_Route 1
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb], [0], [dnl
>> +172.16.1.10/32
>> +])
>> +
>> +# Add LR with distributed LRP connected to GW router through join LS
>> +check ovn-nbctl --wait=sb \
>> +    -- \
>> +    lrp-add lr0 lr0-join 00:00:00:00:ff:02 10.42.0.1/24 \
>> +    -- \
>> +    ls-add ls-join \
>> +    -- \
>> +    lsp-add ls-join lsp-join-to-lr0 \
>> +    -- \
>> +    lsp-set-type lsp-join-to-lr0 router \
>> +    -- \
>> +    lsp-set-options lsp-join-to-lr0 router-port=lr0-join \
>> +    -- \
>> +    lsp-set-addresses lsp-join-to-lr0 router \
>> +    -- \
>> +    lr-add lr-guest0 \
>> +    -- \
>> +    lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 10.51.0.1/24 \
>> +    -- \
>> +    lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 10.42.0.2/24 \
>> +    -- \
>> +    lrp-set-options lrp-guest0-join dynamic-routing-lb-vips=true \
>> +    -- \
>> +    lsp-add ls-join lsp-join-to-guest0 \
>> +    -- \
>> +    lsp-set-type lsp-join-to-guest0 router \
>> +    -- \
>> +    lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \
>> +    -- \
>> +    lrp-set-gateway-chassis lrp-guest0-join hv1
>> +
>> +pb2=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest0-join)
>> +
>> +check ovn-nbctl --wait=sb \
>> +    --add-route lb-add lb1 172.16.2.10:80 192.168.2.10:80,192.168.2.11:80 \
>> +    -- \
>> +    lr-lb-add lr-guest0 lb1
>> +
>> +check_row_count Advertised_Route 2
>> +ovn-sbctl list advertised_route
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port="$pb"], [0], [dnl
>> +172.16.1.10/32
>> +])
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb2], [0], [dnl
>> +172.16.2.10/32
>> +])
>> +
>> +# Add nonlocal LR with distributed LRP connected to GW router through join 
>> LS
>> +check ovn-nbctl --wait=sb \
>> +    -- \
>> +    lr-add lr-guest1 \
>> +    -- \
>> +    lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 10.51.1.1/24 \
>> +    -- \
>> +    lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 10.42.0.3/24 \
>> +    -- \
>> +    lrp-set-options lrp-guest1-join dynamic-routing-lb-vips=true \
>> +    -- \
>> +    lsp-add ls-join lsp-join-to-guest1 \
>> +    -- \
>> +    lsp-set-type lsp-join-to-guest1 router \
>> +    -- \
>> +    lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \
>> +    -- \
>> +    lrp-set-gateway-chassis lrp-guest1-join nonlocalhv
>> +
>> +pb3=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest1-join)
>> +
>> +check ovn-nbctl --wait=sb \
>> +    --add-route lb-add lb2 172.16.3.10:80 192.168.3.10:80,192.168.3.11:80 \
>> +    -- \
>> +    lr-lb-add lr-guest1 lb2
>> +check_row_count Advertised_Route 3
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb], [0], [dnl
>> +172.16.1.10/32
>> +])
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb2], [0], [dnl
>> +172.16.2.10/32
>> +])
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb3], [0], [dnl
>> +172.16.3.10/32
>> +])
>> +
>> +# removing the option:dynamic-routing removes all routes
>> +check ovn-nbctl --wait=sb remove Logical_Router lr0 option dynamic-routing
>> +check_row_count Advertised_Route 0
>> +
>> +# and setting it again adds them again
>> +check ovn-nbctl --wait=sb set Logical_Router lr0 option:dynamic-routing=true
>> +check_row_count Advertised_Route 3
>> +
>> +# removing the lr will remove all routes
>> +check ovn-nbctl --wait=sb lr-del lr0
>> +check_row_count Advertised_Route 0
>> +
>> +AT_CLEANUP
>> +])
>> +
>> +OVN_FOR_EACH_NORTHD_NO_HV([
>> +AT_SETUP([dynamic-routing - LB sync to sb IPv6])
>> +AT_KEYWORDS([dynamic-routing])
>> +ovn_start
>> +
>> +# Start with GW router and a single LRP
>> +check ovn-nbctl lr-add lr0
>> +check ovn-nbctl \
>> +    -- \
>> +    set Logical_Router lr0 options:dynamic-routing=true \
>> +                           options:chassis=hv1
>> +check ovn-nbctl --wait=sb \
>> +    -- \
>> +    lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 2001:db0::1/64
>> +
>> +check_row_count Advertised_Route 0
>> +
>> +datapath=$(ovn-sbctl --bare --columns _uuid list datapath_binding lr0)
>> +pb=$(ovn-sbctl --bare --columns _uuid list port_binding lr0-sw0)
>> +
>> +# Adding LRP dynamic-routing-lb-vips option and LB VIP rule advertises a 
>> route entry
>> +check ovn-nbctl --wait=sb \
>> +    -- \
>> +    lrp-set-options lr0-sw0 dynamic-routing-lb-vips=true \
>> +    -- \
>> +    lb-add lb0 [[2001:db1::10]]:80 [[2001:db2::10]]:80,[[2001:db2::11]]:80 \
>> +    -- \
>> +    lr-lb-add lr0 lb0
>> +
>> +ovn-nbctl list Load_Balancer
>> +ovn-sbctl list Advertised_Route
>> +ovn-sbctl lflow-list
>> +
>> +check_row_count Advertised_Route 1
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb], [0], [dnl
>> +2001:db1::10/128
>> +])
>> +
>> +# Add LR with distributed LRP connected to GW router through join LS
>> +check ovn-nbctl --wait=sb \
>> +    -- \
>> +    lrp-add lr0 lr0-join 00:00:00:00:ff:02 2001:db42::1/64 \
>> +    -- \
>> +    ls-add ls-join \
>> +    -- \
>> +    lsp-add ls-join lsp-join-to-lr0 \
>> +    -- \
>> +    lsp-set-type lsp-join-to-lr0 router \
>> +    -- \
>> +    lsp-set-options lsp-join-to-lr0 router-port=lr0-join \
>> +    -- \
>> +    lsp-set-addresses lsp-join-to-lr0 router \
>> +    -- \
>> +    lr-add lr-guest0 \
>> +    -- \
>> +    lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 2001:db52::1/64 \
>> +    -- \
>> +    lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 2001:db42::2/64 \
>> +    -- \
>> +    lrp-set-options lrp-guest0-join dynamic-routing-lb-vips=true \
>> +    -- \
>> +    lsp-add ls-join lsp-join-to-guest0 \
>> +    -- \
>> +    lsp-set-type lsp-join-to-guest0 router \
>> +    -- \
>> +    lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \
>> +    -- \
>> +    lrp-set-gateway-chassis lrp-guest0-join hv1
>> +
>> +pb2=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest0-join)
>> +
>> +check ovn-nbctl --wait=sb \
>> +    --add-route lb-add lb1 [[2001:db3::10]]:80 
>> [[2001:db4::10]]:80,[[2001:db4::11]]:80 \
>> +    -- \
>> +    lr-lb-add lr-guest0 lb1
>> +
>> +check_row_count Advertised_Route 2
>> +ovn-sbctl list advertised_route
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port="$pb"], [0], [dnl
>> +2001:db1::10/128
>> +])
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb2], [0], [dnl
>> +2001:db3::10/128
>> +])
>> +
>> +# Add nonlocal LR with distributed LRP connected to GW router through join 
>> LS
>> +check ovn-nbctl --wait=sb \
>> +    -- \
>> +    lr-add lr-guest1 \
>> +    -- \
>> +    lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 2001:db2::1/64 \
>> +    -- \
>> +    lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 2001:db42::3/64 \
>> +    -- \
>> +    lrp-set-options lrp-guest1-join dynamic-routing-lb-vips=true \
>> +    -- \
>> +    lsp-add ls-join lsp-join-to-guest1 \
>> +    -- \
>> +    lsp-set-type lsp-join-to-guest1 router \
>> +    -- \
>> +    lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \
>> +    -- \
>> +    lrp-set-gateway-chassis lrp-guest1-join nonlocalhv
>> +
>> +pb3=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest1-join)
>> +
>> +check ovn-nbctl --wait=sb \
>> +    --add-route lb-add lb2 [[2001:db5::10]]:80 
>> [[2001:db4::10]]:80,[[2001:db4::11]]:80 \
>> +    -- \
>> +    lr-lb-add lr-guest1 lb2
>> +check_row_count Advertised_Route 3
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb], [0], [dnl
>> +2001:db1::10/128
>> +])
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb2], [0], [dnl
>> +2001:db3::10/128
>> +])
>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>> +datapath=$datapath logical_port=$pb3], [0], [dnl
>> +2001:db5::10/128
>> +])
>> +
>> +# removing the option:dynamic-routing removes all routes
>> +check ovn-nbctl --wait=sb remove Logical_Router lr0 option dynamic-routing
>> +check_row_count Advertised_Route 0
>> +
>> +# and setting it again adds them again
>> +check ovn-nbctl --wait=sb set Logical_Router lr0 option:dynamic-routing=true
>> +check_row_count Advertised_Route 3
>> +
>> +# removing the lr will remove all routes
>> +check ovn-nbctl --wait=sb lr-del lr0
>> +check_row_count Advertised_Route 0
>> +
>> +AT_CLEANUP
>> +])
>> +
>> diff --git a/tests/system-ovn.at b/tests/system-ovn.at
>> index 5c4364663..918266ad2 100644
>> --- a/tests/system-ovn.at
>> +++ b/tests/system-ovn.at
>> @@ -16033,3 +16033,382 @@ OVS_TRAFFIC_VSWITCHD_STOP(["/.*error receiving.*/d
>>  AT_CLEANUP
>>  ])
>>  
>> +OVN_FOR_EACH_NORTHD([
>> +AT_SETUP([route-exchange for LB VIPs with gateway router IPv4])
>> +AT_KEYWORDS([route-exchange])
>> +
>> +CHECK_VRF()
>> +CHECK_CONNTRACK()
>> +CHECK_CONNTRACK_NAT()
>> +ovn_start
>> +OVS_TRAFFIC_VSWITCHD_START()
>> +ADD_BR([br-int])
>> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone])
>> +
>> +# Set external-ids in br-int needed for ovn-controller
>> +ovs-vsctl \
>> +        -- set Open_vSwitch . external-ids:system-id=hv1 \
>> +        -- set Open_vSwitch . 
>> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
>> +        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
>> +        -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
>> +        -- set bridge br-int fail-mode=secure 
>> other-config:disable-in-band=true
>> +
>> +# Start ovn-controller
>> +start_daemon ovn-controller
>> +
>> +ovn-appctl vlog/set route_exchange
>> +check ovn-nbctl -- lr-add R1 \
>> +                -- set Logical_Router R1 options:requested-tnl-key=1000 
>> options:dynamic-routing=true
>> +
>> +check ovn-nbctl ls-add sw0
>> +check ovn-nbctl ls-add public
>> +
>> +check ovn-nbctl --wait=hv sync
>> +
>> +AT_CHECK([ip link | grep -q ovnvrf1000:.*UP], [1])
>> +
>> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 192.168.1.1/24
>> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 172.16.1.1/24 \
>> +                -- lrp-set-options rp-public \
>> +                       dynamic-routing-maintain-vrf=true \
>> +                       dynamic-routing-lb-vips=true
>> +
>> +check ovn-nbctl set logical_router R1 options:chassis=hv1
>> +
>> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \
>> +    type=router options:router-port=rp-sw0 \
>> +    -- lsp-set-addresses sw0-rp router
>> +
>> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port 
>> public-rp \
>> +    type=router options:router-port=rp-public \
>> +    -- lsp-set-addresses public-rp router
>> +
>> +check ovs-vsctl set Open_vSwitch . 
>> external-ids:ovn-bridge-mappings=phynet:br-ext
>> +
>> +check ovn-nbctl lsp-add public public1 \
>> +        -- lsp-set-addresses public1 unknown \
>> +        -- lsp-set-type public1 localnet \
>> +        -- lsp-set-options public1 network_name=phynet
>> +
>> +check ovn-nbctl --wait=hv sync
>> +
>> +AT_CHECK([test `ip route show table 1000 | wc -l` -eq 1], [1])
>> +
>> +
>> +# Create a load balancer and associate to R1
>> +check ovn-nbctl lb-add lb1 172.16.1.150:80 172.16.1.100:80
>> +check ovn-nbctl lr-lb-add R1 lb1
>> +
>> +check ovn-nbctl --wait=hv sync
>> +
>> +AT_CHECK([ip link | grep -q ovnvrf1000:.*UP])
>> +AT_CHECK([test `ip route show table 1000 | wc -l` -eq 1])
>> +AT_CHECK([ip route show table 1000 | grep -q 172.16.1.150])
>> +
>> +OVS_APP_EXIT_AND_WAIT([ovn-controller])
>> +
>> +# Ensure system resources are cleaned up
>> +AT_CHECK([ip link | grep -q ovnvrf1000:.*UP], [1])
>> +AT_CHECK([test `ip route show table 1000 | wc -l` -eq 1], [1])
>> +
>> +as ovn-sb
>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>> +
>> +as ovn-nb
>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>> +
>> +as northd
>> +OVS_APP_EXIT_AND_WAIT([ovn-northd])
>> +
>> +as
>> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
>> +/Failed to acquire.*/d
>> +/connection dropped.*/d"])
>> +AT_CLEANUP
>> +])
>> +
>> +OVN_FOR_EACH_NORTHD([
>> +AT_SETUP([route-exchange for LB VIPs with gateway router IPv6])
>> +AT_KEYWORDS([route-exchange])
>> +
>> +CHECK_VRF()
>> +CHECK_CONNTRACK()
>> +CHECK_CONNTRACK_NAT()
>> +ovn_start
>> +OVS_TRAFFIC_VSWITCHD_START()
>> +ADD_BR([br-int])
>> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone])
>> +
>> +# Set external-ids in br-int needed for ovn-controller
>> +ovs-vsctl \
>> +        -- set Open_vSwitch . external-ids:system-id=hv1 \
>> +        -- set Open_vSwitch . 
>> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
>> +        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
>> +        -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
>> +        -- set bridge br-int fail-mode=secure 
>> other-config:disable-in-band=true
>> +
>> +# Start ovn-controller
>> +start_daemon ovn-controller
>> +
>> +ovn-appctl vlog/set route_exchange
>> +check ovn-nbctl -- lr-add R1 \
>> +                -- set Logical_Router R1 options:requested-tnl-key=1001 
>> options:dynamic-routing=true
>> +
>> +check ovn-nbctl ls-add sw0
>> +check ovn-nbctl ls-add public
>> +
>> +check ovn-nbctl --wait=hv sync
>> +
>> +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP], [1])
>> +
>> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 2001:db8:100::1/64
>> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 
>> 2001:db8:1001::1/64 \
>> +                -- lrp-set-options rp-public \
>> +                       dynamic-routing-maintain-vrf=true \
>> +                       dynamic-routing-lb-vips=true
>> +
>> +check ovn-nbctl set logical_router R1 options:chassis=hv1
>> +
>> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \
>> +    type=router options:router-port=rp-sw0 \
>> +    -- lsp-set-addresses sw0-rp router
>> +
>> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port 
>> public-rp \
>> +    type=router options:router-port=rp-public \
>> +    -- lsp-set-addresses public-rp router
>> +
>> +check ovs-vsctl set Open_vSwitch . 
>> external-ids:ovn-bridge-mappings=phynet:br-ext
>> +
>> +check ovn-nbctl lsp-add public public1 \
>> +        -- lsp-set-addresses public1 unknown \
>> +        -- lsp-set-type public1 localnet \
>> +        -- lsp-set-options public1 network_name=phynet
>> +
>> +check ovn-nbctl --wait=hv sync
>> +
>> +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1], [1])
>> +
>> +# Create a load balancer and associate to R1
>> +check ovn-nbctl lb-add lb1 [[2001:db8:1001::150]]:80 
>> [[2001:db8:1001::100]]:80
>> +check ovn-nbctl lr-lb-add R1 lb1
>> +
>> +check ovn-nbctl --wait=hv sync
>> +
>> +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP])
>> +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1])
>> +AT_CHECK([ip -6 route show table 1001 | grep -q 2001:db8:1001::150])
>> +
>> +
>> +OVS_APP_EXIT_AND_WAIT([ovn-controller])
>> +
>> +# Ensure system resources are cleaned up
>> +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP], [1])
>> +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1], [1])
>> +
>> +as ovn-sb
>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>> +
>> +as ovn-nb
>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>> +
>> +as northd
>> +OVS_APP_EXIT_AND_WAIT([ovn-northd])
>> +
>> +as
>> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
>> +/Failed to acquire.*/d
>> +/connection dropped.*/d"])
>> +AT_CLEANUP
>> +])
>> +
>> +OVN_FOR_EACH_NORTHD([
>> +AT_SETUP([route-exchange for DNAT and DNAT_AND_SNAT with gateway router 
>> IPv4])
>> +AT_KEYWORDS([route-exchange])
>> +
>> +CHECK_VRF()
>> +CHECK_CONNTRACK()
>> +CHECK_CONNTRACK_NAT()
>> +ovn_start
>> +OVS_TRAFFIC_VSWITCHD_START()
>> +ADD_BR([br-int])
>> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone])
>> +
>> +# Set external-ids in br-int needed for ovn-controller
>> +ovs-vsctl \
>> +        -- set Open_vSwitch . external-ids:system-id=hv1 \
>> +        -- set Open_vSwitch . 
>> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
>> +        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
>> +        -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
>> +        -- set bridge br-int fail-mode=secure 
>> other-config:disable-in-band=true
>> +
>> +# Start ovn-controller
>> +start_daemon ovn-controller
>> +
>> +ovn-appctl vlog/set route_exchange
>> +check ovn-nbctl -- lr-add R1 \
>> +                -- set Logical_Router R1 options:requested-tnl-key=1002 
>> options:dynamic-routing=true
>> +
>> +check ovn-nbctl ls-add sw0
>> +check ovn-nbctl ls-add public
>> +
>> +check ovn-nbctl --wait=hv sync
>> +
>> +AT_CHECK([ip link | grep -q ovnvrf1002:.*UP], [1])
>> +
>> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 192.168.1.1/24
>> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 172.16.1.1/24 \
>> +                -- lrp-set-options rp-public \
>> +                       dynamic-routing-maintain-vrf=true \
>> +                       dynamic-routing-nat=true
>> +
>> +check ovn-nbctl set logical_router R1 options:chassis=hv1
>> +
>> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \
>> +    type=router options:router-port=rp-sw0 \
>> +    -- lsp-set-addresses sw0-rp router
>> +
>> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port 
>> public-rp \
>> +    type=router options:router-port=rp-public \
>> +    -- lsp-set-addresses public-rp router
>> +
>> +check ovs-vsctl set Open_vSwitch . 
>> external-ids:ovn-bridge-mappings=phynet:br-ext
>> +
>> +check ovn-nbctl lsp-add public public1 \
>> +        -- lsp-set-addresses public1 unknown \
>> +        -- lsp-set-type public1 localnet \
>> +        -- lsp-set-options public1 network_name=phynet
>> +
>> +check ovn-nbctl --wait=hv sync
>> +
>> +AT_CHECK([test `ip route show table 1002 | wc -l` -eq 2], [1])
>> +
>> +# Create dnat_and_snat, dnat rules in R1
>> +check ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.10 192.168.1.10
>> +check ovn-nbctl lr-nat-add R1 dnat 172.16.1.11 192.168.1.11
>> +
>> +check ovn-nbctl --wait=hv sync
>> +
>> +AT_CHECK([ip link | grep -q ovnvrf1002:.*UP])
>> +AT_CHECK([test `ip route show table 1002 | wc -l` -eq 2])
>> +AT_CHECK([ip route show table 1002 | grep -q 172.16.1.10])
>> +AT_CHECK([ip route show table 1002 | grep -q 172.16.1.11])
>> +
>> +
>> +OVS_APP_EXIT_AND_WAIT([ovn-controller])
>> +
>> +# Ensure system resources are cleaned up
>> +AT_CHECK([ip link | grep -q ovnvrf1002:.*UP], [1])
>> +AT_CHECK([test `ip route show table 1002 | wc -l` -eq 1], [1])
>> +
>> +as ovn-sb
>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>> +
>> +as ovn-nb
>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>> +
>> +as northd
>> +OVS_APP_EXIT_AND_WAIT([ovn-northd])
>> +
>> +as
>> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
>> +/Failed to acquire.*/d
>> +/connection dropped.*/d"])
>> +AT_CLEANUP
>> +])
>> +
>> +OVN_FOR_EACH_NORTHD([
>> +AT_SETUP([route-exchange for DNAT and DNAT_AND_SNAT with gateway router 
>> IPv6])
>> +AT_KEYWORDS([route-exchange])
>> +
>> +CHECK_VRF()
>> +CHECK_CONNTRACK()
>> +CHECK_CONNTRACK_NAT()
>> +ovn_start
>> +OVS_TRAFFIC_VSWITCHD_START()
>> +ADD_BR([br-int])
>> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone])
>> +
>> +# Set external-ids in br-int needed for ovn-controller
>> +ovs-vsctl \
>> +        -- set Open_vSwitch . external-ids:system-id=hv1 \
>> +        -- set Open_vSwitch . 
>> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
>> +        -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
>> +        -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
>> +        -- set bridge br-int fail-mode=secure 
>> other-config:disable-in-band=true
>> +
>> +# Start ovn-controller
>> +start_daemon ovn-controller
>> +
>> +ovn-appctl vlog/set route_exchange
>> +check ovn-nbctl -- lr-add R1 \
>> +                -- set Logical_Router R1 options:requested-tnl-key=1003 
>> options:dynamic-routing=true
>> +
>> +check ovn-nbctl ls-add sw0
>> +check ovn-nbctl ls-add public
>> +
>> +check ovn-nbctl --wait=hv sync
>> +
>> +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP], [1])
>> +
>> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 2001:db8:100::1/64
>> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 
>> 2001:db8:1003::1/64 \
>> +                -- lrp-set-options rp-public \
>> +                       dynamic-routing-maintain-vrf=true \
>> +                       dynamic-routing-nat=true
>> +
>> +check ovn-nbctl set logical_router R1 options:chassis=hv1
>> +
>> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \
>> +    type=router options:router-port=rp-sw0 \
>> +    -- lsp-set-addresses sw0-rp router
>> +
>> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port 
>> public-rp \
>> +    type=router options:router-port=rp-public \
>> +    -- lsp-set-addresses public-rp router
>> +
>> +check ovs-vsctl set Open_vSwitch . 
>> external-ids:ovn-bridge-mappings=phynet:br-ext
>> +
>> +check ovn-nbctl lsp-add public public1 \
>> +        -- lsp-set-addresses public1 unknown \
>> +        -- lsp-set-type public1 localnet \
>> +        -- lsp-set-options public1 network_name=phynet
>> +
>> +check ovn-nbctl --wait=hv sync
>> +
>> +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2], [1])
>> +
>> +# Create dnat_and_snat, dnat rules in R1
>> +check ovn-nbctl lr-nat-add R1 \
>> +    dnat_and_snat 2001:db8:1003::150 2001:db8:100::100
>> +check ovn-nbctl lr-nat-add R1 \
>> +    dnat 2001:db8:1003::151 2001:db8:100::100
>> +
>> +check ovn-nbctl --wait=hv sync
>> +
>> +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP])
>> +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2])
>> +AT_CHECK([ip -6 route show table 1003 | grep -q 2001:db8:1003::150])
>> +AT_CHECK([ip -6 route show table 1003 | grep -q 2001:db8:1003::151])
>> +
>> +OVS_APP_EXIT_AND_WAIT([ovn-controller])
>> +
>> +# Ensure system resources are cleaned up
>> +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP], [1])
>> +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2], [1])
>> +
>> +as ovn-sb
>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>> +
>> +as ovn-nb
>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>> +
>> +as northd
>> +OVS_APP_EXIT_AND_WAIT([ovn-northd])
>> +
>> +as
>> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
>> +/Failed to acquire.*/d
>> +/connection dropped.*/d"])
>> +AT_CLEANUP
>> +])
>> +
> 
> I didn't review the tests yet but I do plan to.  I wanted to start the
> discussion about the rest of the implementation first though.  I'd love
> it if we could get this feature in 25.03.0.
> 
> Regards,
> Dumitru
> 

_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Reply via email to