On 2/6/25 10:45 PM, Dumitru Ceara wrote:
> On 2/6/25 10:38 PM, Dumitru Ceara wrote:
>> On 2/5/25 10:21 AM, Martin Kalcok wrote:
>>> This change builds on top of the new "dynamic routing" OVN feature
>>> that allows advertising routes to the fabric network. When LR option
>>> "dynamic-routing" is set on the router, following two new LRP options
>>> become available:
>>>
>>> * dynamic-routing-nat - When set to "true", ovn-controller will advertise
>>> routes for external NAT IPs valid for the LRP.
>>> * dynamic-routing-lb-vips - When set to "true", ovn-controller will
>>> advertise
>>> host routes to LB VIPs via the LRP.
>>>
>>> Co-authored-by: Frode Nordahl <[email protected]>
>>> Signed-off-by: Frode Nordahl <[email protected]>
>>> Signed-off-by: Martin Kalcok <[email protected]>
>>> ---
>>
>> Hi Martin, Frode,
>>
>> Martin, I know you had this patchset up for review for quite a long time
>> and I apologize for not finding time before to closely look at the
>> implementation.
>>
>> I now have some concerns related to both desing and also scalability.
>>
>> I know the incremental processing engine in northd (or OVN in general)
>> is not that developer friendly but we do already have the information
>> that's needed for the changes you're making in the en_routes node:
>>
>> - load balancers applied on each router (directly or through lb groups)
>> are parsed in the lb_data I-P node
>> - nats are parsed in the lr_nat I-P node
>>
>> This also gets propagated in the lr_stateful I-P node data:
>>
>> struct lr_stateful_record {
>> ...
>> /* UUID of the NB Logical Router. */
>> struct uuid nbr_uuid;
>> ...
>> /* This lrnat_rec comes from the en_lrnat engine node data. */
>> const struct lr_nat_record *lrnat_rec;
>>
>> bool has_lb_vip;
>>
>> /* Load Balancer vIPs relevant for this datapath. */
>> struct ovn_lb_ip_set *lb_ips;
>> ...
>> }
>>
>> It's true that we don't have per LB IP information that tells us which
>> LB that IP came from but if needed we could try to add that too.
>>
>> So we shouldn't have to re-parse NB.NAT/NB.Load_Balancer records. I'm
>> detailing this concern lower in publish_lb_route() but it also applies
>> to publish_nat_route().
>>
Looking at the tests I realized that we don't cover any of the
"tracked_port" related cases. So I have another question/suggestion below.
>>> NEWS | 17 +
>>> northd/en-advertised-route-sync.c | 198 +++++++++++
>>> northd/en-northd.c | 5 +-
>>> northd/inc-proc-northd.c | 5 +
>>> northd/northd.c | 134 ++++++-
>>> northd/northd.h | 8 +-
>>> ovn-nb.xml | 32 ++
>>> ovs | 2 +-
>>> tests/ovn-northd.at | 556 ++++++++++++++++++++++++++++++
>>> tests/system-ovn.at | 379 ++++++++++++++++++++
>>> 10 files changed, 1332 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/NEWS b/NEWS
>>> index eca17b728..137ff48fd 100644
>>> --- a/NEWS
>>> +++ b/NEWS
>>> @@ -61,6 +61,23 @@ Post v24.09.0
>>> * Add the option "dynamic-routing-ifname" to LRPs. If set only routes
>>> learned from a linux iterfaces with that name are treated as
>>> relevant
>>> routes for this LRP.
>>> + - Add the option "dynamic-routing" to Logical Routers. If set to true
>>> all
>>> + static and connected routes attached to the router are shared to the
>>> + southbound "Route" table for sharing outside of OVN.
>>> + The routes can furthe be filtered by setting
>>> `dynamic-routing-connected`
>>> + and `dynamic-routing-static` on the LR or LRP. The LRP settings
>>> overwrite
>>> + the LR settings for all routes using this interface as an exit.
>>> + - Allow Logical Routers to dynamically learn routes from outside the
>>> fabric.
>>> + Routes entered into the "Route" table in the southbound database will
>>> be
>>> + learned by the respective LR. They are included in the route table
>>> with
>>> + a lower priority than static routes.
>>> + - Add the option "dynamic-routing-connected-as-host-routes" to LRPs. If
>>> set
>>> + to true then connected routes are announced as individual host routes.
>>> + - Add 'dynamic-routing-lb-vips' LRP option. If set to true, the LRP can
>>> be
>>> + used to advertise host paths to the Load Balancer VIPs associated
>>> with the
>>> + LR.
>>> + - Add 'dynamic-routing-nat' LRP option. If set to true, the LRP can be
>>> used
>>> + to advertise external NAT IPs associated with it.
>>>
>>> OVN v24.09.0 - 13 Sep 2024
>>> --------------------------
>>> diff --git a/northd/en-advertised-route-sync.c
>>> b/northd/en-advertised-route-sync.c
>>> index 02b3aba7a..b6b759201 100644
>>> --- a/northd/en-advertised-route-sync.c
>>> +++ b/northd/en-advertised-route-sync.c
>>> @@ -16,6 +16,7 @@
>>>
>>> #include <config.h>
>>>
>>> +#include "openvswitch/vlog.h"
>>> #include "smap.h"
>>> #include "stopwatch.h"
>>> #include "northd.h"
>>> @@ -26,6 +27,8 @@
>>> #include "openvswitch/hmap.h"
>>> #include "ovn-util.h"
>>>
>>> +VLOG_DEFINE_THIS_MODULE(en_advertised_route_sync);
>>> +
>>> static void
>>> advertised_route_table_sync(
>>> struct ovsdb_idl_txn *ovnsb_txn,
>>> @@ -312,6 +315,185 @@ publish_host_routes(struct hmap *sync_routes,
>>> }
>>> }
>>>
>>> +/* This function searches for an ovn_port with specific "op_ip"
>>> + * IP address in all LS datapaths directly connected to the
>>> + * LR datapath "od".
>>> + * If no match is found, this function returns NULL.*/
>>> +static struct ovn_port *
>>> +find_port_in_connected_ls(struct ovn_datapath *od, char *op_ip)
>>> +{
>>> + struct in6_addr *ip_addr = xmalloc(sizeof *ip_addr);
>>> + unsigned int plen;
>>> + bool is_ipv4 = true;
>>> +
>>> + /* Return immediately if op_ip isn't a host route.*/
>>
>> We shouldn't have to check this, it should be the caller doing it. We
>> shouldn't parse values coming directly from the database here. We
>> should receive an already correct struct in6_addr * instead as argument.
>>
>>> + if (!ip46_parse_cidr(op_ip, ip_addr, &plen)) {
>>> + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
>>> + VLOG_WARN_RL(&rl, "bad IP for OVN port: %s", op_ip);
>>> + free(ip_addr);
>>> + return NULL;
>>> + }
>>> + if (IN6_IS_ADDR_V4MAPPED(ip_addr) && plen != 32) {
>>> + if (plen != 32) {
>>> + free(ip_addr);
>>> + return NULL;
>>> + }
>>> + } else {
>>> + is_ipv4 = false;
>>> + if (plen != 128) {
>>> + free(ip_addr);
>>> + return NULL;
>>> + }
>>> + }
>>> + free(ip_addr);
>>> +
>>> + for (int i = 0; i < od->n_ls_peers; i++) {
>>
>> Nit: size_t (in a few places)
>>
>>> + struct ovn_datapath *peer_od = od->ls_peers[i];
>>> + struct ovn_port *op;
>>> + HMAP_FOR_EACH (op, dp_node, &peer_od->ports) {
>>> + for (int j = 0; j < op->n_lsp_addrs; j++) {
>>> + struct lport_addresses *addrs = &op->lsp_addrs[j];
>>> + if (is_ipv4) {
>>> + for (int k = 0; k < addrs->n_ipv4_addrs; k++) {
>>> + if (!strcmp(op_ip, addrs->ipv4_addrs[k].addr_s)) {
>>> + return op;
>>> + }
>>> + }
>>> + } else {
>>> + for (int k = 0; k < addrs->n_ipv6_addrs; k++) {
>>> + if (!strcmp(op_ip, addrs->ipv6_addrs[k].addr_s)) {
>>> + return op;
>>> + }
>>> + }
>>> + }
>>> + }
>>> + }
>>> + }
>>
>> 'struct ovn_datapath' has the following field:
>>
>> struct sset router_ips; /* Router port IPs except the IPv6 LLAs. */
>>
>> Instead of walking all ports of all connected switches of 'od' maybe we
>> should also store all switch port IPs inside 'struct ovn_datapath'.
>> Then we'd just have to do a sset_find() for each connected switch.
>>
>> We're calling this function for each IP of a NAT/LB we're advertising (i
>> expect quite a few of those) so all these linear lookups quickly become
>> very costly.
>>
>>> + return NULL;
>>> +}
>>> +
>>> +/* Publish parsed_route as a Advertised_Route record. The main purpose
>>> + * of this function is to find LSP with IP address that matches the
>>> + * logical_ip of a NAT rule on the route's out_port datapath. This port is
>>> + * then set as a tracked_port for the route.
>>> + * Search is performed in all LS datapaths directly connected to the
>>> + * route's out_port datapath.
>>> + * If no suitable tracked_port is found, route is still published, but
>>> + * without the tracked_port set.*/
>>> +static void
>>> +publish_nat_route(struct hmap *route_map,
>>> + const struct parsed_route *route)
>>> +{
>>> + const struct ovn_datapath *advertised_dp = route->od;
>>> + const struct ovn_port *ext_nat_port = route->out_port;
>>> +
>>> + if (!advertised_dp->nbr || !ext_nat_port->od) {
>>> + return;
>>> + }
>>> +
>>> + char *ip_prefix = normalize_v46_prefix(&route->prefix,
>>> + route->plen);
>>> + char *logical_ip = NULL;
>>> +
>>> + /* Find NAT rule with external IP that matches the route's
>>> + * ip_prefix.*/
>>> + for (int i = 0; i < ext_nat_port->od->nbr->n_nat; i++) {
>>
>> Nit: size_t
>>
>>> + struct nbrec_nat *nat = ext_nat_port->od->nbr->nat[i];
>>> + if (!strcmp(nat->external_ip, ip_prefix)) {
>>> + logical_ip = nat->logical_ip;
>>> + break;
>>> + }
>>> + }
>>> +
>>> + if (!logical_ip) {
>>> + return;
>>> + }
>>> +
>>> + const struct sbrec_port_binding *tracked_pb = NULL;
>>> + struct ovn_port *tracked_port =
>>> find_port_in_connected_ls(ext_nat_port->od,
>>> + logical_ip);
>>> + if (tracked_port) {
>>> + tracked_pb = tracked_port->sb;
>>> + }
If I comment out the part that tries to find the tracked_port all the
new system and unit tests still pass.
>>> +
>>> + char *ip_with_mask = xasprintf("%s/%d", ip_prefix, route->plen);
>>> + ar_alloc_entry(route_map,
>>> + route->od->sb,
>>> + route->out_port->sb,
>>> + ip_with_mask,
>>> + tracked_pb);
>>> + free(ip_prefix);
>>> +}
>>> +
>>> +/* Publish parsed_route as a Advertised_Route record. The main purpose
>>> + * of this function is to find LSP with IP address that matches the
>>> + * endpoint IP of a LB on the route's out_port datapath. This port is then
>>> + * set as a tracked_port for the route.
>>> + * Search is performed in all LS datapaths directly connected to the
>>> + * route's out_port datapath.
>>> + * If no suitable tracked_port is found, route is still published, but
>>> + * without the tracked_port set.*/
>>> +static void
>>> +publish_lb_route(struct hmap *route_map,
>>> + const struct parsed_route *route)
>>> +{
>>> + const struct ovn_datapath *advertised_dp = route->od;
>>> + const struct ovn_port *ext_lb_port = route->out_port;
>>> +
>>> + if (!advertised_dp->nbr || !ext_lb_port->od) {
>>> + return;
>>> + }
>>> +
>>> + char *ip_prefix = normalize_v46_prefix(&route->prefix,
>>> + route->plen);
>>> +
>>> + for (int i = 0; i < ext_lb_port->od->nbr->n_load_balancer; i++) {
>>
>> Nit: size_t
>>
>> Also, this is not correct/complete. Load balancers can also be applied
>> through load balancer groups. So you'd have to also iterate on
>> od->nbr->load_balancer_groups.
>>
>> However, the publish_lb_route() receives an argument a parsed_route that
>> has been built based on a load balancer VIP address.
>>
>> Now we're iterating through all load balancers configured on the route's
>> datapath to try to find one that has the same VIP as what was used when
>> the route was built. We actually do more than that, we actually reparse
>> each and every load balancer attached to that router.
>>
>> That seems very inefficient. We should find a way to store this
>> information in the route structure itself at creation time because it
>> gets built from a load balancer.
>>
>
> Re-reading this sentence I realized it might not be that clear. I meant:
>
> 1. based on information computed when parsing load balancer
> configuration (lb_data -> lr_stateful_rec)
> 2. we create 'struct parsed_route' records - the calls to
> parsed_routes_add_lb()
> 3. then later, if the route was created from a load balancer (type ==
> ROUTE_SOURCE_LB) we try to go back and find the load balancer
> information that triggered the route creation (at step "2").
>
> We should, when doing step "2", store a pointer to the parsed LB
> information we'll be needing at step "3".
>
>> This comment also applies to the publish_nat_route() function.
>>
>>> + struct nbrec_load_balancer *lb =
>>> + ext_lb_port->od->nbr->load_balancer[i];
>>> + struct smap lb_vips = SMAP_INITIALIZER(&lb_vips);
>>> + struct smap_node *node;
>>> + SMAP_FOR_EACH (node, &lb->vips) {
>>> + struct ovn_lb_vip *lb_vip = xmalloc(sizeof(*lb_vip));
>>> + char *error = ovn_lb_vip_init(lb_vip, node->key,
>>> + node->value, false, AF_INET6);
>>> + if (error) {
>>> + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5,
>>> 1);
>>> + VLOG_WARN_RL(&rl, "Failed to parse LB VIP: %s", error);
>>> + ovn_lb_vip_destroy(lb_vip);
>>> + free(error);
>>> + continue;
>>> + }
>>> +
>>> + /* Continue only for LBs whose VIP matches route's ip_prefix.*/
>>> + if (strcmp(lb_vip->vip_str, ip_prefix)) {
>>> + ovn_lb_vip_destroy(lb_vip);
>>> + continue;
>>> + }
>>> +
>>> + /* Find LSP that matches at least one endpoint IP of the LB.*/
>>> + const struct sbrec_port_binding *tracked_pb = NULL;
>>> + for (int j = 0; j < lb_vip->n_backends; j++) {
>>
>> Nit: size_t
>>
>>> + char *backend = lb_vip->backends[j].ip_str;
>>> + struct ovn_port *tracked_port =
>>> + find_port_in_connected_ls(ext_lb_port->od, backend);
>>
>> This is a function call but it hides a LOT of complexity (see the
>> comment above on the function definition).
>>
>>> + if (tracked_port) {
>>> + tracked_pb = tracked_port->sb;
>>> + break;
>>
>> This makes me wonder why we even use tracked_port for load balancers.
>> Load balancer backends could be distributed across different physical nodes.
>>
>> Why advertise the route only with one port?
>>
>> OR, why not advertise the route with all ports who own backend IPs?
>>
>>> + }
>>> + }
Here too, if I comment out the part that tries to find the tracked_port,
all new tests still pass.
Finding the tracked_port (especially for load balancers but for NATs
too) seems to be the very costly operation here. However we don't seem
to have any tests that cover that.
This makes me wonder, is it really a requirement for this feature that
the tracked port is set for these routes? Not having to implement it
now seems to simplify the logic and would remove the need to trace back
from lb/nat parsed_routes to their NB configurations.
Would that be an acceptable compromise?
>>> +
>>> + char *ip_with_mask = xasprintf("%s/%d", ip_prefix,
>>> route->plen);
>>> + ar_alloc_entry(route_map,
>>> + route->od->sb,
>>> + route->out_port->sb,
>>> + ip_with_mask,
>>> + tracked_pb);
>>> + ovn_lb_vip_destroy(lb_vip);
>>> + free(ip_prefix);
Don't we need to break here in any case?
>>> + }
>>> + }
>>> +}
>>> +
>>> static void
>>> advertised_route_table_sync(
>>> struct ovsdb_idl_txn *ovnsb_txn,
>>> @@ -359,6 +541,22 @@ advertised_route_table_sync(
>>> !route->out_port->dynamic_routing_static) {
>>> continue;
>>> }
>>> + if (route->source == ROUTE_SOURCE_NAT) {
>>> + if (!smap_get_bool(&route->out_port->nbrp->options,
>>> + "dynamic-routing-nat", false)) {
>>> + continue;
>>> + }
>>> + publish_nat_route(&sync_routes, route);
>>> + continue;
>>> + }
>>> + if (route->source == ROUTE_SOURCE_LB) {
>>> + if (!smap_get_bool(&route->out_port->nbrp->options,
>>> + "dynamic-routing-lb-vips", false)) {
>>> + continue;
>>> + }
>>> + publish_lb_route(&sync_routes, route);
>>> + continue;
>>> + }
>>>
>>> char *ip_prefix = normalize_v46_prefix(&route->prefix,
>>> route->plen);
>>> diff --git a/northd/en-northd.c b/northd/en-northd.c
>>> index c7d1ebcb3..60e295628 100644
>>> --- a/northd/en-northd.c
>>> +++ b/northd/en-northd.c
>>> @@ -316,6 +316,8 @@ en_routes_run(struct engine_node *node, void *data)
>>> {
>>> struct northd_data *northd_data = engine_get_input_data("northd",
>>> node);
>>> struct bfd_data *bfd_data = engine_get_input_data("bfd", node);
>>> + struct ed_type_lr_stateful *lr_stateful_data =
>>> + engine_get_input_data("lr_stateful", node);
>>> struct routes_data *routes_data = data;
>>>
>>> routes_destroy(data);
>>> @@ -330,7 +332,8 @@ en_routes_run(struct engine_node *node, void *data)
>>> route_table_name);
>>> }
>>>
>>> - build_parsed_routes(od, &northd_data->lr_ports,
>>> + build_parsed_routes(od, &lr_stateful_data->table,
>>> + &northd_data->lr_ports,
>>> &bfd_data->bfd_connections,
>>> &routes_data->parsed_routes,
>>> &routes_data->route_tables,
>>> diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c
>>> index 86b7b999e..d74c2e3dc 100644
>>> --- a/northd/inc-proc-northd.c
>>> +++ b/northd/inc-proc-northd.c
>>> @@ -267,6 +267,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb,
>>> engine_add_input(&en_routes, &en_bfd, NULL);
>>> engine_add_input(&en_routes, &en_northd,
>>> routes_northd_change_handler);
>>> + engine_add_input(&en_routes, &en_lr_nat,
>>> + NULL);
>>> + engine_add_input(&en_routes, &en_lb_data,
>>> + NULL);
>>> + engine_add_input(&en_routes, &en_lr_stateful, engine_noop_handler);
>>
>> As mentioned in my other reply, this is a performance regression and I
>> don't really think we need the en_lr_nat and en_lb_data nodes as direct
>> inputs for en_routes:
>>
>> https://mail.openvswitch.org/pipermail/ovs-dev/2025-February/420765.html
>>
>>>
>>> engine_add_input(&en_bfd_sync, &en_bfd, NULL);
>>> engine_add_input(&en_bfd_sync, &en_nb_bfd, NULL);
>>> diff --git a/northd/northd.c b/northd/northd.c
>>> index 9f8d3118c..bff69e545 100644
>>> --- a/northd/northd.c
>>> +++ b/northd/northd.c
>>> @@ -11076,8 +11076,111 @@ parsed_routes_add_connected(const struct
>>> ovn_datapath *od,
>>> }
>>> }
>>>
>>> +/* Add "parsed_route"s to the "routes" for each IP address in laddrs.*/
>>> +static void
>>> +lport_addrs_to_parsed_routes(const struct ovn_datapath *od,
>>> + const struct ovn_port *op,
>>> + const struct lport_addresses *laddrs,
>>> + enum route_source route_type,
>>> + struct hmap *routes,
>>> + bool install_lflow)
>>> +{
>>> + for (int i = 0; i < laddrs->n_ipv4_addrs; i++) {
>>
>> Nit: size_t
>>
>>> + struct ipv4_netaddr *addr = &laddrs->ipv4_addrs[i];
>>> + struct in6_addr prefix;
>>> + ip46_parse(addr->network_s, &prefix);
>>> + parsed_route_add(od, NULL, &prefix, addr->plen,
>>> + false, addr->addr_s, op,
>>> + 0, false,
>>> + false, NULL, route_type,
>>> + &op->nbrp->header_, routes, install_lflow);
>>> + }
>>> + for (int i = 0; i < laddrs->n_ipv6_addrs; i++) {
>>
>> Nit: size_t
>>
>>> + struct ipv6_netaddr *addr = &laddrs->ipv6_addrs[i];
>>> + parsed_route_add(od, NULL, &addr->addr, addr->plen,
>>> + false, addr->addr_s, op,
>>> + 0, false,
>>> + false, NULL, route_type,
>>> + &op->nbrp->header_, routes, install_lflow);
>>> + }
>>> +}
>>> +
>>> +/* Add routes of all NAT external_addresses on the ovn_port "op", to be
>>> + * advertised by the datapath "od".
>>> + * If "routeable_only" is true, only NATs with option "add_route=true" will
>>> + * be considered.
>>> + * Note that the "op" does not necessarily need to be in the "od" datapath.
>>> + * Such use-case is for advertising external NAT IPs of LRPs in neighboring
>>> + * routers and should be paired with routable_only=true.
>>> + */
>>> +static void
>>> +parsed_routes_add_nat(const struct ovn_datapath *od,
>>> + const struct ovn_port *op,
>>> + struct hmap *routes,
>>> + bool routable_only)
>>> +{
>>> + if (!op->nbrp || !smap_get_bool(&op->nbrp->options,
>>> + "dynamic-routing-nat", false)) {
>>> + return;
>>> + }
>>> + size_t n_nats = 0;
>>> + char **nats = NULL;
>>> + nats = get_nat_addresses(op, &n_nats, routable_only, false, NULL,
>>> true);
>>> +
>>> + for (size_t i = 0; i < n_nats; i++) {
>>> + struct lport_addresses *laddrs = xzalloc(sizeof *laddrs);
>>
>> We leak this struct. Actually, we don't need to xalloc() this, it could
>> just be a variable on the stack AFAICT.
>>
>>> + int ofs = 0;
>>> + extract_addresses(nats[i], laddrs, &ofs);
>>> + lport_addrs_to_parsed_routes(od, op, laddrs, ROUTE_SOURCE_NAT,
>>> + routes, false);
>>> + destroy_lport_addresses(laddrs);
>>> + free(nats[i]);
>>> + }
>>> + free(nats);
>>> +}
>>> +
>>> +/* Add routes of all LB VIPs on the ovn_port "op", to be advertised by the
>>> + * datapath "od".
>>> + * If "routeable_only" is true, only LBs with option "add_route=true" will
>>> + * be considered.
>>> + * Note that the "op" does not necessarily need to be in the "od" datapath.
>>> + * Such use-case is for advertising LB VIPs of LRPs in neighboring routers
>>> + * and should be paired with routable_only=true.
>>> + */
>>> +static void
>>> +parsed_routes_add_lb(const struct ovn_datapath *od,
>>> + const struct ovn_port *op,
>>> + struct hmap *routes,
>>> + const struct lr_stateful_record *lr_stateful_rec,
>>> + bool routable_only)
>>> +{
>>> + if (!op->nbrp || !smap_get_bool(&op->nbrp->options,
>>> + "dynamic-routing-lb-vips", false)) {
>>> + return;
>>> + }
>>> +
>>> + if (!lr_stateful_rec) {
>>> + return;
>>> + }
>>> +
>>> + struct ds addresses = DS_EMPTY_INITIALIZER;
>>> + get_lb_addresses(lr_stateful_rec, &addresses, routable_only);
>>> +
>>> + struct lport_addresses *laddrs = xzalloc(sizeof *laddrs);
>>
>> Same here, we leak laddrs.
>>
>>> + extract_ip_addresses(ds_cstr(&addresses), laddrs);
>>> +
>>> + lport_addrs_to_parsed_routes(od, op, laddrs, ROUTE_SOURCE_LB,
>>> + routes, false);
>>> +
>>> + destroy_lport_addresses(laddrs);
>>> + ds_destroy(&addresses);
>>> +
>>> +}
>>> +
>>> void
>>> -build_parsed_routes(const struct ovn_datapath *od, const struct hmap
>>> *lr_ports,
>>> +build_parsed_routes(const struct ovn_datapath *od,
>>> + const struct lr_stateful_table *lr_stateful_table,
>>> + const struct hmap *lr_ports,
>>> const struct hmap *bfd_connections, struct hmap
>>> *routes,
>>> struct simap *route_tables,
>>> struct hmap *bfd_active_connections)
>>> @@ -11097,7 +11200,34 @@ build_parsed_routes(const struct ovn_datapath *od,
>>> const struct hmap *lr_ports,
>>>
>>> const struct ovn_port *op;
>>> HMAP_FOR_EACH (op, dp_node, &od->ports) {
>>> + const struct lr_stateful_record *lr_stateful_rec = NULL;
>>> + lr_stateful_rec = lr_stateful_table_find_by_index(
>>> + lr_stateful_table, op->od->index);
>>> +
>>> parsed_routes_add_connected(od, op, routes);
>>> + parsed_routes_add_nat(od, op, routes, false);
>>> + parsed_routes_add_lb(od, op, routes, lr_stateful_rec, false);
>>> + }
>>> +
>>> + /* For GW routers we traverse also neighboring LR in search for
>>> + * LBs and NATs with "add_route" option set to "true". Such NATs/LBs
>>> + * install logical flows for their external IP addresses to their
>>> + * neighboring routers, and as such, we can advertise them as routes.
>>> + * The LRP on the neighboring router needs to have appropriate
>>> + * "dynamic-routing-{nat,lb-vips}" option set for this to take
>>> effect.*/
I guess we don't have any system tests covering this? As far as I can
tell the tests this patch adds only use a single GW router and no other
neighbor LRs.
If I comment this part out all the new system tests still pass.
>>> + for (size_t i = 0; od->is_gw_router && i < od->n_ls_peers; i++) {
>>
>> What about direct router peers? Two logical routers can be directly
>> attached.
>>
>>> + for (size_t j = 0; j < od->ls_peers[i]->n_router_ports; j++) {
>>> + struct ovn_port *router_port;
>>> + router_port = od->ls_peers[i]->router_ports[j]->peer;
>>> +
>>> + const struct lr_stateful_record *lr_stateful_rec = NULL;
>>> + lr_stateful_rec = lr_stateful_table_find_by_index(
>>> + lr_stateful_table, router_port->od->index);
>>> +
>>> + parsed_routes_add_nat(od, router_port, routes, true);
>>> + parsed_routes_add_lb(od, router_port, routes,
>>> + lr_stateful_rec, true);
>>> + }
>>> }
>>>
>>> HMAP_FOR_EACH_SAFE (pr, key_node, routes) {
>>> @@ -11279,6 +11409,8 @@ route_source_to_offset(enum route_source source)
>>> {
>>> switch (source) {
>>> case ROUTE_SOURCE_CONNECTED:
>>> + case ROUTE_SOURCE_NAT:
>>> + case ROUTE_SOURCE_LB:
>>> return ROUTE_PRIO_OFFSET_CONNECTED;
>>> case ROUTE_SOURCE_STATIC:
>>> return ROUTE_PRIO_OFFSET_STATIC;
>>> diff --git a/northd/northd.h b/northd/northd.h
>>> index fffaab3a0..d55909f76 100644
>>> --- a/northd/northd.h
>>> +++ b/northd/northd.h
>>> @@ -708,6 +708,10 @@ enum route_source {
>>> ROUTE_SOURCE_CONNECTED,
>>> /* The route is derived from a northbound static route entry. */
>>> ROUTE_SOURCE_STATIC,
>>> + /* Host route generated from NAT's external IP. */
>>> + ROUTE_SOURCE_NAT,
>>> + /* Host route generated from LB's external IP. */
>>> + ROUTE_SOURCE_LB,
>>> /* The route is dynamically learned by an ovn-controller. */
>>> ROUTE_SOURCE_LEARNED,
>>> };
>>> @@ -780,7 +784,9 @@ void northd_indices_create(struct northd_data *data,
>>>
>>> void route_policies_init(struct route_policies_data *);
>>> void route_policies_destroy(struct route_policies_data *);
>>> -void build_parsed_routes(const struct ovn_datapath *, const struct hmap *,
>>> +void build_parsed_routes(const struct ovn_datapath *,
>>> + const struct lr_stateful_table *lr_stateful_table,
>>> + const struct hmap *,
>>> const struct hmap *, struct hmap *, struct simap
>>> *,
>>> struct hmap *);
>>> uint32_t get_route_table_id(struct simap *, const char *);
>>> diff --git a/ovn-nb.xml b/ovn-nb.xml
>>> index 734fcf2bd..e8d9d5491 100644
>>> --- a/ovn-nb.xml
>>> +++ b/ovn-nb.xml
>>> @@ -2971,6 +2971,10 @@ or
>>> table="Logical_Router_Port"/></li>
>>> <li><ref column="options" key="dynamic-routing-static"
>>> table="Logical_Router_Port"/></li>
>>> + <li><ref column="options" key="dynamic-routing-lb-vips"
>>> + table="Logical_Router_Port"/></li>
>>> + <li><ref column="options" key="dynamic-routing-nat"
>>> + table="Logical_Router_Port"/></li>
>>> </ul>
>>> </column>
>>>
>>> @@ -3828,6 +3832,34 @@ or
>>> This option allows to have a 1:1 mapping between a single LRP and
>>> an
>>> individual link.
>>> </column>
>>> +
>>> + <column name="options" key="dynamic-routing-lb-vips"
>>> + type='{"type": "boolean"}'>
>>> + <p>
>>> + Only relevant if <ref column="options" key="dynamic-routing"
>>> + table="Logical_Router"/> on the respective Logical_Router is set
>>> + to <code>true</code>.
>>> +
>>> + If this option is <code>true</code>, northd will create host
>>> route
>>> + entries in the southbound <ref table="Advertised_Route"
>>> + db="OVN_Southbound"/> table, associated with this LRP, for each
>>> LB
>>> + VIP.
>>> + </p>
>>> + </column>
>>> +
>>> + <column name="options" key="dynamic-routing-nat"
>>> + type='{"type": "boolean"}'>
>>> + <p>
>>> + Only relevant if <ref column="options" key="dynamic-routing"
>>> + table="Logical_Router"/> on the respective Logical_Router is set
>>> + to <code>true</code>.
>>> +
>>> + If this option is <code>true</code>, northd will create host
>>> route
>>> + entries in the southbound <ref table="Advertised_Route"
>>> + db="OVN_Southbound"/> table, for external IP addresses of NAT
>>> rules
>>> + associated with this LRP.
>>> + </p>
>>> + </column>
>>> </group>
>>>
>>> <group title="Attachment">
>>> diff --git a/ovs b/ovs
>>> index a3c06c309..c648ee626 160000
>>> --- a/ovs
>>> +++ b/ovs
>>> @@ -1 +1 @@
>>> -Subproject commit a3c06c309ab9f21fd9ce4e8dca542119d46be9c5
>>> +Subproject commit c648ee626efdc959ed0cff37c2b9128816450f15
>>
>> This ovs submodule change is accidental I guess.
>>
>>> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
>>> index 44dc92869..bad9ca0b5 100644
>>> --- a/tests/ovn-northd.at
>>> +++ b/tests/ovn-northd.at
>>> @@ -15134,3 +15134,559 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE
>>> AT_CLEANUP
>>> ])
>>>
>>> +OVN_FOR_EACH_NORTHD_NO_HV([
>>> +AT_SETUP([dynamic-routing - nat sync to sb IPv4])
>>> +AT_KEYWORDS([dynamic-routing])
>>> +ovn_start
>>> +
>>> +# Start with GW router and a single LRP
>>> +check ovn-nbctl lr-add lr0
>>> +check ovn-nbctl \
>>> + -- \
>>> + set Logical_Router lr0 options:dynamic-routing=true \
>>> + options:chassis=hv1
>>> +check ovn-nbctl --wait=sb \
>>> + -- \
>>> + lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
>>> +
>>> +check_row_count Advertised_Route 0
>>> +
>>> +datapath=$(ovn-sbctl --bare --columns _uuid list datapath_binding lr0)
>>> +pb=$(ovn-sbctl --bare --columns _uuid list port_binding lr0-sw0)
>>> +
>>> +# Adding LRP dynamic-routing-nat option and NAT rule advertises a route
>>> entry
>>> +check ovn-nbctl --wait=sb \
>>> + -- \
>>> + lrp-set-options lr0-sw0 dynamic-routing-nat=true \
>>> + -- \
>>> + lr-nat-add lr0 dnat_and_snat 172.16.1.10 192.168.1.10
>>> +
>>> +ovn-nbctl list NAT
>>> +ovn-sbctl list Advertised_Route
>>> +ovn-sbctl lflow-list
>>> +
>>> +check_row_count Advertised_Route 1
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb], [0], [dnl
>>> +172.16.1.10/32
>>> +])
>>> +
>>> +# Add LR with distributed LRP connected to GW router through join LS
>>> +check ovn-nbctl --wait=sb \
>>> + -- \
>>> + lrp-add lr0 lr0-join 00:00:00:00:ff:02 10.42.0.1/24 \
>>> + -- \
>>> + ls-add ls-join \
>>> + -- \
>>> + lsp-add ls-join lsp-join-to-lr0 \
>>> + -- \
>>> + lsp-set-type lsp-join-to-lr0 router \
>>> + -- \
>>> + lsp-set-options lsp-join-to-lr0 router-port=lr0-join \
>>> + -- \
>>> + lsp-set-addresses lsp-join-to-lr0 router \
>>> + -- \
>>> + lr-add lr-guest0 \
>>> + -- \
>>> + lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 10.51.0.1/24 \
>>> + -- \
>>> + lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 10.42.0.2/24 \
>>> + -- \
>>> + lrp-set-options lrp-guest0-join dynamic-routing-nat=true \
>>> + -- \
>>> + lsp-add ls-join lsp-join-to-guest0 \
>>> + -- \
>>> + lsp-set-type lsp-join-to-guest0 router \
>>> + -- \
>>> + lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \
>>> + -- \
>>> + lrp-set-gateway-chassis lrp-guest0-join hv1
>>> +
>>> +pb2=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest0-join)
>>> +
>>> +check ovn-nbctl --wait=sb \
>>> + --add-route lr-nat-add lr-guest0 dnat_and_snat 172.16.2.10 192.168.2.10
>>> +
>>> +check_row_count Advertised_Route 2
>>> +ovn-sbctl list advertised_route
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port="$pb"], [0], [dnl
>>> +172.16.1.10/32
>>> +])
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb2], [0], [dnl
>>> +172.16.2.10/32
>>> +])
>>> +
>>> +# Add nonlocal LR with distributed LRP connected to GW router through join
>>> LS
>>> +check ovn-nbctl --wait=sb \
>>> + -- \
>>> + lr-add lr-guest1 \
>>> + -- \
>>> + lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 10.51.1.1/24 \
>>> + -- \
>>> + lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 10.42.0.3/24 \
>>> + -- \
>>> + lrp-set-options lrp-guest1-join dynamic-routing-nat=true \
>>> + -- \
>>> + lsp-add ls-join lsp-join-to-guest1 \
>>> + -- \
>>> + lsp-set-type lsp-join-to-guest1 router \
>>> + -- \
>>> + lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \
>>> + -- \
>>> + lrp-set-gateway-chassis lrp-guest1-join nonlocalhv
>>> +
>>> +pb3=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest1-join)
>>> +
>>> +check ovn-nbctl --wait=sb \
>>> + --add-route lr-nat-add lr-guest1 dnat_and_snat 172.16.3.10 192.168.3.10
>>> +check_row_count Advertised_Route 3
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb], [0], [dnl
>>> +172.16.1.10/32
>>> +])
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb2], [0], [dnl
>>> +172.16.2.10/32
>>> +])
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb3], [0], [dnl
>>> +172.16.3.10/32
>>> +])
>>> +
>>> +# removing the option:dynamic-routing removes all routes
>>> +check ovn-nbctl --wait=sb remove Logical_Router lr0 option dynamic-routing
>>> +check_row_count Advertised_Route 0
>>> +
>>> +# and setting it again adds them again
>>> +check ovn-nbctl --wait=sb set Logical_Router lr0
>>> option:dynamic-routing=true
>>> +check_row_count Advertised_Route 3
>>> +
>>> +# removing the lr will remove all routes
>>> +check ovn-nbctl --wait=sb lr-del lr0
>>> +check_row_count Advertised_Route 0
>>> +
>>> +AT_CLEANUP
>>> +])
>>> +
>>> +OVN_FOR_EACH_NORTHD_NO_HV([
>>> +AT_SETUP([dynamic-routing - nat sync to sb IPv6])
>>> +AT_KEYWORDS([dynamic-routing])
>>> +ovn_start
>>> +
>>> +# Start with GW router and a single LRP
>>> +check ovn-nbctl lr-add lr0
>>> +check ovn-nbctl \
>>> + -- \
>>> + set Logical_Router lr0 options:dynamic-routing=true \
>>> + options:chassis=hv1
>>> +check ovn-nbctl --wait=sb \
>>> + -- \
>>> + lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 2001:db0::1/64
>>> +
>>> +check_row_count Advertised_Route 0
>>> +
>>> +datapath=$(ovn-sbctl --bare --columns _uuid list datapath_binding lr0)
>>> +pb=$(ovn-sbctl --bare --columns _uuid list port_binding lr0-sw0)
>>> +
>>> +# Adding LRP dynamic-routing-nat option and NAT rule advertises a route
>>> entry
>>> +check ovn-nbctl --wait=sb \
>>> + -- \
>>> + lrp-set-options lr0-sw0 dynamic-routing-nat=true \
>>> + -- \
>>> + lr-nat-add lr0 dnat_and_snat 2001:db1::10 2001:db2::10
>>> +
>>> +ovn-nbctl list NAT
>>> +ovn-sbctl list Advertised_Route
>>> +ovn-sbctl lflow-list
>>> +
>>> +check_row_count Advertised_Route 1
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb], [0], [dnl
>>> +2001:db1::10/128
>>> +])
>>> +
>>> +# Add LR with distributed LRP connected to GW router through join LS
>>> +check ovn-nbctl --wait=sb \
>>> + -- \
>>> + lrp-add lr0 lr0-join 00:00:00:00:ff:02 2001:db42::1/64 \
>>> + -- \
>>> + ls-add ls-join \
>>> + -- \
>>> + lsp-add ls-join lsp-join-to-lr0 \
>>> + -- \
>>> + lsp-set-type lsp-join-to-lr0 router \
>>> + -- \
>>> + lsp-set-options lsp-join-to-lr0 router-port=lr0-join \
>>> + -- \
>>> + lsp-set-addresses lsp-join-to-lr0 router \
>>> + -- \
>>> + lr-add lr-guest0 \
>>> + -- \
>>> + lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 2001:db51::1/64 \
>>> + -- \
>>> + lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 2001:db42::2/64 \
>>> + -- \
>>> + lrp-set-options lrp-guest0-join dynamic-routing-nat=true \
>>> + -- \
>>> + lsp-add ls-join lsp-join-to-guest0 \
>>> + -- \
>>> + lsp-set-type lsp-join-to-guest0 router \
>>> + -- \
>>> + lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \
>>> + -- \
>>> + lrp-set-gateway-chassis lrp-guest0-join hv1
>>> +
>>> +pb2=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest0-join)
>>> +
>>> +check ovn-nbctl --wait=sb \
>>> + --add-route lr-nat-add lr-guest0 dnat_and_snat 2001:db3::10
>>> 2001:db4::10
>>> +
>>> +check_row_count Advertised_Route 2
>>> +ovn-sbctl list advertised_route
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port="$pb"], [0], [dnl
>>> +2001:db1::10/128
>>> +])
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb2], [0], [dnl
>>> +2001:db3::10/128
>>> +])
>>> +
>>> +# Add nonlocal LR with distributed LRP connected to GW router through join
>>> LS
>>> +check ovn-nbctl --wait=sb \
>>> + -- \
>>> + lr-add lr-guest1 \
>>> + -- \
>>> + lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 2001:db52::1/64 \
>>> + -- \
>>> + lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 2001:db42::3/64 \
>>> + -- \
>>> + lrp-set-options lrp-guest1-join dynamic-routing-nat=true \
>>> + -- \
>>> + lsp-add ls-join lsp-join-to-guest1 \
>>> + -- \
>>> + lsp-set-type lsp-join-to-guest1 router \
>>> + -- \
>>> + lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \
>>> + -- \
>>> + lrp-set-gateway-chassis lrp-guest1-join nonlocalhv
>>> +
>>> +pb3=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest1-join)
>>> +
>>> +check ovn-nbctl --wait=sb \
>>> + --add-route lr-nat-add lr-guest1 dnat_and_snat 2001:db5::10
>>> 2001:db6::10
>>> +check_row_count Advertised_Route 3
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb], [0], [dnl
>>> +2001:db1::10/128
>>> +])
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb2], [0], [dnl
>>> +2001:db3::10/128
>>> +])
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb3], [0], [dnl
>>> +2001:db5::10/128
>>> +])
>>> +
>>> +# removing the option:dynamic-routing removes all routes
>>> +check ovn-nbctl --wait=sb remove Logical_Router lr0 option dynamic-routing
>>> +check_row_count Advertised_Route 0
>>> +
>>> +# and setting it again adds them again
>>> +check ovn-nbctl --wait=sb set Logical_Router lr0
>>> option:dynamic-routing=true
>>> +check_row_count Advertised_Route 3
>>> +
>>> +# removing the lr will remove all routes
>>> +check ovn-nbctl --wait=sb lr-del lr0
>>> +check_row_count Advertised_Route 0
>>> +
>>> +AT_CLEANUP
>>> +])
>>> +
>>> +OVN_FOR_EACH_NORTHD_NO_HV([
>>> +AT_SETUP([dynamic-routing - LB sync to sb IPv4])
>>> +AT_KEYWORDS([dynamic-routing])
>>> +ovn_start
>>> +
>>> +# Start with GW router and a single LRP
>>> +check ovn-nbctl lr-add lr0
>>> +check ovn-nbctl \
>>> + -- \
>>> + set Logical_Router lr0 options:dynamic-routing=true \
>>> + options:chassis=hv1
>>> +check ovn-nbctl --wait=sb \
>>> + -- \
>>> + lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
>>> +
>>> +check_row_count Advertised_Route 0
>>> +
>>> +datapath=$(ovn-sbctl --bare --columns _uuid list datapath_binding lr0)
>>> +pb=$(ovn-sbctl --bare --columns _uuid list port_binding lr0-sw0)
>>> +
>>> +# Adding LRP dynamic-routing-lb-vips option and LB VIP rule advertises a
>>> route entry
>>> +check ovn-nbctl --wait=sb \
>>> + -- \
>>> + lrp-set-options lr0-sw0 dynamic-routing-lb-vips=true \
>>> + -- \
>>> + lb-add lb0 172.16.1.10:80 192.168.1.10:80,192.168.1.11:80 \
>>> + -- \
>>> + lr-lb-add lr0 lb0
>>> +
>>> +ovn-nbctl list Load_Balancer
>>> +ovn-sbctl list Advertised_Route
>>> +ovn-sbctl lflow-list
>>> +
>>> +check_row_count Advertised_Route 1
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb], [0], [dnl
>>> +172.16.1.10/32
>>> +])
>>> +
>>> +# Add LR with distributed LRP connected to GW router through join LS
>>> +check ovn-nbctl --wait=sb \
>>> + -- \
>>> + lrp-add lr0 lr0-join 00:00:00:00:ff:02 10.42.0.1/24 \
>>> + -- \
>>> + ls-add ls-join \
>>> + -- \
>>> + lsp-add ls-join lsp-join-to-lr0 \
>>> + -- \
>>> + lsp-set-type lsp-join-to-lr0 router \
>>> + -- \
>>> + lsp-set-options lsp-join-to-lr0 router-port=lr0-join \
>>> + -- \
>>> + lsp-set-addresses lsp-join-to-lr0 router \
>>> + -- \
>>> + lr-add lr-guest0 \
>>> + -- \
>>> + lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 10.51.0.1/24 \
>>> + -- \
>>> + lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 10.42.0.2/24 \
>>> + -- \
>>> + lrp-set-options lrp-guest0-join dynamic-routing-lb-vips=true \
>>> + -- \
>>> + lsp-add ls-join lsp-join-to-guest0 \
>>> + -- \
>>> + lsp-set-type lsp-join-to-guest0 router \
>>> + -- \
>>> + lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \
>>> + -- \
>>> + lrp-set-gateway-chassis lrp-guest0-join hv1
>>> +
>>> +pb2=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest0-join)
>>> +
>>> +check ovn-nbctl --wait=sb \
>>> + --add-route lb-add lb1 172.16.2.10:80 192.168.2.10:80,192.168.2.11:80 \
>>> + -- \
>>> + lr-lb-add lr-guest0 lb1
>>> +
>>> +check_row_count Advertised_Route 2
>>> +ovn-sbctl list advertised_route
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port="$pb"], [0], [dnl
>>> +172.16.1.10/32
>>> +])
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb2], [0], [dnl
>>> +172.16.2.10/32
>>> +])
>>> +
>>> +# Add nonlocal LR with distributed LRP connected to GW router through join
>>> LS
>>> +check ovn-nbctl --wait=sb \
>>> + -- \
>>> + lr-add lr-guest1 \
>>> + -- \
>>> + lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 10.51.1.1/24 \
>>> + -- \
>>> + lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 10.42.0.3/24 \
>>> + -- \
>>> + lrp-set-options lrp-guest1-join dynamic-routing-lb-vips=true \
>>> + -- \
>>> + lsp-add ls-join lsp-join-to-guest1 \
>>> + -- \
>>> + lsp-set-type lsp-join-to-guest1 router \
>>> + -- \
>>> + lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \
>>> + -- \
>>> + lrp-set-gateway-chassis lrp-guest1-join nonlocalhv
>>> +
>>> +pb3=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest1-join)
>>> +
>>> +check ovn-nbctl --wait=sb \
>>> + --add-route lb-add lb2 172.16.3.10:80 192.168.3.10:80,192.168.3.11:80 \
>>> + -- \
>>> + lr-lb-add lr-guest1 lb2
>>> +check_row_count Advertised_Route 3
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb], [0], [dnl
>>> +172.16.1.10/32
>>> +])
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb2], [0], [dnl
>>> +172.16.2.10/32
>>> +])
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb3], [0], [dnl
>>> +172.16.3.10/32
>>> +])
>>> +
>>> +# removing the option:dynamic-routing removes all routes
>>> +check ovn-nbctl --wait=sb remove Logical_Router lr0 option dynamic-routing
>>> +check_row_count Advertised_Route 0
>>> +
>>> +# and setting it again adds them again
>>> +check ovn-nbctl --wait=sb set Logical_Router lr0
>>> option:dynamic-routing=true
>>> +check_row_count Advertised_Route 3
>>> +
>>> +# removing the lr will remove all routes
>>> +check ovn-nbctl --wait=sb lr-del lr0
>>> +check_row_count Advertised_Route 0
>>> +
>>> +AT_CLEANUP
>>> +])
>>> +
>>> +OVN_FOR_EACH_NORTHD_NO_HV([
>>> +AT_SETUP([dynamic-routing - LB sync to sb IPv6])
>>> +AT_KEYWORDS([dynamic-routing])
>>> +ovn_start
>>> +
>>> +# Start with GW router and a single LRP
>>> +check ovn-nbctl lr-add lr0
>>> +check ovn-nbctl \
>>> + -- \
>>> + set Logical_Router lr0 options:dynamic-routing=true \
>>> + options:chassis=hv1
>>> +check ovn-nbctl --wait=sb \
>>> + -- \
>>> + lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 2001:db0::1/64
>>> +
>>> +check_row_count Advertised_Route 0
>>> +
>>> +datapath=$(ovn-sbctl --bare --columns _uuid list datapath_binding lr0)
>>> +pb=$(ovn-sbctl --bare --columns _uuid list port_binding lr0-sw0)
>>> +
>>> +# Adding LRP dynamic-routing-lb-vips option and LB VIP rule advertises a
>>> route entry
>>> +check ovn-nbctl --wait=sb \
>>> + -- \
>>> + lrp-set-options lr0-sw0 dynamic-routing-lb-vips=true \
>>> + -- \
>>> + lb-add lb0 [[2001:db1::10]]:80 [[2001:db2::10]]:80,[[2001:db2::11]]:80
>>> \
>>> + -- \
>>> + lr-lb-add lr0 lb0
>>> +
>>> +ovn-nbctl list Load_Balancer
>>> +ovn-sbctl list Advertised_Route
>>> +ovn-sbctl lflow-list
>>> +
>>> +check_row_count Advertised_Route 1
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb], [0], [dnl
>>> +2001:db1::10/128
>>> +])
>>> +
>>> +# Add LR with distributed LRP connected to GW router through join LS
>>> +check ovn-nbctl --wait=sb \
>>> + -- \
>>> + lrp-add lr0 lr0-join 00:00:00:00:ff:02 2001:db42::1/64 \
>>> + -- \
>>> + ls-add ls-join \
>>> + -- \
>>> + lsp-add ls-join lsp-join-to-lr0 \
>>> + -- \
>>> + lsp-set-type lsp-join-to-lr0 router \
>>> + -- \
>>> + lsp-set-options lsp-join-to-lr0 router-port=lr0-join \
>>> + -- \
>>> + lsp-set-addresses lsp-join-to-lr0 router \
>>> + -- \
>>> + lr-add lr-guest0 \
>>> + -- \
>>> + lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01 2001:db52::1/64 \
>>> + -- \
>>> + lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02 2001:db42::2/64 \
>>> + -- \
>>> + lrp-set-options lrp-guest0-join dynamic-routing-lb-vips=true \
>>> + -- \
>>> + lsp-add ls-join lsp-join-to-guest0 \
>>> + -- \
>>> + lsp-set-type lsp-join-to-guest0 router \
>>> + -- \
>>> + lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join \
>>> + -- \
>>> + lrp-set-gateway-chassis lrp-guest0-join hv1
>>> +
>>> +pb2=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest0-join)
>>> +
>>> +check ovn-nbctl --wait=sb \
>>> + --add-route lb-add lb1 [[2001:db3::10]]:80
>>> [[2001:db4::10]]:80,[[2001:db4::11]]:80 \
>>> + -- \
>>> + lr-lb-add lr-guest0 lb1
>>> +
>>> +check_row_count Advertised_Route 2
>>> +ovn-sbctl list advertised_route
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port="$pb"], [0], [dnl
>>> +2001:db1::10/128
>>> +])
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb2], [0], [dnl
>>> +2001:db3::10/128
>>> +])
>>> +
>>> +# Add nonlocal LR with distributed LRP connected to GW router through join
>>> LS
>>> +check ovn-nbctl --wait=sb \
>>> + -- \
>>> + lr-add lr-guest1 \
>>> + -- \
>>> + lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01 2001:db2::1/64 \
>>> + -- \
>>> + lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02 2001:db42::3/64 \
>>> + -- \
>>> + lrp-set-options lrp-guest1-join dynamic-routing-lb-vips=true \
>>> + -- \
>>> + lsp-add ls-join lsp-join-to-guest1 \
>>> + -- \
>>> + lsp-set-type lsp-join-to-guest1 router \
>>> + -- \
>>> + lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join \
>>> + -- \
>>> + lrp-set-gateway-chassis lrp-guest1-join nonlocalhv
>>> +
>>> +pb3=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-guest1-join)
>>> +
>>> +check ovn-nbctl --wait=sb \
>>> + --add-route lb-add lb2 [[2001:db5::10]]:80
>>> [[2001:db4::10]]:80,[[2001:db4::11]]:80 \
>>> + -- \
>>> + lr-lb-add lr-guest1 lb2
>>> +check_row_count Advertised_Route 3
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb], [0], [dnl
>>> +2001:db1::10/128
>>> +])
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb2], [0], [dnl
>>> +2001:db3::10/128
>>> +])
>>> +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find Advertised_Route \
>>> +datapath=$datapath logical_port=$pb3], [0], [dnl
>>> +2001:db5::10/128
>>> +])
>>> +
>>> +# removing the option:dynamic-routing removes all routes
>>> +check ovn-nbctl --wait=sb remove Logical_Router lr0 option dynamic-routing
>>> +check_row_count Advertised_Route 0
>>> +
>>> +# and setting it again adds them again
>>> +check ovn-nbctl --wait=sb set Logical_Router lr0
>>> option:dynamic-routing=true
>>> +check_row_count Advertised_Route 3
>>> +
>>> +# removing the lr will remove all routes
>>> +check ovn-nbctl --wait=sb lr-del lr0
>>> +check_row_count Advertised_Route 0
>>> +
>>> +AT_CLEANUP
>>> +])
>>> +
>>> diff --git a/tests/system-ovn.at b/tests/system-ovn.at
>>> index 5c4364663..918266ad2 100644
>>> --- a/tests/system-ovn.at
>>> +++ b/tests/system-ovn.at
>>> @@ -16033,3 +16033,382 @@ OVS_TRAFFIC_VSWITCHD_STOP(["/.*error receiving.*/d
>>> AT_CLEANUP
>>> ])
>>>
>>> +OVN_FOR_EACH_NORTHD([
>>> +AT_SETUP([route-exchange for LB VIPs with gateway router IPv4])
>>> +AT_KEYWORDS([route-exchange])
>>> +
>>> +CHECK_VRF()
>>> +CHECK_CONNTRACK()
>>> +CHECK_CONNTRACK_NAT()
>>> +ovn_start
>>> +OVS_TRAFFIC_VSWITCHD_START()
>>> +ADD_BR([br-int])
>>> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone])
>>> +
>>> +# Set external-ids in br-int needed for ovn-controller
>>> +ovs-vsctl \
>>> + -- set Open_vSwitch . external-ids:system-id=hv1 \
>>> + -- set Open_vSwitch .
>>> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
>>> + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
>>> + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
>>> + -- set bridge br-int fail-mode=secure
>>> other-config:disable-in-band=true
>>> +
>>> +# Start ovn-controller
>>> +start_daemon ovn-controller
>>> +
>>> +ovn-appctl vlog/set route_exchange
>>> +check ovn-nbctl -- lr-add R1 \
>>> + -- set Logical_Router R1 options:requested-tnl-key=1000
>>> options:dynamic-routing=true
>>> +
>>> +check ovn-nbctl ls-add sw0
>>> +check ovn-nbctl ls-add public
>>> +
>>> +check ovn-nbctl --wait=hv sync
>>> +
>>> +AT_CHECK([ip link | grep -q ovnvrf1000:.*UP], [1])
>>> +
>>> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 192.168.1.1/24
>>> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 172.16.1.1/24 \
>>> + -- lrp-set-options rp-public \
>>> + dynamic-routing-maintain-vrf=true \
>>> + dynamic-routing-lb-vips=true
>>> +
>>> +check ovn-nbctl set logical_router R1 options:chassis=hv1
>>> +
>>> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \
>>> + type=router options:router-port=rp-sw0 \
>>> + -- lsp-set-addresses sw0-rp router
>>> +
>>> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port
>>> public-rp \
>>> + type=router options:router-port=rp-public \
>>> + -- lsp-set-addresses public-rp router
>>> +
>>> +check ovs-vsctl set Open_vSwitch .
>>> external-ids:ovn-bridge-mappings=phynet:br-ext
>>> +
>>> +check ovn-nbctl lsp-add public public1 \
>>> + -- lsp-set-addresses public1 unknown \
>>> + -- lsp-set-type public1 localnet \
>>> + -- lsp-set-options public1 network_name=phynet
>>> +
>>> +check ovn-nbctl --wait=hv sync
>>> +
>>> +AT_CHECK([test `ip route show table 1000 | wc -l` -eq 1], [1])
>>> +
>>> +
>>> +# Create a load balancer and associate to R1
>>> +check ovn-nbctl lb-add lb1 172.16.1.150:80 172.16.1.100:80
>>> +check ovn-nbctl lr-lb-add R1 lb1
>>> +
>>> +check ovn-nbctl --wait=hv sync
>>> +
>>> +AT_CHECK([ip link | grep -q ovnvrf1000:.*UP])
>>> +AT_CHECK([test `ip route show table 1000 | wc -l` -eq 1])
>>> +AT_CHECK([ip route show table 1000 | grep -q 172.16.1.150])
>>> +
>>> +OVS_APP_EXIT_AND_WAIT([ovn-controller])
>>> +
>>> +# Ensure system resources are cleaned up
>>> +AT_CHECK([ip link | grep -q ovnvrf1000:.*UP], [1])
>>> +AT_CHECK([test `ip route show table 1000 | wc -l` -eq 1], [1])
>>> +
>>> +as ovn-sb
>>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>>> +
>>> +as ovn-nb
>>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>>> +
>>> +as northd
>>> +OVS_APP_EXIT_AND_WAIT([ovn-northd])
>>> +
>>> +as
>>> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
>>> +/Failed to acquire.*/d
>>> +/connection dropped.*/d"])
>>> +AT_CLEANUP
>>> +])
>>> +
>>> +OVN_FOR_EACH_NORTHD([
>>> +AT_SETUP([route-exchange for LB VIPs with gateway router IPv6])
>>> +AT_KEYWORDS([route-exchange])
>>> +
>>> +CHECK_VRF()
>>> +CHECK_CONNTRACK()
>>> +CHECK_CONNTRACK_NAT()
>>> +ovn_start
>>> +OVS_TRAFFIC_VSWITCHD_START()
>>> +ADD_BR([br-int])
>>> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone])
>>> +
>>> +# Set external-ids in br-int needed for ovn-controller
>>> +ovs-vsctl \
>>> + -- set Open_vSwitch . external-ids:system-id=hv1 \
>>> + -- set Open_vSwitch .
>>> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
>>> + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
>>> + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
>>> + -- set bridge br-int fail-mode=secure
>>> other-config:disable-in-band=true
>>> +
>>> +# Start ovn-controller
>>> +start_daemon ovn-controller
>>> +
>>> +ovn-appctl vlog/set route_exchange
>>> +check ovn-nbctl -- lr-add R1 \
>>> + -- set Logical_Router R1 options:requested-tnl-key=1001
>>> options:dynamic-routing=true
>>> +
>>> +check ovn-nbctl ls-add sw0
>>> +check ovn-nbctl ls-add public
>>> +
>>> +check ovn-nbctl --wait=hv sync
>>> +
>>> +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP], [1])
>>> +
>>> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 2001:db8:100::1/64
>>> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03
>>> 2001:db8:1001::1/64 \
>>> + -- lrp-set-options rp-public \
>>> + dynamic-routing-maintain-vrf=true \
>>> + dynamic-routing-lb-vips=true
>>> +
>>> +check ovn-nbctl set logical_router R1 options:chassis=hv1
>>> +
>>> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \
>>> + type=router options:router-port=rp-sw0 \
>>> + -- lsp-set-addresses sw0-rp router
>>> +
>>> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port
>>> public-rp \
>>> + type=router options:router-port=rp-public \
>>> + -- lsp-set-addresses public-rp router
>>> +
>>> +check ovs-vsctl set Open_vSwitch .
>>> external-ids:ovn-bridge-mappings=phynet:br-ext
>>> +
>>> +check ovn-nbctl lsp-add public public1 \
>>> + -- lsp-set-addresses public1 unknown \
>>> + -- lsp-set-type public1 localnet \
>>> + -- lsp-set-options public1 network_name=phynet
>>> +
>>> +check ovn-nbctl --wait=hv sync
>>> +
>>> +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1], [1])
>>> +
>>> +# Create a load balancer and associate to R1
>>> +check ovn-nbctl lb-add lb1 [[2001:db8:1001::150]]:80
>>> [[2001:db8:1001::100]]:80
>>> +check ovn-nbctl lr-lb-add R1 lb1
>>> +
>>> +check ovn-nbctl --wait=hv sync
>>> +
>>> +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP])
>>> +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1])
>>> +AT_CHECK([ip -6 route show table 1001 | grep -q 2001:db8:1001::150])
>>> +
>>> +
>>> +OVS_APP_EXIT_AND_WAIT([ovn-controller])
>>> +
>>> +# Ensure system resources are cleaned up
>>> +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP], [1])
>>> +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1], [1])
>>> +
>>> +as ovn-sb
>>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>>> +
>>> +as ovn-nb
>>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>>> +
>>> +as northd
>>> +OVS_APP_EXIT_AND_WAIT([ovn-northd])
>>> +
>>> +as
>>> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
>>> +/Failed to acquire.*/d
>>> +/connection dropped.*/d"])
>>> +AT_CLEANUP
>>> +])
>>> +
>>> +OVN_FOR_EACH_NORTHD([
>>> +AT_SETUP([route-exchange for DNAT and DNAT_AND_SNAT with gateway router
>>> IPv4])
>>> +AT_KEYWORDS([route-exchange])
>>> +
>>> +CHECK_VRF()
>>> +CHECK_CONNTRACK()
>>> +CHECK_CONNTRACK_NAT()
>>> +ovn_start
>>> +OVS_TRAFFIC_VSWITCHD_START()
>>> +ADD_BR([br-int])
>>> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone])
>>> +
>>> +# Set external-ids in br-int needed for ovn-controller
>>> +ovs-vsctl \
>>> + -- set Open_vSwitch . external-ids:system-id=hv1 \
>>> + -- set Open_vSwitch .
>>> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
>>> + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
>>> + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
>>> + -- set bridge br-int fail-mode=secure
>>> other-config:disable-in-band=true
>>> +
>>> +# Start ovn-controller
>>> +start_daemon ovn-controller
>>> +
>>> +ovn-appctl vlog/set route_exchange
>>> +check ovn-nbctl -- lr-add R1 \
>>> + -- set Logical_Router R1 options:requested-tnl-key=1002
>>> options:dynamic-routing=true
>>> +
>>> +check ovn-nbctl ls-add sw0
>>> +check ovn-nbctl ls-add public
>>> +
>>> +check ovn-nbctl --wait=hv sync
>>> +
>>> +AT_CHECK([ip link | grep -q ovnvrf1002:.*UP], [1])
>>> +
>>> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 192.168.1.1/24
>>> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03 172.16.1.1/24 \
>>> + -- lrp-set-options rp-public \
>>> + dynamic-routing-maintain-vrf=true \
>>> + dynamic-routing-nat=true
>>> +
>>> +check ovn-nbctl set logical_router R1 options:chassis=hv1
>>> +
>>> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \
>>> + type=router options:router-port=rp-sw0 \
>>> + -- lsp-set-addresses sw0-rp router
>>> +
>>> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port
>>> public-rp \
>>> + type=router options:router-port=rp-public \
>>> + -- lsp-set-addresses public-rp router
>>> +
>>> +check ovs-vsctl set Open_vSwitch .
>>> external-ids:ovn-bridge-mappings=phynet:br-ext
>>> +
>>> +check ovn-nbctl lsp-add public public1 \
>>> + -- lsp-set-addresses public1 unknown \
>>> + -- lsp-set-type public1 localnet \
>>> + -- lsp-set-options public1 network_name=phynet
>>> +
>>> +check ovn-nbctl --wait=hv sync
>>> +
>>> +AT_CHECK([test `ip route show table 1002 | wc -l` -eq 2], [1])
>>> +
>>> +# Create dnat_and_snat, dnat rules in R1
>>> +check ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.10 192.168.1.10
>>> +check ovn-nbctl lr-nat-add R1 dnat 172.16.1.11 192.168.1.11
>>> +
>>> +check ovn-nbctl --wait=hv sync
>>> +
>>> +AT_CHECK([ip link | grep -q ovnvrf1002:.*UP])
>>> +AT_CHECK([test `ip route show table 1002 | wc -l` -eq 2])
>>> +AT_CHECK([ip route show table 1002 | grep -q 172.16.1.10])
>>> +AT_CHECK([ip route show table 1002 | grep -q 172.16.1.11])
>>> +
>>> +
>>> +OVS_APP_EXIT_AND_WAIT([ovn-controller])
>>> +
>>> +# Ensure system resources are cleaned up
>>> +AT_CHECK([ip link | grep -q ovnvrf1002:.*UP], [1])
>>> +AT_CHECK([test `ip route show table 1002 | wc -l` -eq 1], [1])
>>> +
>>> +as ovn-sb
>>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>>> +
>>> +as ovn-nb
>>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>>> +
>>> +as northd
>>> +OVS_APP_EXIT_AND_WAIT([ovn-northd])
>>> +
>>> +as
>>> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
>>> +/Failed to acquire.*/d
>>> +/connection dropped.*/d"])
>>> +AT_CLEANUP
>>> +])
>>> +
>>> +OVN_FOR_EACH_NORTHD([
>>> +AT_SETUP([route-exchange for DNAT and DNAT_AND_SNAT with gateway router
>>> IPv6])
>>> +AT_KEYWORDS([route-exchange])
>>> +
>>> +CHECK_VRF()
>>> +CHECK_CONNTRACK()
>>> +CHECK_CONNTRACK_NAT()
>>> +ovn_start
>>> +OVS_TRAFFIC_VSWITCHD_START()
>>> +ADD_BR([br-int])
>>> +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone])
>>> +
>>> +# Set external-ids in br-int needed for ovn-controller
>>> +ovs-vsctl \
>>> + -- set Open_vSwitch . external-ids:system-id=hv1 \
>>> + -- set Open_vSwitch .
>>> external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
>>> + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
>>> + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
>>> + -- set bridge br-int fail-mode=secure
>>> other-config:disable-in-band=true
>>> +
>>> +# Start ovn-controller
>>> +start_daemon ovn-controller
>>> +
>>> +ovn-appctl vlog/set route_exchange
>>> +check ovn-nbctl -- lr-add R1 \
>>> + -- set Logical_Router R1 options:requested-tnl-key=1003
>>> options:dynamic-routing=true
>>> +
>>> +check ovn-nbctl ls-add sw0
>>> +check ovn-nbctl ls-add public
>>> +
>>> +check ovn-nbctl --wait=hv sync
>>> +
>>> +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP], [1])
>>> +
>>> +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 2001:db8:100::1/64
>>> +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03
>>> 2001:db8:1003::1/64 \
>>> + -- lrp-set-options rp-public \
>>> + dynamic-routing-maintain-vrf=true \
>>> + dynamic-routing-nat=true
>>> +
>>> +check ovn-nbctl set logical_router R1 options:chassis=hv1
>>> +
>>> +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp \
>>> + type=router options:router-port=rp-sw0 \
>>> + -- lsp-set-addresses sw0-rp router
>>> +
>>> +check ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port
>>> public-rp \
>>> + type=router options:router-port=rp-public \
>>> + -- lsp-set-addresses public-rp router
>>> +
>>> +check ovs-vsctl set Open_vSwitch .
>>> external-ids:ovn-bridge-mappings=phynet:br-ext
>>> +
>>> +check ovn-nbctl lsp-add public public1 \
>>> + -- lsp-set-addresses public1 unknown \
>>> + -- lsp-set-type public1 localnet \
>>> + -- lsp-set-options public1 network_name=phynet
>>> +
>>> +check ovn-nbctl --wait=hv sync
>>> +
>>> +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2], [1])
>>> +
>>> +# Create dnat_and_snat, dnat rules in R1
>>> +check ovn-nbctl lr-nat-add R1 \
>>> + dnat_and_snat 2001:db8:1003::150 2001:db8:100::100
>>> +check ovn-nbctl lr-nat-add R1 \
>>> + dnat 2001:db8:1003::151 2001:db8:100::100
>>> +
>>> +check ovn-nbctl --wait=hv sync
>>> +
>>> +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP])
>>> +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2])
>>> +AT_CHECK([ip -6 route show table 1003 | grep -q 2001:db8:1003::150])
>>> +AT_CHECK([ip -6 route show table 1003 | grep -q 2001:db8:1003::151])
>>> +
>>> +OVS_APP_EXIT_AND_WAIT([ovn-controller])
>>> +
>>> +# Ensure system resources are cleaned up
>>> +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP], [1])
>>> +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2], [1])
>>> +
>>> +as ovn-sb
>>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>>> +
>>> +as ovn-nb
>>> +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
>>> +
>>> +as northd
>>> +OVS_APP_EXIT_AND_WAIT([ovn-northd])
>>> +
>>> +as
>>> +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
>>> +/Failed to acquire.*/d
>>> +/connection dropped.*/d"])
>>> +AT_CLEANUP
>>> +])
>>> +
>>
>> I didn't review the tests yet but I do plan to. I wanted to start the
>> discussion about the rest of the implementation first though. I'd love
>> it if we could get this feature in 25.03.0.
>>
>> Regards,
>> Dumitru
>>
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
