Hi Martin,
On Wed, Feb 05, 2025 at 01:30:49AM +0100, [email protected] wrote:
> Hi Felix,
> thank you for great feedback, please see my comments below.
>
> On Tue, 2025-02-04 at 16:50 +0100, Felix Huettner wrote:
> > On Tue, Feb 04, 2025 at 12:39:11AM +0100, Martin Kalcok wrote:
> > > This change builds on top of the new "dynamic routing" OVN feature
> > > that allows advertising routes to the fabric network. When LR
> > > option
> > > "dynamic-routing" is set on the router, following two new LRP
> > > options
> > > become available:
> > >
> > > * dynamic-routing-nat - When set to "true", ovn-controller will
> > > advertise
> > > routes for external NAT IPs valid for the
> > > LRP.
> > > * dynamic-routing-lb-vips - When set to "true", ovn-controller will
> > > advertise
> > > host routes to LB VIPs via the LRP.
> > >
> > > Co-authored-by: Frode Nordahl <[email protected]>
> > > Signed-off-by: Frode Nordahl <[email protected]>
> > > Signed-off-by: Martin Kalcok <[email protected]>
> >
> > Hi Martin,
> >
> > thanks for the patch, i'll just comment on what i found.
> >
> > > ---
> > > NEWS | 17 ++
> > > northd/en-advertised-route-sync.c | 170 ++++++++++++++
> > > northd/en-northd.c | 5 +-
> > > northd/inc-proc-northd.c | 5 +
> > > northd/northd.c | 138 ++++++++++-
> > > northd/northd.h | 8 +-
> > > ovn-nb.xml | 32 +++
> > > ovs | 2 +-
> > > tests/ovn-northd.at | 139 +++++++++++
> > > tests/system-ovn.at | 379
> > > ++++++++++++++++++++++++++++++
> > > 10 files changed, 890 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/NEWS b/NEWS
> > > index eca17b728..137ff48fd 100644
> > > --- a/NEWS
> > > +++ b/NEWS
> > > @@ -61,6 +61,23 @@ Post v24.09.0
> > > * Add the option "dynamic-routing-ifname" to LRPs. If set
> > > only routes
> > > learned from a linux iterfaces with that name are treated
> > > as relevant
> > > routes for this LRP.
> > > + - Add the option "dynamic-routing" to Logical Routers. If set
> > > to true all
> > > + static and connected routes attached to the router are shared
> > > to the
> > > + southbound "Route" table for sharing outside of OVN.
> > > + The routes can furthe be filtered by setting `dynamic-
> > > routing-connected`
> > > + and `dynamic-routing-static` on the LR or LRP. The LRP
> > > settings overwrite
> > > + the LR settings for all routes using this interface as an
> > > exit.
> > > + - Allow Logical Routers to dynamically learn routes from
> > > outside the fabric.
> > > + Routes entered into the "Route" table in the southbound
> > > database will be
> > > + learned by the respective LR. They are included in the route
> > > table with
> > > + a lower priority than static routes.
> > > + - Add the option "dynamic-routing-connected-as-host-routes" to
> > > LRPs. If set
> > > + to true then connected routes are announced as individual
> > > host routes.
> > > + - Add 'dynamic-routing-lb-vips' LRP option. If set to true, the
> > > LRP can be
> > > + used to advertise host paths to the Load Balancer VIPs
> > > associated with the
> > > + LR.
> > > + - Add 'dynamic-routing-nat' LRP option. If set to true, the LRP
> > > can be used
> > > + to advertise external NAT IPs associated with it.
> > >
> > > OVN v24.09.0 - 13 Sep 2024
> > > --------------------------
> > > diff --git a/northd/en-advertised-route-sync.c b/northd/en-
> > > advertised-route-sync.c
> > > index 02b3aba7a..8a1da02fd 100644
> > > --- a/northd/en-advertised-route-sync.c
> > > +++ b/northd/en-advertised-route-sync.c
> > > @@ -16,6 +16,7 @@
> > >
> > > #include <config.h>
> > >
> > > +#include "openvswitch/vlog.h"
> > > #include "smap.h"
> > > #include "stopwatch.h"
> > > #include "northd.h"
> > > @@ -26,6 +27,8 @@
> > > #include "openvswitch/hmap.h"
> > > #include "ovn-util.h"
> > >
> > > +VLOG_DEFINE_THIS_MODULE(en_advertised_route_sync);
> > > +
> > > static void
> > > advertised_route_table_sync(
> > > struct ovsdb_idl_txn *ovnsb_txn,
> > > @@ -312,6 +315,157 @@ publish_host_routes(struct hmap *sync_routes,
> > > }
> > > }
> > >
> > > +/* This function searches for an ovn_port with specific "op_ip"
> > > + * IP address in all LS datapaths directly connected to the
> > > + * LR datapath "od".
> > > + * If no match is found, this function returns NULL.*/
> >
> > I see multiple points to optimize the function. Not sure if they are
> > all
> > relevant or worth it, but i just want to share the options i see.
> >
> > > +static struct ovn_port *
> > > +find_port_in_connected_ls(struct ovn_datapath *od, char *op_ip)
> > > +{
> >
> > AFAIK there can not be a port found if op_ip is not a host ip (so /32
> > or
> > /128). So this could directly return NULL. This would directly cover
> > e.g. large SNAT pools.
>
> I think you are correct. This function is currently used with NAT
> logical IPs and LB backends, so if there's snat rule with /prefix, we
> could outright skip it.
>
> >
> > > + for (int i = 0; i < od->n_ls_peers; i++) {
> > > + struct ovn_datapath *peer_od = od->ls_peers[i];
> > > + struct ovn_port *op;
> >
> > We would only need to search on the op if the network of the op
> > matches
> > to the op_ip. So there is no need to search missmatched subnets. If
> > such
> > a validation is efficient, i don't know.
>
> By "network of the op", do you mean op->lrp_networks? wouldn't that
> lead to similar iterative check where we first check op->lrp_networks-
> >ipv4_addrs[] / op->lrp_networks->ipv6_addrs and then we'd have to op-
> >lsp_addrs for direct match anyway?
Yes, thats right. So it then does not really help.
>
> >
> > > + HMAP_FOR_EACH (op, dp_node, &peer_od->ports) {
> > > + for (int j = 0; j < op->n_lsp_addrs; j++) {
> > > + struct lport_addresses *addrs = &op->lsp_addrs[j];
> >
> > We could check if op_ip is a v4 or v6 address so we only need one
> > loop
> > based on that.
>
> That is a good point, we'd have to parse op_ip into an IP address
> structure, but I'm sure there are functions for that.
>
> >
> > > + for (int k = 0; k < addrs->n_ipv4_addrs; k++) {
> > > + if (!strcmp(op_ip, addrs-
> > > >ipv4_addrs[k].addr_s)) {
> > > + return op;
> > > + }
> > > + }
> > > + for (int k = 0; k < addrs->n_ipv6_addrs; k++) {
> > > + if (!strcmp(op_ip, addrs-
> > > >ipv6_addrs[k].addr_s)) {
> > > + return op;
> > > + }
> > > + }
> > > + }
> > > + }
> > > + }
> > > + return NULL;
> > > +}
> > > +
> > > +/* Publish parsed_route as a Advertised_Route record. The main
> > > purpose
> > > + * of this function is to find LSP with IP address that matches
> > > the
> > > + * logical_ip of a NAT rule on the route's out_port datapath. This
> > > port is
> > > + * then set as a tracked_port for the route.
> > > + * Search is performed in all LS datapaths directly connected to
> > > the
> > > + * route's out_port datapath.
> > > + * If no suitable tracked_port is found, route is still published,
> > > but
> > > + * without the tracked_port set.*/
> > > +static void
> > > +publish_nat_route(struct hmap *route_map,
> > > + const struct parsed_route *route)
> > > +{
> > > + const struct ovn_datapath *advertised_dp = route->od;
> > > + const struct ovn_port *ext_nat_port = route->out_port;
> > > +
> > > + if (!advertised_dp->nbr || !ext_nat_port->od) {
> > > + return;
> > > + }
> > > +
> > > + char *ip_prefix = normalize_v46_prefix(&route->prefix,
> > > + route->plen);
> > > + char *logical_ip = NULL;
> > > +
> > > + /* Find NAT rule with external IP that matches the route's
> > > + * ip_prefix.*/
> > > + for (int i = 0; i < ext_nat_port->od->nbr->n_nat; i++) {
> > > + struct nbrec_nat *nat = ext_nat_port->od->nbr->nat[i];
> > > + if (!strcmp(nat->external_ip, ip_prefix)) {
> > > + logical_ip = nat->logical_ip;
> > > + break;
> > > + }
> > > + }
> > > +
> > > + if (!logical_ip) {
> > > + return;
> > > + }
> > > +
> > > + const struct sbrec_port_binding *tracked_pb = NULL;
> > > + struct ovn_port *tracked_port =
> > > find_port_in_connected_ls(ext_nat_port->od,
> > > +
> > > logical_ip);
> > > + if (tracked_port) {
> > > + tracked_pb = tracked_port->sb;
> > > + }
> > > +
> > > + char *ip_with_mask = xasprintf("%s/%d", ip_prefix, route-
> > > >plen);
> > > + ar_alloc_entry(route_map,
> > > + route->od->sb,
> > > + route->out_port->sb,
> > > + ip_with_mask,
> > > + tracked_pb);
> > > + free(ip_prefix);
> > > +}
> > > +
> > > +/* Publish parsed_route as a Advertised_Route record. The main
> > > purpose
> > > + * of this function is to find LSP with IP address that matches
> > > the
> > > + * endpoint IP of a LB on the route's out_port datapath. This port
> > > is then
> > > + * set as a tracked_port for the route.
> > > + * Search is performed in all LS datapaths directly connected to
> > > the
> > > + * route's out_port datapath.
> > > + * If no suitable tracked_port is found, route is still published,
> > > but
> > > + * without the tracked_port set.*/
> > > +static void
> > > +publish_lb_route(struct hmap *route_map,
> > > + const struct parsed_route *route)
> > > +{
> > > + const struct ovn_datapath *advertised_dp = route->od;
> > > + const struct ovn_port *ext_lb_port = route->out_port;
> > > +
> > > + if (!advertised_dp->nbr || !ext_lb_port->od) {
> > > + return;
> > > + }
> > > +
> > > + char *ip_prefix = normalize_v46_prefix(&route->prefix,
> > > + route->plen);
> > > +
> > > + for (int i = 0; i < ext_lb_port->od->nbr->n_load_balancer;
> > > i++) {
> > > + struct nbrec_load_balancer *lb =
> > > + ext_lb_port->od->nbr->load_balancer[i];
> > > + struct smap lb_vips = SMAP_INITIALIZER(&lb_vips);
> > > + struct smap_node *node;
> > > + SMAP_FOR_EACH (node, &lb->vips) {
> > > + struct ovn_lb_vip *lb_vip = xmalloc(sizeof(*lb_vip));
> > > + char *error = ovn_lb_vip_init(lb_vip, node->key,
> > > + node->value, false,
> > > AF_INET6);
> > > + if (error) {
> > > + static struct vlog_rate_limit rl =
> > > VLOG_RATE_LIMIT_INIT(5, 1);
> > > + VLOG_WARN_RL(&rl, "Failed to parse LB VIP: %s",
> > > error);
> > > + ovn_lb_vip_destroy(lb_vip);
> > > + free(error);
> > > + continue;
> > > + }
> > > +
> > > + /* Continue only for LBs whose VIP matches route's
> > > ip_prefix.*/
> > > + if (strcmp(lb_vip->vip_str, ip_prefix)) {
> > > + ovn_lb_vip_destroy(lb_vip);
> > > + continue;
> > > + }
> > > +
> > > + /* Find LSP that matches at least one endpoint IP of
> > > the LB.*/
> > > + const struct sbrec_port_binding *tracked_pb = NULL;
> > > + for (int j = 0; j < lb_vip->n_backends; j++) {
> > > + char *backend = lb_vip->backends[j].ip_str;
> > > + struct ovn_port *tracked_port =
> > > + find_port_in_connected_ls(ext_lb_port->od,
> > > backend);
> > > + if (tracked_port) {
> > > + tracked_pb = tracked_port->sb;
> > > + break;
> > > + }
> > > + }
> > > +
> > > + char *ip_with_mask = xasprintf("%s/%d", ip_prefix,
> > > route->plen);
> > > + ar_alloc_entry(route_map,
> > > + route->od->sb,
> > > + route->out_port->sb,
> > > + ip_with_mask,
> > > + tracked_pb);
> > > + ovn_lb_vip_destroy(lb_vip);
> > > + free(ip_prefix);
> > > + }
> > > + }
> > > +}
> > > +
> > > static void
> > > advertised_route_table_sync(
> > > struct ovsdb_idl_txn *ovnsb_txn,
> > > @@ -359,6 +513,22 @@ advertised_route_table_sync(
> > > !route->out_port->dynamic_routing_static) {
> > > continue;
> > > }
> > > + if (route->source == ROUTE_SOURCE_NAT) {
> > > + if (!smap_get_bool(&route->out_port->nbrp->options,
> > > + "dynamic-routing-nat", false)) {
> > > + continue;
> > > + }
> > > + publish_nat_route(&sync_routes, route);
> > > + continue;
> > > + }
> > > + if (route->source == ROUTE_SOURCE_LB) {
> > > + if (!smap_get_bool(&route->out_port->nbrp->options,
> > > + "dynamic-routing-lb-vips", false))
> > > {
> > > + continue;
> > > + }
> > > + publish_lb_route(&sync_routes, route);
> > > + continue;
> > > + }
> > >
> > > char *ip_prefix = normalize_v46_prefix(&route->prefix,
> > > route->plen);
> > > diff --git a/northd/en-northd.c b/northd/en-northd.c
> > > index c7d1ebcb3..60e295628 100644
> > > --- a/northd/en-northd.c
> > > +++ b/northd/en-northd.c
> > > @@ -316,6 +316,8 @@ en_routes_run(struct engine_node *node, void
> > > *data)
> > > {
> > > struct northd_data *northd_data =
> > > engine_get_input_data("northd", node);
> > > struct bfd_data *bfd_data = engine_get_input_data("bfd",
> > > node);
> > > + struct ed_type_lr_stateful *lr_stateful_data =
> > > + engine_get_input_data("lr_stateful", node);
> > > struct routes_data *routes_data = data;
> > >
> > > routes_destroy(data);
> > > @@ -330,7 +332,8 @@ en_routes_run(struct engine_node *node, void
> > > *data)
> > > route_table_name);
> > > }
> > >
> > > - build_parsed_routes(od, &northd_data->lr_ports,
> > > + build_parsed_routes(od, &lr_stateful_data->table,
> > > + &northd_data->lr_ports,
> > > &bfd_data->bfd_connections,
> > > &routes_data->parsed_routes,
> > > &routes_data->route_tables,
> > > diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c
> > > index 86b7b999e..d74c2e3dc 100644
> > > --- a/northd/inc-proc-northd.c
> > > +++ b/northd/inc-proc-northd.c
> > > @@ -267,6 +267,11 @@ void inc_proc_northd_init(struct
> > > ovsdb_idl_loop *nb,
> > > engine_add_input(&en_routes, &en_bfd, NULL);
> > > engine_add_input(&en_routes, &en_northd,
> > > routes_northd_change_handler);
> > > + engine_add_input(&en_routes, &en_lr_nat,
> > > + NULL);
> > > + engine_add_input(&en_routes, &en_lb_data,
> > > + NULL);
> > > + engine_add_input(&en_routes, &en_lr_stateful,
> > > engine_noop_handler);
> > >
> > > engine_add_input(&en_bfd_sync, &en_bfd, NULL);
> > > engine_add_input(&en_bfd_sync, &en_nb_bfd, NULL);
> > > diff --git a/northd/northd.c b/northd/northd.c
> > > index 71adb88d7..a41ad7642 100644
> > > --- a/northd/northd.c
> > > +++ b/northd/northd.c
> > > @@ -11075,8 +11075,111 @@ parsed_routes_add_connected(const struct
> > > ovn_datapath *od,
> > > }
> > > }
> > >
> > > +/* Add "parsed_route"s to the "routes" for each IP address in
> > > laddrs.*/
> > > +static void
> > > +lport_addrs_to_parsed_routes(const struct ovn_datapath *od,
> > > + const struct ovn_port *op,
> > > + const struct lport_addresses *laddrs,
> > > + enum route_source route_type,
> > > + struct hmap *routes,
> > > + bool install_lflow)
> > > +{
> > > + for (int i = 0; i < laddrs->n_ipv4_addrs; i++) {
> > > + struct ipv4_netaddr *addr = &laddrs->ipv4_addrs[i];
> > > + struct in6_addr prefix;
> > > + ip46_parse(addr->network_s, &prefix);
> > > + parsed_route_add(od, NULL, &prefix, addr->plen,
> > > + false, addr->addr_s, op,
> > > + 0, false,
> > > + false, NULL, route_type,
> > > + &op->nbrp->header_, routes,
> > > install_lflow);
> > > + }
> > > + for (int i = 0; i < laddrs->n_ipv6_addrs; i++) {
> > > + struct ipv6_netaddr *addr = &laddrs->ipv6_addrs[i];
> > > + parsed_route_add(od, NULL, &addr->addr, addr->plen,
> > > + false, addr->addr_s, op,
> > > + 0, false,
> > > + false, NULL, route_type,
> > > + &op->nbrp->header_, routes,
> > > install_lflow);
> > > + }
> > > +}
> > > +
> > > +/* Add routes of all NAT external_addresses on the ovn_port "op",
> > > to be
> > > + * advertised by the datapath "od".
> > > + * If "routeable_only" is true, only NATs with option
> > > "add_route=true" will
> > > + * be considered.
> > > + * Note that the "op" does not necessarily need to be in the "od"
> > > datapath.
> > > + * Such use-case is for advertising external NAT IPs of LRPs in
> > > neighboring
> > > + * routers and should be paired with routable_only=true.
> > > + */
> > > +static void
> > > +parsed_routes_add_nat(const struct ovn_datapath *od,
> > > + const struct ovn_port *op,
> > > + struct hmap *routes,
> > > + bool routable_only)
> > > +{
> > > + if (!op->nbrp || !smap_get_bool(&op->nbrp->options,
> > > + "dynamic-routing-nat", false))
> > > {
> > > + return;
> > > + }
> > > + size_t n_nats = 0;
> > > + char **nats = NULL;
> > > + nats = get_nat_addresses(op, &n_nats, routable_only, false,
> > > NULL, true);
> > > +
> > > + for (size_t i = 0; i < n_nats; i++) {
> > > + struct lport_addresses *laddrs = xzalloc(sizeof *laddrs);
> > > + int ofs = 0;
> > > + extract_addresses(nats[i], laddrs, &ofs);
> > > + lport_addrs_to_parsed_routes(od, op, laddrs,
> > > ROUTE_SOURCE_NAT,
> > > + routes, false);
> > > + destroy_lport_addresses(laddrs);
> > > + free(nats[i]);
> > > + }
> > > + free(nats);
> > > +}
> > > +
> > > +/* Add routes of all LB VIPs on the ovn_port "op", to be
> > > advertised by the
> > > + * datapath "od".
> > > + * If "routeable_only" is true, only LBs with option
> > > "add_route=true" will
> > > + * be considered.
> > > + * Note that the "op" does not necessarily need to be in the "od"
> > > datapath.
> > > + * Such use-case is for advertising LB VIPs of LRPs in neighboring
> > > routers
> > > + * and should be paired with routable_only=true.
> > > + */
> > > +static void
> > > +parsed_routes_add_lb(const struct ovn_datapath *od,
> > > + const struct ovn_port *op,
> > > + struct hmap *routes,
> > > + const struct lr_stateful_record
> > > *lr_stateful_rec,
> > > + bool routable_only)
> > > +{
> > > + if (!op->nbrp || !smap_get_bool(&op->nbrp->options,
> > > + "dynamic-routing-lb-vips",
> > > false)) {
> > > + return;
> > > + }
> > > +
> > > + if (!lr_stateful_rec) {
> > > + return;
> > > + }
> > > +
> > > + struct ds addresses = DS_EMPTY_INITIALIZER;
> > > + get_lb_addresses(lr_stateful_rec, &addresses, routable_only);
> > > +
> > > + struct lport_addresses *laddrs = xzalloc(sizeof *laddrs);
> > > + extract_ip_addresses(ds_cstr(&addresses), laddrs);
> > > +
> > > + lport_addrs_to_parsed_routes(od, op, laddrs, ROUTE_SOURCE_LB,
> > > + routes, false);
> > > +
> > > + destroy_lport_addresses(laddrs);
> > > + ds_destroy(&addresses);
> > > +
> > > +}
> > > +
> > > void
> > > -build_parsed_routes(const struct ovn_datapath *od, const struct
> > > hmap *lr_ports,
> > > +build_parsed_routes(const struct ovn_datapath *od,
> > > + const struct lr_stateful_table
> > > *lr_stateful_table,
> > > + const struct hmap *lr_ports,
> > > const struct hmap *bfd_connections, struct
> > > hmap *routes,
> > > struct simap *route_tables,
> > > struct hmap *bfd_active_connections)
> > > @@ -11096,7 +11199,34 @@ build_parsed_routes(const struct
> > > ovn_datapath *od, const struct hmap *lr_ports,
> > >
> > > const struct ovn_port *op;
> > > HMAP_FOR_EACH (op, dp_node, &od->ports) {
> > > + const struct lr_stateful_record *lr_stateful_rec = NULL;
> > > + lr_stateful_rec = lr_stateful_table_find_by_index(
> > > + lr_stateful_table, op->od->index);
> > > +
> > > parsed_routes_add_connected(od, op, routes);
> > > + parsed_routes_add_nat(od, op, routes, false);
> > > + parsed_routes_add_lb(od, op, routes, lr_stateful_rec,
> > > false);
> > > + }
> > > +
> > > + /* For GW routers we traverse also neighboring LR in search
> > > for
> > > + * LBs and NATs with "add_route" option set to "true". Such
> > > NATs/LBs
> > > + * install logical flows for their external IP addresses to
> > > their
> > > + * neighboring routers, and as such, we can advertise them as
> > > routes.
> > > + * The LRP on the neighboring router needs to have appropriate
> > > + * "dynamic-routing-{nat,lb-vips}" option set for this to take
> > > effect.*/
> > > + for (size_t i = 0; od->is_gw_router && i < od->n_ls_peers;
> > > i++) {
> > > + for (size_t j = 0; j < od->ls_peers[i]->n_router_ports;
> > > j++) {
> > > + struct ovn_port *router_port;
> > > + router_port = od->ls_peers[i]->router_ports[j]->peer;
> > > +
> > > + const struct lr_stateful_record *lr_stateful_rec =
> > > NULL;
> > > + lr_stateful_rec = lr_stateful_table_find_by_index(
> > > + lr_stateful_table, router_port->od->index);
> > > +
> > > + parsed_routes_add_nat(od, router_port, routes, true);
> > > + parsed_routes_add_lb(od, router_port, routes,
> > > + lr_stateful_rec, true);
> > > + }
> >
> > I am not sure if I understood this correctly.
> > But this seems for me to similar to what publish_host_routes in
> > https://patchwork.ozlabs.org/project/ovn/patch/8b1bbafc0ebaaf1e3182a182161894203af5b8d7.1738676766.git.felix.huettner@stackit.cloud/
> > implements.
> >
> > There we also iterate through all LSPs of one LRP. And if one of
> > these
> > LSPs is a LR we collect addresses from them. This just covers NAT and
> > no
> > LB, but maybe we can combine the logic?
> >
>
> I believe you are right, these do seem very similar in nature. I think
> it should be relatively easy to combine them. I guess the main
> difference is that we went with the recursive search here, when
> creating "parsed_routes", whereas your approach was to do the recursion
> later in the "advertised_route_table_sync". My worry is that if we move
> this recursive logic to a_r_table_sync for NAT and LBs, would the
> following scenario work?
>
> phys0 --- gw0 --- sw0 --- dr1
> |
> '----- dr2
>
> * phys0 is LS providing access to external net
> * gw0 is Gateway Router
> * sw0 is plain Logical Switch
> * dr1 and dr2 are Distributed Routers
> * gw0 has no NAT rules
> * gw0 has port with "dynamic-routing-nat" in phys0
> * dr1 and dr2 have nat rules in with external IPs in sw0
> * dr1 and dr2 have ports in sw0 with "dynamic-routing-nat"
>
> in current implementation we search all neighbors of gw0 for NAT
> external IPs and if they have "--add-route", we add them to
> "parsed_routes" and they get advertised to phys0.
> If we move this logic to "a_r_table_sync", it would not get triggered
> because gw0 has no "nat routes" on its own. right?
There is a testcase in
https://patchwork.ozlabs.org/project/ovn/patch/8b1bbafc0ebaaf1e3182a182161894203af5b8d7.1738676766.git.felix.huettner@stackit.cloud/
that checks i think exactly that scenario.
The implementation also tracks all LS and LRs that the routes depend on,
so that we can correctly recompute when needed.
For mapping the names in the test to your picture:
* gw0 -> lr0
* sw0 -> sw0
* dr1 -> lr1
We there advertise a route for a nat rule configured on dr1(lr1) on the
gw0(lr0) datapath. It is advertised using the logical port of gw->sw0
and a tracked port of dr1->sw0.
Do you think that covers that scenario?
>
> On the topic of "publish_host_routes" advertising neighboring NATs
> unconditionally, I don't think that's correct (see also [0]). Those
> NATs should be imo announced only if their corresponding port has
> "dynamic-routing-nat" option.
I treated options:dynamic-routing-connected-as-host-routes as a
specialization of the dynamic-routing-redistribute="connected".
Redistributing a connected route would have also included the NAT addresses so
i saw no need for an additional setting. But that can of course be changed.
But for me it feels like two different approaches on the same topic.
>
> And regarding the "publish_host_routes" only working for NATs, now that
> the "get_lb_addresses" function exists (see v2 of this series), it
> wouldn't be hard to add functionality for publishing LBs too.
I would guess it would just be similar to the NAT rules. We would just
need to figure out how to trigger recomputes when the LBs change.
Thanks a lot,
Felix
>
> [0]
> https://mail.openvswitch.org/pipermail/ovs-dev/2025-February/420382.html
>
> Thanks,
> Martin.
>
> > Thanks a lot,
> > Felix
> >
> > > }
> > >
> > > HMAP_FOR_EACH_SAFE (pr, key_node, routes) {
> > > @@ -11278,6 +11408,8 @@ route_source_to_offset(enum route_source
> > > source)
> > > {
> > > switch (source) {
> > > case ROUTE_SOURCE_CONNECTED:
> > > + case ROUTE_SOURCE_NAT:
> > > + case ROUTE_SOURCE_LB:
> > > return ROUTE_PRIO_OFFSET_CONNECTED;
> > > case ROUTE_SOURCE_STATIC:
> > > return ROUTE_PRIO_OFFSET_STATIC;
> > > @@ -13560,7 +13692,9 @@ build_route_flows_for_lrouter(
> > > struct parsed_route *route;
> > > HMAP_FOR_EACH_WITH_HASH (route, key_node, uuid_hash(&od->key),
> > > parsed_routes) {
> > > - if (route->source == ROUTE_SOURCE_CONNECTED) {
> > > + if (route->source == ROUTE_SOURCE_CONNECTED ||
> > > + route->source == ROUTE_SOURCE_NAT ||
> > > + route->source == ROUTE_SOURCE_LB) {
> > > unique_routes_add(&unique_routes, route);
> > > continue;
> > > }
> > > diff --git a/northd/northd.h b/northd/northd.h
> > > index fffaab3a0..d55909f76 100644
> > > --- a/northd/northd.h
> > > +++ b/northd/northd.h
> > > @@ -708,6 +708,10 @@ enum route_source {
> > > ROUTE_SOURCE_CONNECTED,
> > > /* The route is derived from a northbound static route entry.
> > > */
> > > ROUTE_SOURCE_STATIC,
> > > + /* Host route generated from NAT's external IP. */
> > > + ROUTE_SOURCE_NAT,
> > > + /* Host route generated from LB's external IP. */
> > > + ROUTE_SOURCE_LB,
> > > /* The route is dynamically learned by an ovn-controller. */
> > > ROUTE_SOURCE_LEARNED,
> > > };
> > > @@ -780,7 +784,9 @@ void northd_indices_create(struct northd_data
> > > *data,
> > >
> > > void route_policies_init(struct route_policies_data *);
> > > void route_policies_destroy(struct route_policies_data *);
> > > -void build_parsed_routes(const struct ovn_datapath *, const struct
> > > hmap *,
> > > +void build_parsed_routes(const struct ovn_datapath *,
> > > + const struct lr_stateful_table
> > > *lr_stateful_table,
> > > + const struct hmap *,
> > > const struct hmap *, struct hmap *,
> > > struct simap *,
> > > struct hmap *);
> > > uint32_t get_route_table_id(struct simap *, const char *);
> > > diff --git a/ovn-nb.xml b/ovn-nb.xml
> > > index 734fcf2bd..e8d9d5491 100644
> > > --- a/ovn-nb.xml
> > > +++ b/ovn-nb.xml
> > > @@ -2971,6 +2971,10 @@ or
> > > table="Logical_Router_Port"/></li>
> > > <li><ref column="options" key="dynamic-routing-static"
> > > table="Logical_Router_Port"/></li>
> > > + <li><ref column="options" key="dynamic-routing-lb-vips"
> > > + table="Logical_Router_Port"/></li>
> > > + <li><ref column="options" key="dynamic-routing-nat"
> > > + table="Logical_Router_Port"/></li>
> > > </ul>
> > > </column>
> > >
> > > @@ -3828,6 +3832,34 @@ or
> > > This option allows to have a 1:1 mapping between a single
> > > LRP and an
> > > individual link.
> > > </column>
> > > +
> > > + <column name="options" key="dynamic-routing-lb-vips"
> > > + type='{"type": "boolean"}'>
> > > + <p>
> > > + Only relevant if <ref column="options" key="dynamic-
> > > routing"
> > > + table="Logical_Router"/> on the respective
> > > Logical_Router is set
> > > + to <code>true</code>.
> > > +
> > > + If this option is <code>true</code>, northd will create
> > > host route
> > > + entries in the southbound <ref table="Advertised_Route"
> > > + db="OVN_Southbound"/> table, associated with this LRP,
> > > for each LB
> > > + VIP.
> > > + </p>
> > > + </column>
> > > +
> > > + <column name="options" key="dynamic-routing-nat"
> > > + type='{"type": "boolean"}'>
> > > + <p>
> > > + Only relevant if <ref column="options" key="dynamic-
> > > routing"
> > > + table="Logical_Router"/> on the respective
> > > Logical_Router is set
> > > + to <code>true</code>.
> > > +
> > > + If this option is <code>true</code>, northd will create
> > > host route
> > > + entries in the southbound <ref table="Advertised_Route"
> > > + db="OVN_Southbound"/> table, for external IP addresses
> > > of NAT rules
> > > + associated with this LRP.
> > > + </p>
> > > + </column>
> > > </group>
> > >
> > > <group title="Attachment">
> > > diff --git a/ovs b/ovs
> > > index a3c06c309..c648ee626 160000
> > > --- a/ovs
> > > +++ b/ovs
> > > @@ -1 +1 @@
> > > -Subproject commit a3c06c309ab9f21fd9ce4e8dca542119d46be9c5
> > > +Subproject commit c648ee626efdc959ed0cff37c2b9128816450f15
> > > diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> > > index 44dc92869..9baa0ea15 100644
> > > --- a/tests/ovn-northd.at
> > > +++ b/tests/ovn-northd.at
> > > @@ -15134,3 +15134,142 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE
> > > AT_CLEANUP
> > > ])
> > >
> > > +OVN_FOR_EACH_NORTHD_NO_HV([
> > > +AT_SETUP([dynamic-routing - nat sync to sb])
> > > +AT_KEYWORDS([dynamic-routing])
> > > +ovn_start
> > > +
> > > +# Start with GW router and a single LRP
> > > +check ovn-nbctl lr-add lr0
> > > +check ovn-nbctl \
> > > + -- \
> > > + set Logical_Router lr0 options:dynamic-routing=true \
> > > + options:chassis=hv1
> > > +check ovn-nbctl --wait=sb \
> > > + -- \
> > > + lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> > > +
> > > +check_row_count Advertised_Route 0
> > > +
> > > +datapath=$(ovn-sbctl --bare --columns _uuid list datapath_binding
> > > lr0)
> > > +pb=$(ovn-sbctl --bare --columns _uuid list port_binding lr0-sw0)
> > > +
> > > +# Adding LRP dynamic-routing-nat option and NAT rule advertises a
> > > route entry
> > > +check ovn-nbctl --wait=sb \
> > > + -- \
> > > + lrp-set-options lr0-sw0 dynamic-routing-nat=true \
> > > + -- \
> > > + lr-nat-add lr0 dnat_and_snat 172.16.1.10 192.168.1.10
> > > +
> > > +ovn-nbctl list NAT
> > > +ovn-sbctl list Advertised_Route
> > > +ovn-sbctl lflow-list
> > > +
> > > +check_row_count Advertised_Route 1
> > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find
> > > Advertised_Route \
> > > +datapath=$datapath logical_port=$pb], [0], [dnl
> > > +172.16.1.10/32
> > > +])
> > > +
> > > +# Add LR with distributed LRP connected to GW router through join
> > > LS
> > > +check ovn-nbctl \
> > > + -- \
> > > + lrp-add lr0 lr0-join 00:00:00:00:ff:02 10.42.0.1/24 \
> > > + -- \
> > > + ls-add ls-join \
> > > + -- \
> > > + lsp-add ls-join lsp-join-to-lr0 \
> > > + -- \
> > > + lsp-set-type lsp-join-to-lr0 router \
> > > + -- \
> > > + lsp-set-options lsp-join-to-lr0 router-port=lr0-join \
> > > + -- \
> > > + lsp-set-addresses lsp-join-to-lr0 router \
> > > + -- \
> > > + lr-add lr-guest0 \
> > > + -- \
> > > + lrp-add lr-guest0 lrp-guest0-sw0 00:00:00:00:fe:01
> > > 10.51.0.1/24 \
> > > + -- \
> > > + lrp-add lr-guest0 lrp-guest0-join 00:00:00:00:fe:02
> > > 10.42.0.2/24 \
> > > + -- \
> > > + lrp-set-options lrp-guest0-join dynamic-routing-nat=true \
> > > + -- \
> > > + lsp-add ls-join lsp-join-to-guest0 \
> > > + -- \
> > > + lsp-set-type lsp-join-to-guest0 router \
> > > + -- \
> > > + lsp-set-options lsp-join-to-guest0 router-port=lrp-guest0-join
> > > \
> > > + -- \
> > > + lrp-set-gateway-chassis lrp-guest0-join hv1
> > > +
> > > +pb2=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-
> > > guest0-join)
> > > +
> > > +check ovn-nbctl --wait=sb \
> > > + --add-route lr-nat-add lr-guest0 dnat_and_snat 172.16.2.10
> > > 192.168.2.10
> > > +
> > > +check_row_count Advertised_Route 2
> > > +ovn-sbctl list advertised_route
> > > +echo $datapath
> > > +echo $pb
> > > +echo $pb2
> > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find
> > > Advertised_Route \
> > > +datapath=$datapath logical_port="$pb"], [0], [dnl
> > > +172.16.1.10/32
> > > +])
> > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find
> > > Advertised_Route \
> > > +datapath=$datapath logical_port=$pb2], [0], [dnl
> > > +172.16.2.10/32
> > > +])
> > > +
> > > +# Add nonlocal LR with distributed LRP connected to GW router
> > > through join LS
> > > +check ovn-nbctl \
> > > + -- \
> > > + lr-add lr-guest1 \
> > > + -- \
> > > + lrp-add lr-guest1 lrp-guest1-sw0 00:00:00:00:fd:01
> > > 10.51.1.1/24 \
> > > + -- \
> > > + lrp-add lr-guest1 lrp-guest1-join 00:00:00:00:fd:02
> > > 10.42.0.3/24 \
> > > + -- \
> > > + lrp-set-options lrp-guest1-join dynamic-routing-nat=true \
> > > + -- \
> > > + lsp-add ls-join lsp-join-to-guest1 \
> > > + -- \
> > > + lsp-set-type lsp-join-to-guest1 router \
> > > + -- \
> > > + lsp-set-options lsp-join-to-guest1 router-port=lrp-guest1-join
> > > \
> > > + -- \
> > > + lrp-set-gateway-chassis lrp-guest1-join nonlocalhv
> > > +
> > > +pb3=$(ovn-sbctl --bare --columns _uuid list port_binding lrp-
> > > guest1-join)
> > > +
> > > +check ovn-nbctl --wait=sb \
> > > + --add-route lr-nat-add lr-guest1 dnat_and_snat 172.16.3.10
> > > 192.168.3.10
> > > +check_row_count Advertised_Route 3
> > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find
> > > Advertised_Route \
> > > +datapath=$datapath logical_port=$pb], [0], [dnl
> > > +172.16.1.10/32
> > > +])
> > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find
> > > Advertised_Route \
> > > +datapath=$datapath logical_port=$pb2], [0], [dnl
> > > +172.16.2.10/32
> > > +])
> > > +AT_CHECK([ovn-sbctl --columns ip_prefix --bare find
> > > Advertised_Route \
> > > +datapath=$datapath logical_port=$pb3], [0], [dnl
> > > +172.16.3.10/32
> > > +])
> > > +
> > > +# removing the option:dynamic-routing removes all routes
> > > +check ovn-nbctl --wait=sb remove Logical_Router lr0 option
> > > dynamic-routing
> > > +check_row_count Advertised_Route 0
> > > +
> > > +# and setting it again adds them again
> > > +check ovn-nbctl --wait=sb set Logical_Router lr0 option:dynamic-
> > > routing=true
> > > +check_row_count Advertised_Route 3
> > > +
> > > +# removing the lr will remove all routes
> > > +check ovn-nbctl --wait=sb lr-del lr0
> > > +check_row_count Advertised_Route 0
> > > +
> > > +AT_CLEANUP
> > > +])
> > > +
> > > diff --git a/tests/system-ovn.at b/tests/system-ovn.at
> > > index 5c4364663..918266ad2 100644
> > > --- a/tests/system-ovn.at
> > > +++ b/tests/system-ovn.at
> > > @@ -16033,3 +16033,382 @@ OVS_TRAFFIC_VSWITCHD_STOP(["/.*error
> > > receiving.*/d
> > > AT_CLEANUP
> > > ])
> > >
> > > +OVN_FOR_EACH_NORTHD([
> > > +AT_SETUP([route-exchange for LB VIPs with gateway router IPv4])
> > > +AT_KEYWORDS([route-exchange])
> > > +
> > > +CHECK_VRF()
> > > +CHECK_CONNTRACK()
> > > +CHECK_CONNTRACK_NAT()
> > > +ovn_start
> > > +OVS_TRAFFIC_VSWITCHD_START()
> > > +ADD_BR([br-int])
> > > +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone])
> > > +
> > > +# Set external-ids in br-int needed for ovn-controller
> > > +ovs-vsctl \
> > > + -- set Open_vSwitch . external-ids:system-id=hv1 \
> > > + -- set Open_vSwitch . external-ids:ovn-
> > > remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> > > + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> > > + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1
> > > \
> > > + -- set bridge br-int fail-mode=secure other-
> > > config:disable-in-band=true
> > > +
> > > +# Start ovn-controller
> > > +start_daemon ovn-controller
> > > +
> > > +ovn-appctl vlog/set route_exchange
> > > +check ovn-nbctl -- lr-add R1 \
> > > + -- set Logical_Router R1 options:requested-tnl-
> > > key=1000 options:dynamic-routing=true
> > > +
> > > +check ovn-nbctl ls-add sw0
> > > +check ovn-nbctl ls-add public
> > > +
> > > +check ovn-nbctl --wait=hv sync
> > > +
> > > +AT_CHECK([ip link | grep -q ovnvrf1000:.*UP], [1])
> > > +
> > > +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 192.168.1.1/24
> > > +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03
> > > 172.16.1.1/24 \
> > > + -- lrp-set-options rp-public \
> > > + dynamic-routing-maintain-vrf=true \
> > > + dynamic-routing-lb-vips=true
> > > +
> > > +check ovn-nbctl set logical_router R1 options:chassis=hv1
> > > +
> > > +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-
> > > rp \
> > > + type=router options:router-port=rp-sw0 \
> > > + -- lsp-set-addresses sw0-rp router
> > > +
> > > +check ovn-nbctl lsp-add public public-rp -- set
> > > Logical_Switch_Port public-rp \
> > > + type=router options:router-port=rp-public \
> > > + -- lsp-set-addresses public-rp router
> > > +
> > > +check ovs-vsctl set Open_vSwitch . external-ids:ovn-bridge-
> > > mappings=phynet:br-ext
> > > +
> > > +check ovn-nbctl lsp-add public public1 \
> > > + -- lsp-set-addresses public1 unknown \
> > > + -- lsp-set-type public1 localnet \
> > > + -- lsp-set-options public1 network_name=phynet
> > > +
> > > +check ovn-nbctl --wait=hv sync
> > > +
> > > +AT_CHECK([test `ip route show table 1000 | wc -l` -eq 1], [1])
> > > +
> > > +
> > > +# Create a load balancer and associate to R1
> > > +check ovn-nbctl lb-add lb1 172.16.1.150:80 172.16.1.100:80
> > > +check ovn-nbctl lr-lb-add R1 lb1
> > > +
> > > +check ovn-nbctl --wait=hv sync
> > > +
> > > +AT_CHECK([ip link | grep -q ovnvrf1000:.*UP])
> > > +AT_CHECK([test `ip route show table 1000 | wc -l` -eq 1])
> > > +AT_CHECK([ip route show table 1000 | grep -q 172.16.1.150])
> > > +
> > > +OVS_APP_EXIT_AND_WAIT([ovn-controller])
> > > +
> > > +# Ensure system resources are cleaned up
> > > +AT_CHECK([ip link | grep -q ovnvrf1000:.*UP], [1])
> > > +AT_CHECK([test `ip route show table 1000 | wc -l` -eq 1], [1])
> > > +
> > > +as ovn-sb
> > > +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> > > +
> > > +as ovn-nb
> > > +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> > > +
> > > +as northd
> > > +OVS_APP_EXIT_AND_WAIT([ovn-northd])
> > > +
> > > +as
> > > +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
> > > +/Failed to acquire.*/d
> > > +/connection dropped.*/d"])
> > > +AT_CLEANUP
> > > +])
> > > +
> > > +OVN_FOR_EACH_NORTHD([
> > > +AT_SETUP([route-exchange for LB VIPs with gateway router IPv6])
> > > +AT_KEYWORDS([route-exchange])
> > > +
> > > +CHECK_VRF()
> > > +CHECK_CONNTRACK()
> > > +CHECK_CONNTRACK_NAT()
> > > +ovn_start
> > > +OVS_TRAFFIC_VSWITCHD_START()
> > > +ADD_BR([br-int])
> > > +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone])
> > > +
> > > +# Set external-ids in br-int needed for ovn-controller
> > > +ovs-vsctl \
> > > + -- set Open_vSwitch . external-ids:system-id=hv1 \
> > > + -- set Open_vSwitch . external-ids:ovn-
> > > remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> > > + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> > > + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1
> > > \
> > > + -- set bridge br-int fail-mode=secure other-
> > > config:disable-in-band=true
> > > +
> > > +# Start ovn-controller
> > > +start_daemon ovn-controller
> > > +
> > > +ovn-appctl vlog/set route_exchange
> > > +check ovn-nbctl -- lr-add R1 \
> > > + -- set Logical_Router R1 options:requested-tnl-
> > > key=1001 options:dynamic-routing=true
> > > +
> > > +check ovn-nbctl ls-add sw0
> > > +check ovn-nbctl ls-add public
> > > +
> > > +check ovn-nbctl --wait=hv sync
> > > +
> > > +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP], [1])
> > > +
> > > +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03
> > > 2001:db8:100::1/64
> > > +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03
> > > 2001:db8:1001::1/64 \
> > > + -- lrp-set-options rp-public \
> > > + dynamic-routing-maintain-vrf=true \
> > > + dynamic-routing-lb-vips=true
> > > +
> > > +check ovn-nbctl set logical_router R1 options:chassis=hv1
> > > +
> > > +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-
> > > rp \
> > > + type=router options:router-port=rp-sw0 \
> > > + -- lsp-set-addresses sw0-rp router
> > > +
> > > +check ovn-nbctl lsp-add public public-rp -- set
> > > Logical_Switch_Port public-rp \
> > > + type=router options:router-port=rp-public \
> > > + -- lsp-set-addresses public-rp router
> > > +
> > > +check ovs-vsctl set Open_vSwitch . external-ids:ovn-bridge-
> > > mappings=phynet:br-ext
> > > +
> > > +check ovn-nbctl lsp-add public public1 \
> > > + -- lsp-set-addresses public1 unknown \
> > > + -- lsp-set-type public1 localnet \
> > > + -- lsp-set-options public1 network_name=phynet
> > > +
> > > +check ovn-nbctl --wait=hv sync
> > > +
> > > +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1], [1])
> > > +
> > > +# Create a load balancer and associate to R1
> > > +check ovn-nbctl lb-add lb1 [[2001:db8:1001::150]]:80
> > > [[2001:db8:1001::100]]:80
> > > +check ovn-nbctl lr-lb-add R1 lb1
> > > +
> > > +check ovn-nbctl --wait=hv sync
> > > +
> > > +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP])
> > > +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1])
> > > +AT_CHECK([ip -6 route show table 1001 | grep -q
> > > 2001:db8:1001::150])
> > > +
> > > +
> > > +OVS_APP_EXIT_AND_WAIT([ovn-controller])
> > > +
> > > +# Ensure system resources are cleaned up
> > > +AT_CHECK([ip link | grep -q ovnvrf1001:.*UP], [1])
> > > +AT_CHECK([test `ip -6 route show table 1001 | wc -l` -eq 1], [1])
> > > +
> > > +as ovn-sb
> > > +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> > > +
> > > +as ovn-nb
> > > +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> > > +
> > > +as northd
> > > +OVS_APP_EXIT_AND_WAIT([ovn-northd])
> > > +
> > > +as
> > > +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
> > > +/Failed to acquire.*/d
> > > +/connection dropped.*/d"])
> > > +AT_CLEANUP
> > > +])
> > > +
> > > +OVN_FOR_EACH_NORTHD([
> > > +AT_SETUP([route-exchange for DNAT and DNAT_AND_SNAT with gateway
> > > router IPv4])
> > > +AT_KEYWORDS([route-exchange])
> > > +
> > > +CHECK_VRF()
> > > +CHECK_CONNTRACK()
> > > +CHECK_CONNTRACK_NAT()
> > > +ovn_start
> > > +OVS_TRAFFIC_VSWITCHD_START()
> > > +ADD_BR([br-int])
> > > +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone])
> > > +
> > > +# Set external-ids in br-int needed for ovn-controller
> > > +ovs-vsctl \
> > > + -- set Open_vSwitch . external-ids:system-id=hv1 \
> > > + -- set Open_vSwitch . external-ids:ovn-
> > > remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> > > + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> > > + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1
> > > \
> > > + -- set bridge br-int fail-mode=secure other-
> > > config:disable-in-band=true
> > > +
> > > +# Start ovn-controller
> > > +start_daemon ovn-controller
> > > +
> > > +ovn-appctl vlog/set route_exchange
> > > +check ovn-nbctl -- lr-add R1 \
> > > + -- set Logical_Router R1 options:requested-tnl-
> > > key=1002 options:dynamic-routing=true
> > > +
> > > +check ovn-nbctl ls-add sw0
> > > +check ovn-nbctl ls-add public
> > > +
> > > +check ovn-nbctl --wait=hv sync
> > > +
> > > +AT_CHECK([ip link | grep -q ovnvrf1002:.*UP], [1])
> > > +
> > > +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 192.168.1.1/24
> > > +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03
> > > 172.16.1.1/24 \
> > > + -- lrp-set-options rp-public \
> > > + dynamic-routing-maintain-vrf=true \
> > > + dynamic-routing-nat=true
> > > +
> > > +check ovn-nbctl set logical_router R1 options:chassis=hv1
> > > +
> > > +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-
> > > rp \
> > > + type=router options:router-port=rp-sw0 \
> > > + -- lsp-set-addresses sw0-rp router
> > > +
> > > +check ovn-nbctl lsp-add public public-rp -- set
> > > Logical_Switch_Port public-rp \
> > > + type=router options:router-port=rp-public \
> > > + -- lsp-set-addresses public-rp router
> > > +
> > > +check ovs-vsctl set Open_vSwitch . external-ids:ovn-bridge-
> > > mappings=phynet:br-ext
> > > +
> > > +check ovn-nbctl lsp-add public public1 \
> > > + -- lsp-set-addresses public1 unknown \
> > > + -- lsp-set-type public1 localnet \
> > > + -- lsp-set-options public1 network_name=phynet
> > > +
> > > +check ovn-nbctl --wait=hv sync
> > > +
> > > +AT_CHECK([test `ip route show table 1002 | wc -l` -eq 2], [1])
> > > +
> > > +# Create dnat_and_snat, dnat rules in R1
> > > +check ovn-nbctl lr-nat-add R1 dnat_and_snat 172.16.1.10
> > > 192.168.1.10
> > > +check ovn-nbctl lr-nat-add R1 dnat 172.16.1.11 192.168.1.11
> > > +
> > > +check ovn-nbctl --wait=hv sync
> > > +
> > > +AT_CHECK([ip link | grep -q ovnvrf1002:.*UP])
> > > +AT_CHECK([test `ip route show table 1002 | wc -l` -eq 2])
> > > +AT_CHECK([ip route show table 1002 | grep -q 172.16.1.10])
> > > +AT_CHECK([ip route show table 1002 | grep -q 172.16.1.11])
> > > +
> > > +
> > > +OVS_APP_EXIT_AND_WAIT([ovn-controller])
> > > +
> > > +# Ensure system resources are cleaned up
> > > +AT_CHECK([ip link | grep -q ovnvrf1002:.*UP], [1])
> > > +AT_CHECK([test `ip route show table 1002 | wc -l` -eq 1], [1])
> > > +
> > > +as ovn-sb
> > > +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> > > +
> > > +as ovn-nb
> > > +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> > > +
> > > +as northd
> > > +OVS_APP_EXIT_AND_WAIT([ovn-northd])
> > > +
> > > +as
> > > +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
> > > +/Failed to acquire.*/d
> > > +/connection dropped.*/d"])
> > > +AT_CLEANUP
> > > +])
> > > +
> > > +OVN_FOR_EACH_NORTHD([
> > > +AT_SETUP([route-exchange for DNAT and DNAT_AND_SNAT with gateway
> > > router IPv6])
> > > +AT_KEYWORDS([route-exchange])
> > > +
> > > +CHECK_VRF()
> > > +CHECK_CONNTRACK()
> > > +CHECK_CONNTRACK_NAT()
> > > +ovn_start
> > > +OVS_TRAFFIC_VSWITCHD_START()
> > > +ADD_BR([br-int])
> > > +ADD_BR([br-ext], [set Bridge br-ext fail-mode=standalone])
> > > +
> > > +# Set external-ids in br-int needed for ovn-controller
> > > +ovs-vsctl \
> > > + -- set Open_vSwitch . external-ids:system-id=hv1 \
> > > + -- set Open_vSwitch . external-ids:ovn-
> > > remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
> > > + -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
> > > + -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1
> > > \
> > > + -- set bridge br-int fail-mode=secure other-
> > > config:disable-in-band=true
> > > +
> > > +# Start ovn-controller
> > > +start_daemon ovn-controller
> > > +
> > > +ovn-appctl vlog/set route_exchange
> > > +check ovn-nbctl -- lr-add R1 \
> > > + -- set Logical_Router R1 options:requested-tnl-
> > > key=1003 options:dynamic-routing=true
> > > +
> > > +check ovn-nbctl ls-add sw0
> > > +check ovn-nbctl ls-add public
> > > +
> > > +check ovn-nbctl --wait=hv sync
> > > +
> > > +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP], [1])
> > > +
> > > +check ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03
> > > 2001:db8:100::1/64
> > > +check ovn-nbctl -- lrp-add R1 rp-public 00:00:02:01:02:03
> > > 2001:db8:1003::1/64 \
> > > + -- lrp-set-options rp-public \
> > > + dynamic-routing-maintain-vrf=true \
> > > + dynamic-routing-nat=true
> > > +
> > > +check ovn-nbctl set logical_router R1 options:chassis=hv1
> > > +
> > > +check ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-
> > > rp \
> > > + type=router options:router-port=rp-sw0 \
> > > + -- lsp-set-addresses sw0-rp router
> > > +
> > > +check ovn-nbctl lsp-add public public-rp -- set
> > > Logical_Switch_Port public-rp \
> > > + type=router options:router-port=rp-public \
> > > + -- lsp-set-addresses public-rp router
> > > +
> > > +check ovs-vsctl set Open_vSwitch . external-ids:ovn-bridge-
> > > mappings=phynet:br-ext
> > > +
> > > +check ovn-nbctl lsp-add public public1 \
> > > + -- lsp-set-addresses public1 unknown \
> > > + -- lsp-set-type public1 localnet \
> > > + -- lsp-set-options public1 network_name=phynet
> > > +
> > > +check ovn-nbctl --wait=hv sync
> > > +
> > > +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2], [1])
> > > +
> > > +# Create dnat_and_snat, dnat rules in R1
> > > +check ovn-nbctl lr-nat-add R1 \
> > > + dnat_and_snat 2001:db8:1003::150 2001:db8:100::100
> > > +check ovn-nbctl lr-nat-add R1 \
> > > + dnat 2001:db8:1003::151 2001:db8:100::100
> > > +
> > > +check ovn-nbctl --wait=hv sync
> > > +
> > > +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP])
> > > +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2])
> > > +AT_CHECK([ip -6 route show table 1003 | grep -q
> > > 2001:db8:1003::150])
> > > +AT_CHECK([ip -6 route show table 1003 | grep -q
> > > 2001:db8:1003::151])
> > > +
> > > +OVS_APP_EXIT_AND_WAIT([ovn-controller])
> > > +
> > > +# Ensure system resources are cleaned up
> > > +AT_CHECK([ip link | grep -q ovnvrf1003:.*UP], [1])
> > > +AT_CHECK([test `ip -6 route show table 1003 | wc -l` -eq 2], [1])
> > > +
> > > +as ovn-sb
> > > +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> > > +
> > > +as ovn-nb
> > > +OVS_APP_EXIT_AND_WAIT([ovsdb-server])
> > > +
> > > +as northd
> > > +OVS_APP_EXIT_AND_WAIT([ovn-northd])
> > > +
> > > +as
> > > +OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
> > > +/Failed to acquire.*/d
> > > +/connection dropped.*/d"])
> > > +AT_CLEANUP
> > > +])
> > > +
> > > --
> > > 2.43.0
> > >
> > > _______________________________________________
> > > dev mailing list
> > > [email protected]
> > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
> _______________________________________________
> dev mailing list
> [email protected]
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
