From: Guillaume Boutry <[email protected]>
When a router redistributes a neighboring router's distributed
NAT address on a shared external switch, forwarding through the
distributed gateway IP follows gateway residency rather than the
backend tracked by the advertisement.
Use the NAT external IP as next hop only when the shared adjacency is
that NAT's distributed gateway. Carry the NB-known external MAC into
the forwarding route so a sibling ARP proxy or an incorrect dynamic
binding cannot override the selected backend's Ethernet destination.
Reconcile MAC-only changes, deduplicate equal MAC contributors
and drop ambiguous next hops whose authoritative MACs disagree.
Preserve peer forwarding for separate transit ports, centralized
NAT and load balancers.
Cover IPv4 and IPv6 flow construction, ECMP, mode and address changes,
withdrawal and incremental/full recomputation. Add kernel packet
coverage for remote gateway residency, cold traffic, a retained
incorrect binding and MAC/FIP lifecycle changes.
The forwarding route was introduced by commit cfb2b9b3ca8a ("northd:
Install forwarding routes for redistribute={lb, nat}.").
Assisted-By: Codex (gpt-6)
Signed-off-by: Guillaume Boutry <[email protected]>
---
A router redistributing a neighbor's distributed NAT address through a
shared external switch can forward to the gateway MAC instead of the NAT
backend. The advertisement tracks the backend LSP, but the forwarding
route uses the distributed gateway's IP. If the backend and gateway are
on different chassis, traffic can miss the distributed NAT path.
Reproducer
----------
Connect the advertising LR and the NAT router's DGP through a localnet
switch. Enable dynamic-routing and dynamic-routing-redistribute=nat on
the advertising side. Configure dnat_and_snat with logical_port and
external_mac, with the backend local and the DGP on a remote chassis.
In the added flow fixture, the old route for 10.0.0.10 loads
reg0=10.0.0.1, the gateway IP. It should use the FIP and its configured NAT
external MAC. Changing only the next-hop IP is insufficient: a sibling ARP
proxy can supply its own MAC for that FIP and send traffic back to the
wrong router. The patch corrects both choices together.
The packet fixture keeps an incorrect dynamic MAC_Binding, checks its
UUID/MAC remain unchanged, and requires client-to-FIP traffic to succeed:
tests/system-ovn.at:
dynamic-routing - distributed NAT next hop on shared localnet
Run in a configured native build:
make check TESTSUITEFLAGS='-d -j2 -k dynamic-routing'
make check-kernel TESTSUITEFLAGS='-d -k distributed-nat-nexthop'
The supported configuration is documented under NAT.external_mac,
NAT.logical_port and Logical_Router.options:dynamic-routing-redistribute:
https://www.ovn.org/support/dist-docs/ovn-nb.5.html
The forwarding-route commit also explains the intended backend reachability:
https://github.com/ovn-org/ovn/commit/cfb2b9b3ca8ad1cc623bd89bb761b6835d8cdbb9
Validation
----------
OVN 26.09.90, upstream base:
1c8d7419a73743f79ff54102c6b66f6c8061b756
The standalone NAT tree passed a Werror build, all 39 selected native
routing tests with zero skips, and both kernel packet variants. Coverage
includes MAC-only changes, equal/conflicting contributors, ECMP, NAT mode
changes, withdrawal, recomputation and MAC/FIP lifecycle. Separate transit,
centralized NAT and load-balancer forwarding remain unchanged.
IPv6 coverage checks logical flows; packet coverage is IPv4. The full
suite, distcheck and older-kernel tests were not run, and no affected
release range has been established.
northd/en-advertised-route-sync.c | 36 ++++-
northd/en-learned-route-sync.c | 5 +-
northd/northd.c | 102 ++++++++++++-
northd/northd.h | 2 +
tests/ovn-northd.at | 228 ++++++++++++++++++++++++++++++
tests/system-ovn.at | 194 +++++++++++++++++++++++++
6 files changed, 555 insertions(+), 12 deletions(-)
diff --git a/northd/en-advertised-route-sync.c
b/northd/en-advertised-route-sync.c
index eb8a24a39..d6d964f8f 100644
--- a/northd/en-advertised-route-sync.c
+++ b/northd/en-advertised-route-sync.c
@@ -248,8 +248,10 @@ dynamic_routes_track_od(struct dynamic_routes_data *data,
/* Install a parsed_route on advertising_od that forwards ip_address (a
* LB VIP or NAT external IP) through advertising_op toward tracked_port,
- * where tracked_port must be a peer LRP on the shared LS so that its
- * first matching-family network address is a valid nexthop.
+ * where tracked_port must be a peer LRP with a matching-family address.
+ * Use its first address as nexthop unless nexthop_override is provided.
+ * When nexthop_mac is set, forwarding uses that authoritative MAC instead of
+ * resolving it from learned neighbor information.
*
* Used by the connected-neighbour redistribution paths
* (build_{lb,nat}_connected_routes) so the advertising LR can
@@ -272,6 +274,8 @@ add_redistribute_parsed_route(struct hmap
*parsed_routes_out,
const struct ovn_port *advertising_op,
const struct ovn_port *tracked_port,
const char *ip_address,
+ const char *nexthop_override,
+ const struct eth_addr *nexthop_mac,
enum route_source source,
const struct ovsdb_idl_row *source_hint)
{
@@ -313,6 +317,10 @@ add_redistribute_parsed_route(struct hmap
*parsed_routes_out,
return;
}
+ if (nexthop_override) {
+ nexthop_s = nexthop_override;
+ }
+
/* If advertising_op has an address in the nexthop's family, use it as
* eth.src. Otherwise (unnumbered LRP) leave lrp_addr_s NULL so the
* emitted route omits REG_SRC_IPV{4,6}. ARP resolution still works:
@@ -327,7 +335,7 @@ add_redistribute_parsed_route(struct hmap
*parsed_routes_out,
size_t n_routes = hmap_count(parsed_routes_out);
struct parsed_route *new_pr = parsed_route_add(
- advertising_od, nexthop, &prefix, plen, false, lrp_addr_s,
+ advertising_od, nexthop, nexthop_mac, &prefix, plen, false, lrp_addr_s,
advertising_op, 0, false, false, false, NULL, source, false,
source_hint, tracked_port, parsed_routes_out);
@@ -408,9 +416,28 @@ build_nat_route_for_port(const struct ovn_port
*advertising_op,
}
if (forwarding_port && parsed_routes_out) {
+ const char *nexthop_override = NULL;
+ const struct eth_addr *nexthop_mac = NULL;
+ if (nat->is_distributed &&
+ forwarding_port == nat->l3dgw_port &&
+ advertising_op->peer && advertising_op->peer->nbsp) {
+ /* On the shared external LS, the FIP's external MAC
+ * delivers to the backend's chassis. Resolving the DGP's
+ * IP instead selects its gateway MAC and gateway chassis.
+ * Use the known external MAC so a proxy ARP reply cannot
+ * redirect the FIP back to the advertising router. A separate
+ * transit peer still needs its own LRP IP. */
+ nexthop_override = nat->nb->external_ip;
+ if (!eth_addr_is_zero(nat->mac) &&
+ !eth_addr_is_multicast(nat->mac)) {
+ nexthop_mac = &nat->mac;
+ }
+ }
add_redistribute_parsed_route(parsed_routes_out, advertising_od,
advertising_op, forwarding_port,
nat->nb->external_ip,
+ nexthop_override,
+ nexthop_mac,
ROUTE_SOURCE_NAT,
&nat->nb->header_);
}
@@ -557,7 +584,8 @@ build_lb_routes_for_lb(const struct ovn_port
*advertising_op,
if (parsed_routes_out) {
add_redistribute_parsed_route(
parsed_routes_out, advertising_od, advertising_op,
- fallback_tracked_port, vip->vip_str, ROUTE_SOURCE_LB,
+ fallback_tracked_port, vip->vip_str, NULL, NULL,
+ ROUTE_SOURCE_LB,
&nbrec_lb->header_);
}
diff --git a/northd/en-learned-route-sync.c b/northd/en-learned-route-sync.c
index cbd516b68..f7a4075cc 100644
--- a/northd/en-learned-route-sync.c
+++ b/northd/en-learned-route-sync.c
@@ -199,8 +199,9 @@ parse_route_from_sbrec_route(struct hmap *parsed_routes_out,
return NULL;
}
- return parsed_route_add(od, nexthop, &prefix, plen, false, lrp_addr_s,
- out_port, 0, false, false, false, NULL,
+ return parsed_route_add(od, nexthop, NULL, &prefix, plen, false,
+ lrp_addr_s, out_port, 0, false, false, false,
+ NULL,
ROUTE_SOURCE_LEARNED, true, &route->header_, NULL,
parsed_routes_out);
}
diff --git a/northd/northd.c b/northd/northd.c
index 3f59894bd..2729f2304 100644
--- a/northd/northd.c
+++ b/northd/northd.c
@@ -12741,6 +12741,10 @@ parsed_route_lookup(struct hmap *routes, size_t hash,
continue;
}
+ if (!eth_addr_equals(pr->nexthop_mac, new_pr->nexthop_mac)) {
+ continue;
+ }
+
if (!ipv6_addr_equals(&pr->prefix, &new_pr->prefix)) {
continue;
}
@@ -12805,6 +12809,7 @@ parsed_route_lookup(struct hmap *routes, size_t hash,
static struct parsed_route *
parsed_route_init(const struct ovn_datapath *od,
struct in6_addr *nexthop,
+ const struct eth_addr *nexthop_mac,
const struct in6_addr prefix,
unsigned int plen,
bool is_discard_route,
@@ -12826,6 +12831,9 @@ parsed_route_init(const struct ovn_datapath *od,
new_pr->prefix = prefix;
new_pr->plen = plen;
new_pr->nexthop = nexthop;
+ if (nexthop_mac) {
+ new_pr->nexthop_mac = *nexthop_mac;
+ }
new_pr->route_table_id = route_table_id;
new_pr->is_src_route = is_src_route;
new_pr->od = od;
@@ -12858,7 +12866,8 @@ parsed_route_clone(const struct parsed_route *pr)
}
struct parsed_route *new_pr = parsed_route_init(
- pr->od, nexthop, pr->prefix, pr->plen, pr->is_discard_route,
+ pr->od, nexthop, &pr->nexthop_mac, pr->prefix, pr->plen,
+ pr->is_discard_route,
pr->lrp_addr_s, pr->out_port, pr->route_table_id, pr->is_src_route,
pr->ecmp_symmetric_reply, pr->override_connected,
&pr->ecmp_selection_fields, pr->source, pr->dynamic_routing_advertise,
@@ -12913,6 +12922,7 @@ parsed_route_free(struct parsed_route *pr) {
struct parsed_route *
parsed_route_add(const struct ovn_datapath *od,
struct in6_addr *nexthop,
+ const struct eth_addr *nexthop_mac,
const struct in6_addr *prefix,
unsigned int plen,
bool is_discard_route,
@@ -12931,7 +12941,8 @@ parsed_route_add(const struct ovn_datapath *od,
{
struct parsed_route *new_pr
- = parsed_route_init(od, nexthop, *prefix, plen, is_discard_route,
+ = parsed_route_init(od, nexthop, nexthop_mac, *prefix, plen,
+ is_discard_route,
lrp_addr_s, out_port, route_table_id,
is_src_route, ecmp_symmetric_reply,
override_connected, ecmp_selection_fields,
@@ -13079,8 +13090,9 @@ parsed_routes_add_static(const struct ovn_datapath *od,
bool dynamic_routing_advertise = smap_get_bool(&route->options,
"dynamic-routing-advertise",
true);
- struct parsed_route *pr = parsed_route_add(od, nexthop, &prefix, plen,
- is_discard_route, lrp_addr_s,
+ struct parsed_route *pr = parsed_route_add(od, nexthop, NULL, &prefix,
+ plen, is_discard_route,
+ lrp_addr_s,
out_port, route_table_id,
is_src_route,
ecmp_symmetric_reply,
@@ -13102,7 +13114,7 @@ parsed_routes_add_connected(const struct ovn_datapath
*od,
struct in6_addr prefix;
in6_addr_set_mapped_ipv4(&prefix, addr->network);
- parsed_route_add(od, NULL, &prefix, addr->plen,
+ parsed_route_add(od, NULL, NULL, &prefix, addr->plen,
false, addr->addr_s, op, 0, false, false,
false, NULL, ROUTE_SOURCE_CONNECTED,
true, &op->nbrp->header_, NULL, routes);
@@ -13111,7 +13123,7 @@ parsed_routes_add_connected(const struct ovn_datapath
*od,
for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) {
const struct ipv6_netaddr *addr = &op->lrp_networks.ipv6_addrs[i];
- parsed_route_add(od, NULL, &addr->network, addr->plen, false,
+ parsed_route_add(od, NULL, NULL, &addr->network, addr->plen, false,
addr->addr_s, op, 0, false, false, false,
NULL, ROUTE_SOURCE_CONNECTED, true,
&op->nbrp->header_, NULL, routes);
@@ -15792,14 +15804,88 @@ build_default_route_flows_for_lrouter(
}
+struct route_nexthop_mac {
+ const struct parsed_route *route;
+ bool conflict;
+};
+
+/* Collect authoritative MACs before emitting flows so duplicate contributors
+ * cannot install competing actions for the same output port and next hop. */
+static void
+route_nexthop_mac_add(const struct parsed_route *route, struct shash *nexthops)
+{
+ if (route->is_discard_route || !route->out_port || !route->nexthop ||
+ eth_addr_is_zero(route->nexthop_mac)) {
+ return;
+ }
+
+ struct ds key = DS_EMPTY_INITIALIZER;
+ ds_put_format(&key, "%s,", route->out_port->key);
+ ipv6_format_mapped(route->nexthop, &key);
+ struct route_nexthop_mac *nh = shash_find_data(nexthops, ds_cstr(&key));
+ if (nh) {
+ if (!eth_addr_equals(nh->route->nexthop_mac, route->nexthop_mac)) {
+ nh->conflict = true;
+ }
+ } else {
+ nh = xmalloc(sizeof *nh);
+ nh->route = route;
+ nh->conflict = false;
+ shash_add(nexthops, ds_cstr(&key), nh);
+ }
+ ds_destroy(&key);
+}
+
+static void
+build_route_nexthop_mac_flows(const struct shash *nexthops,
+ struct lflow_table *lflows,
+ struct lflow_ref *lflow_ref)
+{
+ struct shash_node *node;
+ SHASH_FOR_EACH (node, nexthops) {
+ const struct route_nexthop_mac *nh = node->data;
+ const struct parsed_route *route = nh->route;
+ struct ds match = DS_EMPTY_INITIALIZER;
+ struct ds actions = DS_EMPTY_INITIALIZER;
+ bool is_ipv4 = IN6_IS_ADDR_V4MAPPED(route->nexthop);
+ ds_put_format(&match, REGBIT_NEXTHOP_IS_IPV4" == %d && "
+ "outport == %s && %s == ", is_ipv4,
+ route->out_port->json_key,
+ is_ipv4 ? REG_NEXT_HOP_IPV4 : REG_NEXT_HOP_IPV6);
+ ipv6_format_mapped(route->nexthop, &match);
+ if (nh->conflict) {
+ /* A neighbor lookup cannot choose between different MACs for
+ * this output port and IP. Drop instead of emitting conflicting
+ * flows or selecting one contributor arbitrarily. */
+ static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
+ VLOG_WARN_RL(&rl, "Conflicting route next-hop MACs for %s",
+ node->name);
+ ds_put_cstr(&actions, debug_drop_action());
+ } else {
+ ds_put_format(&actions, "eth.dst = "ETH_ADDR_FMT"; next;",
+ ETH_ADDR_ARGS(route->nexthop_mac));
+ }
+ ovn_lflow_add(lflows, route->od, S_ROUTER_IN_ARP_RESOLVE, 100,
+ ds_cstr(&match), ds_cstr(&actions), lflow_ref,
+ WITH_HINT(route->source_hint));
+ ds_destroy(&match);
+ ds_destroy(&actions);
+ }
+}
+
void
build_route_data_flows_for_lrouter(
const struct ovn_datapath *od, struct lflow_table *lflows,
const struct group_ecmp_datapath *route_node,
const struct sset *bfd_ports)
{
+ struct shash known_nexthops = SHASH_INITIALIZER(&known_nexthops);
struct ecmp_groups_node *group;
HMAP_FOR_EACH (group, hmap_node, &route_node->ecmp_groups) {
+ const struct ecmp_route_list_node *er;
+ VECTOR_FOR_EACH_PTR (&group->route_list, er) {
+ route_nexthop_mac_add(er->route, &known_nexthops);
+ }
/* add a flow in IP_ROUTING, and one flow for each member in
* IP_ROUTING_ECMP. */
build_ecmp_route_flow(lflows, od, group, route_node->lflow_ref, NULL);
@@ -15818,9 +15904,13 @@ build_route_data_flows_for_lrouter(
}
const struct unique_routes_node *ur;
HMAP_FOR_EACH (ur, hmap_node, &route_node->unique_routes) {
+ route_nexthop_mac_add(ur->route, &known_nexthops);
build_route_flow(lflows, od, ur->route, bfd_ports,
route_node->lflow_ref);
}
+ build_route_nexthop_mac_flows(&known_nexthops, lflows,
+ route_node->lflow_ref);
+ shash_destroy_free_data(&known_nexthops);
}
static void
diff --git a/northd/northd.h b/northd/northd.h
index 3d1fd2d31..4d655f2f9 100644
--- a/northd/northd.h
+++ b/northd/northd.h
@@ -860,6 +860,7 @@ struct parsed_route {
struct hmap_node key_node;
struct in6_addr prefix;
struct in6_addr *nexthop; /* NULL for ROUTE_SOURCE_CONNECTED */
+ struct eth_addr nexthop_mac; /* Zero when neighbor resolution is needed. */
unsigned int plen;
uint32_t route_table_id;
uint32_t hash;
@@ -889,6 +890,7 @@ void parsed_route_free(struct parsed_route *);
struct parsed_route *parsed_route_add(
const struct ovn_datapath *od,
struct in6_addr *nexthop,
+ const struct eth_addr *nexthop_mac,
const struct in6_addr *prefix,
unsigned int plen,
bool is_discard_route,
diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
index 93af177ab..4eb5fc76c 100644
--- a/tests/ovn-northd.at
+++ b/tests/ovn-northd.at
@@ -19395,6 +19395,234 @@ OVN_CLEANUP_NORTHD
AT_CLEANUP
])
+OVN_FOR_EACH_NORTHD_NO_HV([
+AT_SETUP([dynamic-routing - NAT redistribute through a shared distributed
gateway])
+AT_KEYWORDS([dynamic-routing])
+ovn_start
+
+# The advertising router and the NAT's DGP share a localnet-backed LS.
+# Forwarding to the DGP's IP selects its gateway MAC, whose residency can
+# differ from that of the distributed NAT's backend. Resolve the FIP
+# instead, so the packet uses the NAT's external MAC on its owning chassis.
+check ovn-nbctl lr-add lr0
+check ovn-nbctl set Logical_Router lr0 options:dynamic-routing=true
+check ovn-nbctl lrp-add lr0 lr0-up 00:00:00:00:00:01
+check ovn-nbctl lrp-set-options lr0-up dynamic-routing-redistribute=nat
+check ovn-nbctl ls-add provider
+check ovn-nbctl lsp-add-router-port provider prov-lr0 lr0-up
+check ovn-nbctl lsp-add provider ln-prov
+check ovn-nbctl lsp-set-type ln-prov localnet
+check ovn-nbctl lsp-set-options ln-prov network_name=physnet
+
+check ovn-nbctl lr-add lr1
+check ovn-nbctl lrp-add lr1 lr1-up 00:00:00:00:00:02 10.0.0.1/24
+check ovn-nbctl lrp-set-gateway-chassis lr1-up hv2
+check ovn-nbctl lrp-set-options lr1-up reside-on-redirect-chassis=true
+check ovn-nbctl lsp-add-router-port provider prov-lr1 lr1-up
+check ovn-nbctl lrp-add lr1 lr1-be 00:00:00:00:00:03 192.168.1.1/24
+check ovn-nbctl ls-add be
+check ovn-nbctl lsp-add-router-port be be-lr1 lr1-be
+check ovn-nbctl lsp-add be be-vm1
+check ovn-nbctl lsp-set-addresses be-vm1 "00:00:00:00:01:01 192.168.1.10"
+check ovn-nbctl lr-nat-add lr1 dnat_and_snat 10.0.0.10 192.168.1.10 be-vm1
00:00:00:00:01:02
+check ovn-nbctl --wait=sb sync
+
+datapath_lr0=$(fetch_column Datapath_Binding _uuid external_ids:name=lr0)
+pb_lr0_up=$(fetch_column Port_Binding _uuid logical_port=lr0-up)
+pb_be_vm1=$(fetch_column Port_Binding _uuid logical_port=be-vm1)
+
+check_fip_mac() {
+ ovn-sbctl lflow-list lr0 > lr0_flows || return 2
+ grep 'lr_in_arp_resolve.*outport == "lr0-up" && reg0 ==
10.0.0.1@<:@01@:>@' lr0_flows > lr0_mac_flows || return $?
+ ovn_strip_lflows < lr0_mac_flows
+}
+
+check_fip_route() {
+ ovn-sbctl lflow-list lr0 > lr0_flows
+ grep 'lr_in_ip_routing.*10.0.0.1@<:@01@:>@/32' lr0_flows | ovn_strip_lflows
+}
+
+# The advertising LRP has no IPv4 address. The selected next hop is the
+# FIP, not the gateway's 10.0.0.1 address.
+check_row_count Advertised_Route 1 ip_prefix=10.0.0.10 \
+ datapath=$datapath_lr0 logical_port=$pb_lr0_up tracked_port=$pb_be_vm1
+AT_CHECK([check_fip_route], [0], [dnl
+ table=??(lr_in_ip_routing ), priority=1935 , match=(ip4.dst ==
10.0.0.10/32), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 10.0.0.10; eth.src =
00:00:00:00:00:01; outport = "lr0-up"; flags.loopback = 1; reg9[[9]] = 1; next;)
+])
+
+# Forwarding uses the NB-known NAT MAC independently of learned neighbors.
+AT_CHECK([check_fip_mac], [0], [dnl
+ table=??(lr_in_arp_resolve ), priority=100 , match=(reg9[[9]] == 1 &&
outport == "lr0-up" && reg0 == 10.0.0.10), action=(eth.dst = 00:00:00:00:01:02;
next;)
+])
+
+# A MAC-only update must replace the owned resolve flow incrementally.
+nat_uuid=$(ovn-nbctl --bare --columns=_uuid find NAT external_ip=10.0.0.10)
+check ovn-nbctl --wait=sb set NAT $nat_uuid
external_mac="\"00:00:00:00:01:03\""
+AT_CHECK([check_fip_mac], [0], [dnl
+ table=??(lr_in_arp_resolve ), priority=100 , match=(reg9[[9]] == 1 &&
outport == "lr0-up" && reg0 == 10.0.0.10), action=(eth.dst = 00:00:00:00:01:03;
next;)
+])
+check ovn-nbctl --wait=sb set NAT $nat_uuid
external_mac="\"00:00:00:00:01:02\""
+
+# Equal contributors share one MAC flow. An ambiguous MAC for the same
+# output port and next-hop IP must not depend on ECMP iteration order.
+check ovn-nbctl lr-add lr2
+check ovn-nbctl lrp-add lr2 lr2-up 00:00:00:00:00:04 10.0.0.2/24
+check ovn-nbctl lrp-set-gateway-chassis lr2-up hv3
+check ovn-nbctl lsp-add-router-port provider prov-lr2 lr2-up
+check ovn-nbctl lrp-add lr2 lr2-be 00:00:00:00:00:05 192.168.2.1/24
+check ovn-nbctl ls-add be2
+check ovn-nbctl lsp-add-router-port be2 be2-lr2 lr2-be
+check ovn-nbctl lsp-add be2 be-vm2
+check ovn-nbctl lsp-set-addresses be-vm2 "00:00:00:00:02:01 192.168.2.10"
+check ovn-nbctl --wait=sb lr-nat-add lr2 dnat_and_snat \
+ 10.0.0.10 192.168.2.10 be-vm2 00:00:00:00:01:02
+AT_CHECK([check_fip_mac], [0], [dnl
+ table=??(lr_in_arp_resolve ), priority=100 , match=(reg9[[9]] == 1 &&
outport == "lr0-up" && reg0 == 10.0.0.10), action=(eth.dst = 00:00:00:00:01:02;
next;)
+])
+nat2_uuid=$(ovn-nbctl --bare --columns=_uuid find NAT logical_ip=192.168.2.10)
+check ovn-nbctl --wait=sb set NAT $nat2_uuid
external_mac="\"00:00:00:00:02:02\""
+AT_CHECK([check_fip_mac], [0], [dnl
+ table=??(lr_in_arp_resolve ), priority=100 , match=(reg9[[9]] == 1 &&
outport == "lr0-up" && reg0 == 10.0.0.10), action=(drop;)
+])
+check ovn-nbctl --wait=sb lr-nat-del lr2 dnat_and_snat 10.0.0.10
+AT_CHECK([check_fip_mac], [0], [dnl
+ table=??(lr_in_arp_resolve ), priority=100 , match=(reg9[[9]] == 1 &&
outport == "lr0-up" && reg0 == 10.0.0.10), action=(eth.dst = 00:00:00:00:01:02;
next;)
+])
+
+# The same destination through different output ports keeps each selected
+# next hop's MAC, including when the routes form an ECMP group.
+check ovn-nbctl lsp-del prov-lr2
+check ovn-nbctl lrp-del lr2-up
+check ovn-nbctl ls-add provider2
+check ovn-nbctl lsp-add-localnet-port provider2 ln-prov2 physnet2
+check ovn-nbctl lrp-add lr0 lr0-up2 00:00:00:00:00:06
+check ovn-nbctl lrp-set-options lr0-up2 dynamic-routing-redistribute=nat
+check ovn-nbctl lsp-add-router-port provider2 prov2-lr0 lr0-up2
+check ovn-nbctl lrp-add lr2 lr2-up2 00:00:00:00:00:07 10.1.0.1/24
+check ovn-nbctl lrp-set-gateway-chassis lr2-up2 hv3
+check ovn-nbctl lsp-add-router-port provider2 prov2-lr2 lr2-up2
+check ovn-nbctl --wait=sb lr-nat-add lr2 dnat_and_snat \
+ 10.0.0.10 192.168.2.10 be-vm2 00:00:00:00:02:02
+AT_CHECK([check_fip_mac], [0], [dnl
+ table=??(lr_in_arp_resolve ), priority=100 , match=(reg9[[9]] == 1 &&
outport == "lr0-up" && reg0 == 10.0.0.10), action=(eth.dst = 00:00:00:00:01:02;
next;)
+])
+AT_CHECK([grep 'lr_in_arp_resolve.*outport == "lr0-up2" && reg0 == 10.0.0.10'
lr0_flows | ovn_strip_lflows], [0], [dnl
+ table=??(lr_in_arp_resolve ), priority=100 , match=(reg9[[9]] == 1 &&
outport == "lr0-up2" && reg0 == 10.0.0.10), action=(eth.dst =
00:00:00:00:02:02; next;)
+])
+AT_CHECK([grep 'lr_in_ip_routing_ecmp.*reg0 = 10.0.0.10;' lr0_flows | grep -c
'outport = "lr0-up'], [0], [2
+])
+check ovn-nbctl --wait=sb lrp-del lr0-up2
+ovn-sbctl lflow-list lr0 > lr0_flows
+AT_CHECK([grep 'lr_in_arp_resolve.*outport == "lr0-up2"' lr0_flows], [1])
+check ovn-nbctl --wait=sb lr-del lr2
+
+# Changing the FIP replaces both its advertisement and forwarding route.
+nat_uuid=$(ovn-nbctl --bare --columns=_uuid find NAT external_ip=10.0.0.10)
+check ovn-nbctl --wait=sb set NAT $nat_uuid external_ip=10.0.0.11
+check_row_count Advertised_Route 0 ip_prefix=10.0.0.10
+check_row_count Advertised_Route 1 ip_prefix=10.0.0.11 \
+ datapath=$datapath_lr0 logical_port=$pb_lr0_up tracked_port=$pb_be_vm1
+AT_CHECK([check_fip_route], [0], [dnl
+ table=??(lr_in_ip_routing ), priority=1935 , match=(ip4.dst ==
10.0.0.11/32), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 10.0.0.11; eth.src =
00:00:00:00:00:01; outport = "lr0-up"; flags.loopback = 1; reg9[[9]] = 1; next;)
+])
+
+AT_CHECK([check_fip_mac], [0], [dnl
+ table=??(lr_in_arp_resolve ), priority=100 , match=(reg9[[9]] == 1 &&
outport == "lr0-up" && reg0 == 10.0.0.11), action=(eth.dst = 00:00:00:00:01:02;
next;)
+])
+
+# Centralized NAT must continue using the gateway as next hop.
+check ovn-nbctl --wait=sb clear NAT $nat_uuid logical_port external_mac
+AT_CHECK([check_fip_mac], [1])
+ovn-sbctl lflow-list lr0 > lr0_flows
+AT_CHECK([grep 'lr_in_ip_routing.*10.0.0.11/32' lr0_flows | ovn_strip_lflows],
[0], [dnl
+ table=??(lr_in_ip_routing ), priority=1935 , match=(ip4.dst ==
10.0.0.11/32), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 10.0.0.1; eth.src =
00:00:00:00:00:01; outport = "lr0-up"; flags.loopback = 1; reg9[[9]] = 1; next;)
+])
+
+check ovn-nbctl --wait=sb set NAT $nat_uuid \
+ logical_port=be-vm1 external_mac="\"00:00:00:00:01:02\""
+check_row_count Advertised_Route 1 ip_prefix=10.0.0.11 \
+ datapath=$datapath_lr0 logical_port=$pb_lr0_up tracked_port=$pb_be_vm1
+AT_CHECK([check_fip_route], [0], [dnl
+ table=??(lr_in_ip_routing ), priority=1935 , match=(ip4.dst ==
10.0.0.11/32), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 10.0.0.11; eth.src =
00:00:00:00:00:01; outport = "lr0-up"; flags.loopback = 1; reg9[[9]] = 1; next;)
+])
+
+AT_CHECK([check_fip_mac], [0], [dnl
+ table=??(lr_in_arp_resolve ), priority=100 , match=(reg9[[9]] == 1 &&
outport == "lr0-up" && reg0 == 10.0.0.11), action=(eth.dst = 00:00:00:00:01:02;
next;)
+])
+
+# Removing the advertising adjacency removes its owned MAC flow.
+check ovn-nbctl --wait=sb lrp-del lr0-up
+AT_CHECK([check_fip_mac], [1])
+check ovn-nbctl lrp-add lr0 lr0-up 00:00:00:00:00:01
+check ovn-nbctl lrp-set-options lr0-up dynamic-routing-redistribute=nat
+check ovn-nbctl --wait=sb lsp-set-options prov-lr0 router-port=lr0-up
+AT_CHECK([check_fip_mac], [0], [dnl
+ table=??(lr_in_arp_resolve ), priority=100 , match=(reg9[[9]] == 1 &&
outport == "lr0-up" && reg0 == 10.0.0.11), action=(eth.dst = 00:00:00:00:01:02;
next;)
+])
+
+# A full recomputation must produce the same flows as the incremental
+# NAT updates above.
+ovn_strip_lflows < lr0_flows > lr0_flows_before
+check as northd ovn-appctl -t ovn-northd inc-engine/recompute
+check ovn-nbctl --wait=sb sync
+ovn-sbctl lflow-list lr0 | ovn_strip_lflows > lr0_flows_recomputed
+AT_CHECK([diff -u lr0_flows_before lr0_flows_recomputed])
+
+check ovn-nbctl --wait=sb lr-nat-del lr1 dnat_and_snat 10.0.0.11
+check_row_count Advertised_Route 0 ip_prefix=10.0.0.11
+ovn-sbctl lflow-list lr0 > lr0_flows
+AT_CHECK([grep 'lr_in_ip_routing.*10.0.0.11/32' lr0_flows], [1])
+AT_CHECK([check_fip_mac], [1])
+
+OVN_CLEANUP_NORTHD
+AT_CLEANUP
+])
+
+OVN_FOR_EACH_NORTHD_NO_HV([
+AT_SETUP([dynamic-routing - NAT redistribute through a shared distributed
gateway IPv6])
+AT_KEYWORDS([dynamic-routing])
+ovn_start
+
+check ovn-nbctl lr-add lr0
+check ovn-nbctl set Logical_Router lr0 options:dynamic-routing=true
+check ovn-nbctl lrp-add lr0 lr0-up 00:00:00:00:00:01
+check ovn-nbctl lrp-set-options lr0-up dynamic-routing-redistribute=nat
+check ovn-nbctl ls-add provider
+check ovn-nbctl lsp-add-router-port provider prov-lr0 lr0-up
+check ovn-nbctl lsp-add-localnet-port provider ln-prov physnet
+
+check ovn-nbctl lr-add lr1
+check ovn-nbctl lrp-add lr1 lr1-up 00:00:00:00:00:02 2001:db8:1::1/64
+check ovn-nbctl lrp-set-gateway-chassis lr1-up hv2
+check ovn-nbctl lsp-add-router-port provider prov-lr1 lr1-up
+check ovn-nbctl lrp-add lr1 lr1-be 00:00:00:00:00:03 2001:db8:2::1/64
+check ovn-nbctl ls-add be
+check ovn-nbctl lsp-add-router-port be be-lr1 lr1-be
+check ovn-nbctl lsp-add be be-vm1
+check ovn-nbctl lsp-set-addresses be-vm1 "00:00:00:00:01:01 2001:db8:2::10"
+
+# A unique DGP also permits a NAT external IP outside its
+# prefix. The forwarding next hop must still be the FIP, not the DGP IP.
+check ovn-nbctl lr-nat-add lr1 dnat_and_snat 2001:db8:ffff::10 2001:db8:2::10
be-vm1 00:00:00:00:01:02
+check ovn-nbctl --wait=sb sync
+
+pb_be_vm1=$(fetch_column Port_Binding _uuid logical_port=be-vm1)
+check_row_count Advertised_Route 1 \
+ ip_prefix="2001\:db8\:ffff\:\:10" tracked_port=$pb_be_vm1
+ovn-sbctl lflow-list lr0 > lr0_flows
+AT_CHECK([grep 'lr_in_ip_routing.*2001:db8:ffff::10/128' lr0_flows |
ovn_strip_lflows], [0], [dnl
+ table=??(lr_in_ip_routing ), priority=3087 , match=(ip6.dst ==
2001:db8:ffff::10/128), action=(ip.ttl--; reg8[[0..15]] = 0; xxreg0 =
2001:db8:ffff::10; eth.src = 00:00:00:00:00:01; outport = "lr0-up";
flags.loopback = 1; reg9[[9]] = 0; next;)
+])
+
+# The off-prefix IPv6 next hop also uses its known external MAC.
+AT_CHECK([grep 'lr_in_arp_resolve.*outport == "lr0-up" && xxreg0 ==
2001:db8:ffff::10' lr0_flows | ovn_strip_lflows], [0], [dnl
+ table=??(lr_in_arp_resolve ), priority=100 , match=(reg9[[9]] == 0 &&
outport == "lr0-up" && xxreg0 == 2001:db8:ffff::10), action=(eth.dst =
00:00:00:00:01:02; next;)
+])
+
+OVN_CLEANUP_NORTHD
+AT_CLEANUP
+])
+
OVN_FOR_EACH_NORTHD_NO_HV([
AT_SETUP([dynamic-routing - LB redistribute forwarding route IPv6])
AT_KEYWORDS([dynamic-routing])
diff --git a/tests/system-ovn.at b/tests/system-ovn.at
index 3cb4b858a..546a755df 100644
--- a/tests/system-ovn.at
+++ b/tests/system-ovn.at
@@ -16793,6 +16793,200 @@ OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port
patch-.*/d
AT_CLEANUP
])
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([dynamic-routing - distributed NAT next hop on shared localnet])
+AT_KEYWORDS([dynamic-routing distributed-nat-nexthop])
+AT_SKIP_IF([test $HAVE_TCPDUMP = no])
+CHECK_CONNTRACK()
+CHECK_CONNTRACK_NAT()
+VRF_RESERVE([1344])
+
+ovn_start
+OVS_TRAFFIC_VSWITCHD_START()
+ADD_BR([br-int])
+ADD_BR([br-shared], [set Bridge br-shared fail-mode=standalone])
+check ovs-vsctl \
+ -- set Open_vSwitch . external-ids:system-id=hv1 \
+ external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
+ external-ids:ovn-encap-type=geneve \
+ external-ids:ovn-encap-ip=169.0.0.1 \
+ external-ids:ovn-bridge-mappings=shared:br-shared \
+ -- set Bridge br-int fail-mode=secure \
+ other-config:disable-in-band=true
+start_daemon ovn-controller
+
+dnl The backend and advertising router are local. The distributed gateway
+dnl belongs to another chassis, so sending to its MAC cannot reach the VM.
+check ovn-sbctl chassis-add hv2 geneve 169.0.0.2
+OVS_WAIT_UNTIL([
+ tunnel_ofport=$(ovs-vsctl --bare --columns=ofport find Interface \
+ options:local_ip=169.0.0.1 options:remote_ip=169.0.0.2)
+ test -n "$tunnel_ofport" && test "$tunnel_ofport" -ge 0
+])
+
+check ovn-nbctl \
+ -- lr-add adv \
+ -- set Logical_Router adv options:chassis=hv1 \
+ options:dynamic-routing=true options:dynamic-routing-vrf-id=1344 \
+ options:mac_binding_age_threshold=0 \
+ -- lrp-add adv adv-client 00:00:00:00:aa:02 192.0.2.1/24 \
+ -- ls-add clients \
+ -- lsp-add-router-port clients clients-adv adv-client \
+ -- lsp-add clients client \
+ -- lsp-set-addresses client '00:00:00:00:cc:02 192.0.2.2' \
+ -- lrp-add adv adv-shared 00:00:00:00:aa:01 \
+ -- lrp-set-options adv-shared dynamic-routing-maintain-vrf=true \
+ dynamic-routing-redistribute=nat \
+ -- ls-add shared \
+ -- lsp-add-router-port shared shared-adv adv-shared \
+ -- lsp-add-localnet-port shared shared-localnet shared \
+ -- lr-add tenant \
+ -- lrp-add tenant tenant-shared 00:00:00:00:bb:01 172.16.1.1/24 \
+ -- lsp-add-router-port shared shared-tenant tenant-shared \
+ -- set Logical_Switch_Port shared-tenant options:nat-addresses=router \
+ -- lrp-set-gateway-chassis tenant-shared hv2 \
+ -- lrp-add tenant tenant-vms 00:00:00:00:bb:02 10.0.0.1/24 \
+ -- ls-add vms \
+ -- lsp-add-router-port vms vms-tenant tenant-vms \
+ -- lsp-add vms vm \
+ -- lsp-set-addresses vm '00:00:00:00:dd:02 10.0.0.2' \
+ -- lr-route-add tenant 192.0.2.0/24 fe80::200:ff:fe00:aa01 tenant-shared \
+ -- lr-nat-add tenant dnat_and_snat 172.16.1.100 10.0.0.2 \
+ vm 00:00:00:00:ee:01
+
+ADD_NAMESPACES(client, vm)
+ADD_VETH(client, client, br-int, "192.0.2.2/24", \
+ "00:00:00:00:cc:02", "192.0.2.1")
+ADD_VETH(vm, vm, br-int, "10.0.0.2/24", \
+ "00:00:00:00:dd:02", "10.0.0.1")
+check ovn-nbctl --wait=hv sync
+wait_for_ports_up client vm
+wait_row_count Port_Binding 1 logical_port=cr-tenant-shared
+hv1=$(fetch_column Chassis _uuid name=hv1)
+hv2=$(fetch_column Chassis _uuid name=hv2)
+check ovn-sbctl set Port_Binding cr-tenant-shared chassis=$hv2
+check ovn-nbctl --wait=hv sync
+check_row_count Port_Binding 1 logical_port=vm chassis=$hv1
+check_row_count Port_Binding 1 logical_port=cr-tenant-shared chassis=$hv2
+OVN_ROUTE_EQUAL([ovnvrf1344], [dnl
+blackhole 172.16.1.100 proto ovn metric 100])
+
+AS_BOX([Cold ARP and traffic use the distributed NAT next hop])
+check ovn-sbctl --all destroy MAC_Binding
+NS_CHECK_EXEC([client], [ip neigh flush dev client])
+NS_CHECK_EXEC([vm], [ip neigh flush dev vm])
+NETNS_START_TCPDUMP([client], [-nnei client -c 1 arp], [client-arp])
+NS_CHECK_EXEC([client], [ping -q -c 3 -i 0.3 -w 3 172.16.1.100 | FORMAT_PING],
+ [0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+OVS_WAIT_UNTIL([grep -F 'Request who-has 192.0.2.1 tell 192.0.2.2' \
+ client-arp.tcpdump])
+AS_BOX([A sibling proxy cannot override the distributed NAT next-hop MAC])
+dnl Model two localnets joined by the provider bridge. The sibling router
+dnl port proxies FIPs toward the provider network, but its MAC cannot reach
+dnl the distributed NAT backend from adv-shared.
+check ovn-nbctl \
+ -- lrp-add adv adv-interconnect 00:00:00:00:aa:03 172.16.1.254/24 \
+ -- ls-add interconnect \
+ -- lsp-add-router-port interconnect interconnect-adv adv-interconnect \
+ -- set Logical_Switch_Port interconnect-adv \
+ options:arp_proxy=172.16.1.100 \
+ -- lsp-add-localnet-port interconnect interconnect-localnet shared
+check ovn-nbctl --wait=hv sync
+OVS_WAIT_UNTIL([
+ shared_ofport=$(ovs-vsctl get Interface \
+ patch-shared-localnet-to-br-int ofport)
+ interconnect_ofport=$(ovs-vsctl get Interface \
+ patch-interconnect-localnet-to-br-int ofport)
+ test "$shared_ofport" -gt 0 && test "$interconnect_ofport" -gt 0
+])
+check ovs-ofctl add-flow br-shared \
+ "priority=10,in_port=$shared_ofport
actions=set_field:00:00:00:00:aa:03->eth_dst,output:$interconnect_ofport"
+check ovs-ofctl add-flow br-shared \
+ "priority=10,in_port=$interconnect_ofport actions=output:$shared_ofport"
+check ovs-ofctl add-flow br-shared 'priority=0 actions=drop'
+
+dnl Keep the incorrect dynamic binding throughout the packet check. A
+dnl known NAT next hop must take precedence without rewriting that row.
+fip_binding=$(fetch_column MAC_Binding _uuid logical_port=adv-shared \
+ ip='"172.16.1.100"')
+if test -n "$fip_binding"; then
+ check ovn-sbctl destroy MAC_Binding $fip_binding
+fi
+adv_dp=$(fetch_column Port_Binding datapath logical_port=adv-shared)
+AT_CHECK([ovn-sbctl create MAC_Binding datapath=$adv_dp \
+ logical_port=adv-shared ip='"172.16.1.100"' mac='"00:00:00:00:aa:03"'],
+ [0], [ignore])
+fip_binding=$(fetch_column MAC_Binding _uuid logical_port=adv-shared \
+ ip='"172.16.1.100"')
+check_row_count nb:Static_MAC_Binding 0
+NS_CHECK_EXEC([client], [ip neigh flush dev client])
+NS_CHECK_EXEC([vm], [ip neigh flush dev vm])
+NS_CHECK_EXEC([client], [ping -q -c 3 -i 0.3 -w 3 172.16.1.100 | FORMAT_PING],
+ [0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+check_row_count MAC_Binding 1 logical_port=adv-shared \
+ ip='"172.16.1.100"' mac='"00:00:00:00:aa:03"'
+check_column "$fip_binding" MAC_Binding _uuid \
+ logical_port=adv-shared ip='"172.16.1.100"'
+
+AS_BOX([Distributed NAT bypasses the remote redirect chassis])
+check ovn-nbctl --wait=hv set Logical_Router_Port tenant-shared \
+ options:reside-on-redirect-chassis=true
+NS_CHECK_EXEC([client], [ping -q -c 3 -i 0.3 -w 3 172.16.1.100 | FORMAT_PING],
+ [0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+AS_BOX([Native GARP refreshes a warm next hop after its MAC changes])
+nat=$(fetch_column nb:NAT _uuid external_ip='"172.16.1.100"')
+check ovn-nbctl --wait=hv set NAT $nat external_mac='"00:00:00:00:ee:02"'
+wait_row_count MAC_Binding 1 logical_port=adv-shared \
+ ip='"172.16.1.100"' mac='"00:00:00:00:ee:02"'
+check_row_count MAC_Binding 0 logical_port=adv-shared \
+ ip='"172.16.1.100"' mac='"00:00:00:00:ee:01"'
+NS_CHECK_EXEC([client], [ping -q -c 3 -i 0.3 -w 3 172.16.1.100 | FORMAT_PING],
+ [0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+AS_BOX([Resolve the changed MAC again with cold caches])
+check ovn-sbctl --all destroy MAC_Binding
+NS_CHECK_EXEC([client], [ip neigh flush dev client])
+NS_CHECK_EXEC([vm], [ip neigh flush dev vm])
+NS_CHECK_EXEC([client], [ping -q -c 3 -i 0.3 -w 3 172.16.1.100 | FORMAT_PING],
+ [0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+AS_BOX([Changing and removing the FIP withdraws its old route])
+check ovn-nbctl --wait=hv set NAT $nat external_ip='"172.16.1.101"'
+OVN_ROUTE_EQUAL([ovnvrf1344], [dnl
+blackhole 172.16.1.101 proto ovn metric 100])
+check ovn-sbctl --all destroy MAC_Binding
+NS_CHECK_EXEC([client], [ip neigh flush dev client])
+NS_CHECK_EXEC([vm], [ip neigh flush dev vm])
+NS_CHECK_EXEC([client], [ping -q -c 3 -i 0.3 -w 3 172.16.1.101 | FORMAT_PING],
+ [0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+NS_CHECK_EXEC([client], [ping -q -c 1 -W 1 172.16.1.100],
+ [1], [ignore], [ignore])
+check ovn-nbctl --wait=hv lr-nat-del tenant dnat_and_snat 172.16.1.101
+OVS_WAIT_UNTIL([ip route list vrf ovnvrf1344 > routes && test ! -s routes])
+NS_CHECK_EXEC([client], [ping -q -c 1 -W 1 172.16.1.101],
+ [1], [ignore], [ignore])
+
+OVN_CLEANUP_CONTROLLER([hv1])
+OVN_CLEANUP_NORTHD
+as
+OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
+/connection dropped.*/d"])
+AT_CLEANUP
+])
+
OVN_FOR_EACH_NORTHD([
AT_SETUP([dynamic-routing - DNAT and DNAT_AND_SNAT on gateway router IPv6])
AT_KEYWORDS([dynamic-routing])
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
