On Tue, Jun 16, 2026 at 3:41 AM Chanyeol Yoon <[email protected]> wrote:
> When a Logical Router has a 'dnat_and_snat' NAT entry with both > 'external_mac' and 'logical_port' set (a distributed floating IP), its > external IP/MAC are not advertised over BGP-EVPN. Only ports directly > attached to the EVPN-enabled provider Logical Switch are covered, so > floating IPs (a common CMS pattern, e.g. OpenStack Neutron with > distributed FIPs) stay unreachable from EVPN peers. > > Populate the SB Advertised_MAC_Binding for the provider Logical Switch > that carries the NAT's distributed gateway port, using the NAT's > 'external_ip' and 'external_mac'. ovn-controller then programs those > into the EVPN advertise interface FDB and neighbor table, so the > floating IPs are emitted as EVPN Type-2 MAC+IP routes on the chassis > that hosts the workload. > > This is gated by a new 'nat' token of the provider Logical Switch > 'dynamic-routing-redistribute' option, independent of the existing > 'ip' token: 'ip' keeps advertising VIF and router-port addresses while > 'nat' opts in to floating IPs. It mirrors the 'nat' token of the > Logical_Router 'dynamic-routing-redistribute' option, which advertises > the same NAT entries as EVPN Type-5 routes. > > The advertised set now depends on per-datapath EVPN settings, so > en-advertised-mac-binding-sync recomputes on northd and lr_nat changes; > toggling the redistribute tokens therefore takes effect immediately. > > Reported-by: Chanyeol Yoon <[email protected]> > Suggested-by: Ales Musil <[email protected]> > Signed-off-by: Chanyeol Yoon <[email protected]> > --- > Hi Chanyeol, thank you for the patch. We should also extend the section "NAT.options:dynamic-routing-advertise" and mention that this also influences the Type-2 routes. > NEWS | 6 ++ > lib/ovn-util.c | 3 + > lib/ovn-util.h | 3 +- > northd/en-advertised-route-sync.c | 101 +++++++++++++++++++++++++++--- > northd/inc-proc-northd.c | 14 +++-- > ovn-nb.xml | 14 +++++ > tests/ovn-inc-proc-graph-dump.at | 3 +- > tests/ovn-northd.at | 93 +++++++++++++++++++++++++++ > 8 files changed, 221 insertions(+), 16 deletions(-) > > diff --git a/NEWS b/NEWS > index 748ae30eb..01e99689f 100644 > --- a/NEWS > +++ b/NEWS > @@ -15,6 +15,12 @@ Post v26.03.0 > * Add ECMP/multi-homing support for EVPN FDB entries. FDB entries > backed by a kernel nexthop group are load-balanced via OpenFlow > select groups with weighted buckets. > + * EVPN: distributed dnat_and_snat NAT entries (e.g. floating IPs) > + are now populated in the SB Advertised_MAC_Binding table on the > + provider logical switch that carries the distributed gateway > + port, so that they can be advertised as EVPN Type-2 routes. This > + is gated on a new "nat" token of the Logical_Switch > + "dynamic-routing-redistribute" option, independent of "ip". > - Added "override-connected" option to Logical Router Static Routes to > mark > static routes as higher-priority than connected routes, which in > turn led > to changes in administrative distance for specific route types. > Please see > diff --git a/lib/ovn-util.c b/lib/ovn-util.c > index e6143d7a9..4ad5f993c 100644 > --- a/lib/ovn-util.c > +++ b/lib/ovn-util.c > @@ -1754,6 +1754,9 @@ parse_neigh_dynamic_redistribute(const struct smap > *options) > if (!strcmp(token, "ip")) { > mode |= NRM_IP; > } > + if (!strcmp(token, "nat")) { > + mode |= NRM_NAT; > + } > } > free(tokstr); > > diff --git a/lib/ovn-util.h b/lib/ovn-util.h > index bfca178e4..ad1dfa328 100644 > --- a/lib/ovn-util.h > +++ b/lib/ovn-util.h > @@ -796,7 +796,8 @@ char *normalize_addr_str(const char *orig_addr); > > #define NEIGH_REDISTRIBUTE_MODES \ > NEIGH_REDISTRIBUTE_MODE(FDB, 0) \ > - NEIGH_REDISTRIBUTE_MODE(IP, 1) > + NEIGH_REDISTRIBUTE_MODE(IP, 1) \ > + NEIGH_REDISTRIBUTE_MODE(NAT, 2) > > enum neigh_redistribute_mode_bits { > #define NEIGH_REDISTRIBUTE_MODE(PROTOCOL, BIT) NRM_##PROTOCOL##_BIT = BIT, > diff --git a/northd/en-advertised-route-sync.c > b/northd/en-advertised-route-sync.c > index 4a8d13232..3c9d0bcb7 100644 > --- a/northd/en-advertised-route-sync.c > +++ b/northd/en-advertised-route-sync.c > @@ -833,6 +833,18 @@ evpn_ip_redistribution_enabled(const struct > ovn_datapath *od) > return nrm_mode_IP_is_set(mode); > } > > +static bool > +evpn_nat_redistribution_enabled(const struct ovn_datapath *od) > +{ > + if (!od->has_evpn_vni) { > + return false; > + } > + > + enum neigh_redistribute_mode mode = > + parse_neigh_dynamic_redistribute(&od->nbs->other_config); > + return nrm_mode_NAT_is_set(mode); > +} > + > static uint32_t > advertised_mac_binding_get_hash(const struct sbrec_datapath_binding *dp, > const struct sbrec_port_binding *sb, > @@ -890,22 +902,23 @@ advertised_mac_binding_entry_destroy(struct > advertised_mac_binding *e) > } > > static void > -advertised_mac_binding_add(struct hmap *map, > - const struct sbrec_datapath_binding *dp, > - const struct sbrec_port_binding *sb, > - struct lport_addresses *addr) > +advertised_mac_binding_add_with_mac(struct hmap *map, > + const struct sbrec_datapath_binding > *dp, > + const struct sbrec_port_binding *sb, > + const struct lport_addresses *addr, > + const char *mac) > { > - if (!addr) { > + if (!addr || !mac) { > return; > } > > for (size_t i = 0; i < addr->n_ipv4_addrs; i++) { > if (!advertised_mac_binding_entry_find(map, dp, sb, > addr->ipv4_addrs[i].addr_s, > - addr->ea_s)) { > + mac)) { > advertised_mac_binding_entry_add(map, dp, sb, > addr->ipv4_addrs[i].addr_s, > - addr->ea_s); > + mac); > } > } > > @@ -916,14 +929,27 @@ advertised_mac_binding_add(struct hmap *map, > > if (!advertised_mac_binding_entry_find(map, dp, sb, > addr->ipv6_addrs[i].addr_s, > - addr->ea_s)) { > + mac)) { > advertised_mac_binding_entry_add(map, dp, sb, > addr->ipv6_addrs[i].addr_s, > - addr->ea_s); > + mac); > } > } > } > > +static void > +advertised_mac_binding_add(struct hmap *map, > + const struct sbrec_datapath_binding *dp, > + const struct sbrec_port_binding *sb, > + struct lport_addresses *addr) > +{ > + if (!addr) { > + return; > + } > + > + advertised_mac_binding_add_with_mac(map, dp, sb, addr, addr->ea_s); > +} > + > static void > build_advertised_mac_binding(const struct ovn_datapath *od, struct hmap > *map) > { > @@ -956,6 +982,57 @@ build_advertised_mac_binding(const struct > ovn_datapath *od, struct hmap *map) > } > } > > +/* Advertise distributed dnat_and_snat NAT entries (e.g. floating IPs) > over > + * EVPN. The advertisement is attached to the provider Logical Switch > that > + * carries the NAT's distributed gateway port, provided that LS has NAT > + * redistribution enabled via 'dynamic-routing-redistribute=nat'. */ > +static void > +build_advertised_mac_binding_lr(const struct ovn_datapath *od, > + const struct lr_nat_table *lr_nats, > + struct hmap *map) > +{ > + ovs_assert(od->nbr); > + > + const struct lr_nat_record *lrnat_rec = > + lr_nat_table_find_by_uuid(lr_nats, od->nbr->header_.uuid); > + if (!lrnat_rec) { > + return; > + } > + > + for (size_t i = 0; i < lrnat_rec->n_nat_entries; i++) { > + const struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; > + const struct nbrec_nat *nat = nat_entry->nb; > + > + if (!nat_entry->is_valid || !nat_entry->is_distributed || > + nat_entry->type != DNAT_AND_SNAT) { > + continue; > + } > + > + if (!nat->external_mac) { > + continue; > + } > + > + if (!smap_get_bool(&nat->options, "dynamic-routing-advertise", > true)) { > + continue; > + } > + > + const struct ovn_port *dgp = nat_entry->l3dgw_port; > + if (!dgp || !dgp->peer || !dgp->peer->sb || !dgp->peer->od) { > + continue; > + } > + > + const struct ovn_datapath *peer_od = dgp->peer->od; > + if (!peer_od->nbs || !evpn_nat_redistribution_enabled(peer_od)) { > + continue; > + } > + > + advertised_mac_binding_add_with_mac(map, peer_od->sdp->sb_dp, > + dgp->peer->sb, > + &nat_entry->ext_addrs, > + nat->external_mac); > + } > +} > + > void * > en_advertised_mac_binding_sync_init(struct engine_node *node OVS_UNUSED, > struct engine_arg *arg OVS_UNUSED) > @@ -970,6 +1047,8 @@ en_advertised_mac_binding_sync_run(struct engine_node > *node, > struct northd_data *northd_data = engine_get_input_data("northd", > node); > const struct sbrec_advertised_mac_binding_table *sbrec_adv_mb_table = > EN_OVSDB_GET(engine_get_input("SB_advertised_mac_binding", node)); > + struct ed_type_lr_nat_data *lr_nat_data = > + engine_get_input_data("lr_nat", node); > const struct engine_context *eng_ctx = engine_get_context(); > > struct hmap advertised_mac_binding_map = > @@ -979,6 +1058,10 @@ en_advertised_mac_binding_sync_run(struct > engine_node *node, > HMAP_FOR_EACH (od, key_node, &northd_data->ls_datapaths.datapaths) { > build_advertised_mac_binding(od, &advertised_mac_binding_map); > } > + HMAP_FOR_EACH (od, key_node, &northd_data->lr_datapaths.datapaths) { > + build_advertised_mac_binding_lr(od, &lr_nat_data->lr_nats, > + &advertised_mac_binding_map); > + } > > struct advertised_mac_binding *e; > const struct sbrec_advertised_mac_binding *sb_adv_mb; > diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c > index a2b464411..82160d94b 100644 > --- a/northd/inc-proc-northd.c > +++ b/northd/inc-proc-northd.c > @@ -364,11 +364,15 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, > NULL); > engine_add_input(&en_advertised_mac_binding_sync, > &en_sb_advertised_mac_binding, NULL); > - /* No need for an explicit handler for northd changes. > - * We do need to access en_northd (input) data, i.e., to > - * lookup OVN ports. */ > - engine_add_input(&en_advertised_mac_binding_sync, &en_northd, > - engine_noop_handler); > + /* Recompute on northd changes: besides looking up OVN ports, this > node > + * reads per-datapath EVPN settings (dynamic-routing-vni and the > + * dynamic-routing-redistribute 'ip'/'nat' tokens). Toggling those > must > + * re-evaluate which IPs/MACs and floating IPs are advertised. */ > nit: We don't usually put comments for NULL or handlers, the only option when we comment is "engine_noop_handler". > + engine_add_input(&en_advertised_mac_binding_sync, &en_northd, NULL); > First, this feels like a fix for existing behavior, if that's the case please make it a separate commit with an additional test or an extension of existing test. northd is a node that does a lot of processing, I'm afraid that making this handler NULL would cause us to perform many recomputes that are not needed. I suggest utilizing the "struct tracked_dps trk_switches" to check if we should recompute because a given switch might have changed, this will limit the recompute scope significantly. We could go a step further and even try to check if the changed options have an impact but that might not be worth the effort. For this addition we can use "struct hmapx trk_nat_lrs" as an indication that the NATs have changed instead of the en_lr_nat below. > + /* Distributed dnat_and_snat NAT entries (e.g. floating IPs) are > + * advertised via EVPN as well. Trigger a recompute on any lr_nat > + * change so that those entries are picked up. */ > + engine_add_input(&en_advertised_mac_binding_sync, &en_lr_nat, NULL); > We shouldn't need this dependency as described above. > > engine_add_input(&en_learned_route_sync, &en_sb_learned_route, > learned_route_sync_sb_learned_route_change_handler); > diff --git a/ovn-nb.xml b/ovn-nb.xml > index 15fb1d7e8..8e2248f97 100644 > --- a/ovn-nb.xml > +++ b/ovn-nb.xml > @@ -1032,6 +1032,20 @@ > all IPs/MACs bindings that are local to the chassis. This > applies to > VIFs and router ports. > </p> > + > + <p> > + If <code>nat</code> is specified then the external IP/MAC of any > + distributed <code>dnat_and_snat</code> NAT entry (for example a > + floating IP) whose distributed gateway port is attached to this > + logical switch is advertised as an EVPN Type-2 route, using the > + NAT's <ref column="external_mac" table="NAT"/>. This is > independent > + of <code>ip</code>: enabling <code>ip</code> alone does not > advertise > + floating IPs, and enabling <code>nat</code> alone does not > advertise > + VIF or router-port addresses. This mirrors the <code>nat</code> > + token of <ref column="options" > key="dynamic-routing-redistribute" > + table="Logical_Router"/>, which advertises the same NAT entries > as > + EVPN Type-5 routes. > + </p> > </column> > </group> > > diff --git a/tests/ovn-inc-proc-graph-dump.at b/tests/ > ovn-inc-proc-graph-dump.at > index 3750339d0..c37845188 100644 > --- a/tests/ovn-inc-proc-graph-dump.at > +++ b/tests/ovn-inc-proc-graph-dump.at > @@ -229,7 +229,8 @@ digraph "Incremental-Processing-Engine" { > advertised_mac_binding_sync [[style=filled, shape=box, > fillcolor=white, label="advertised_mac_binding_sync"]]; > SB_port_binding -> advertised_mac_binding_sync [[label=""]]; > SB_advertised_mac_binding -> advertised_mac_binding_sync > [[label=""]]; > - northd -> advertised_mac_binding_sync > [[label="engine_noop_handler"]]; > + northd -> advertised_mac_binding_sync [[label=""]]; > + lr_nat -> advertised_mac_binding_sync [[label=""]]; > northd_output [[style=filled, shape=box, fillcolor=white, > label="northd_output"]]; > acl_id -> northd_output [[label="northd_output_acl_id_handler"]]; > sync_from_sb -> northd_output [[label=""]]; > diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at > index 7f4a88d4e..1aa483111 100644 > --- a/tests/ovn-northd.at > +++ b/tests/ovn-northd.at > @@ -20264,6 +20264,99 @@ OVN_CLEANUP_NORTHD > AT_CLEANUP > ]) > > +OVN_FOR_EACH_NORTHD_NO_HV([ > +AT_SETUP([LR EVPN Advertised_MAC_Binding sync for distributed NAT]) > +ovn_start > + > +check ovn-sbctl chassis-add ch geneve 127.0.0.1 > + > +AS_BOX([Build topology: tenant LS, LR with DGP, EVPN-enabled provider LS]) > +check ovn-nbctl ls-add ls-tenant > \ > + -- lsp-add ls-tenant vm1 > \ > + -- lsp-set-addresses vm1 "00:00:00:00:02:00 10.0.0.10" > +check ovn-nbctl ls-add ls-evpn > \ > + -- set logical_switch ls-evpn other_config:dynamic-routing-vni=10 > \ > + -- set logical_switch ls-evpn > other_config:dynamic-routing-redistribute=ip > +dnl A provider logical switch has a localnet port; without it the DGP's > +dnl peer gets a chassisredirect port and the NAT is not considered > +dnl distributed (so floating IPs would never be advertised). > +check ovn-nbctl lsp-add ls-evpn ln-evpn > \ > + -- lsp-set-type ln-evpn localnet > \ > + -- lsp-set-addresses ln-evpn unknown > \ > + -- lsp-set-options ln-evpn network_name=phys > +check ovn-nbctl lr-add lr > \ > + -- lrp-add lr lrp-int 00:00:00:00:01:01 10.0.0.1/24 > \ > + -- lrp-add lr lrp-ext 00:00:00:00:01:02 172.16.0.1/24 > \ > + -- lrp-set-gateway-chassis lrp-ext ch > \ > + -- lsp-add ls-tenant ls-tenant-lrp > \ > + -- lsp-set-type ls-tenant-lrp router > \ > + -- lsp-set-addresses ls-tenant-lrp 00:00:00:00:01:01 > \ > + -- lsp-set-options ls-tenant-lrp router-port=lrp-int > \ > + -- lsp-add ls-evpn ls-evpn-lrp > \ > + -- lsp-set-type ls-evpn-lrp router > \ > + -- lsp-set-addresses ls-evpn-lrp 00:00:00:00:01:02 > \ > + -- lsp-set-options ls-evpn-lrp router-port=lrp-ext > +check ovn-nbctl --wait=sb sync > + > +ls_evpn_uuid=$(fetch_column Datapath_Binding _uuid > external_ids:name=ls-evpn) > + > +AS_BOX([Baseline: LRP MAC/IP advertised, no NAT yet]) > +check_row_count Advertised_MAC_Binding 1 ip=172.16.0.1 > datapath=$ls_evpn_uuid mac='00\:00\:00\:00\:01\:02' > +check_row_count Advertised_MAC_Binding 1 > + > +AS_BOX([Non-distributed NATs are not advertised]) > +check ovn-nbctl --wait=sb lr-nat-add lr snat 172.16.0.50 10.0.0.10 > +check ovn-nbctl --wait=sb lr-nat-add lr dnat 172.16.0.51 10.0.0.10 > +check_row_count Advertised_MAC_Binding 1 > +check ovn-nbctl --wait=sb lr-nat-del lr snat 172.16.0.50 > +check ovn-nbctl --wait=sb lr-nat-del lr dnat 172.16.0.51 > + > +AS_BOX([dnat_and_snat without external_mac is not advertised]) > +check ovn-nbctl --wait=sb lr-nat-add lr dnat_and_snat 172.16.0.52 > 10.0.0.10 > +check_row_count Advertised_MAC_Binding 1 > +check ovn-nbctl --wait=sb lr-nat-del lr dnat_and_snat 172.16.0.52 > + > +AS_BOX([Distributed FIP is not advertised with 'ip' alone (needs 'nat')]) > +check ovn-nbctl --wait=sb lr-nat-add lr dnat_and_snat 172.16.0.100 > 10.0.0.10 vm1 00:11:22:33:44:55 > +check_row_count Advertised_MAC_Binding 0 ip=172.16.0.100 > +check_row_count Advertised_MAC_Binding 1 > + > +AS_BOX([Adding the 'nat' token advertises the FIP]) > +check ovn-nbctl --wait=sb set logical_switch ls-evpn > other_config:dynamic-routing-redistribute=ip,nat > +check_row_count Advertised_MAC_Binding 1 ip=172.16.0.100 > datapath=$ls_evpn_uuid mac='00\:11\:22\:33\:44\:55' > +check_row_count Advertised_MAC_Binding 2 > + > +AS_BOX([Adding a second FIP yields a second advertised entry]) > +check ovn-nbctl --wait=sb lsp-add ls-tenant vm2 > \ > + -- lsp-set-addresses vm2 "00:00:00:00:02:01 10.0.0.11" > +check ovn-nbctl --wait=sb lr-nat-add lr dnat_and_snat 172.16.0.101 > 10.0.0.11 vm2 00:11:22:33:44:66 > +check_row_count Advertised_MAC_Binding 1 ip=172.16.0.101 > datapath=$ls_evpn_uuid mac='00\:11\:22\:33\:44\:66' > +check_row_count Advertised_MAC_Binding 3 > + > +AS_BOX([Deleting a NAT removes its advertised entry]) > +check ovn-nbctl --wait=sb lr-nat-del lr dnat_and_snat 172.16.0.100 > +check_row_count Advertised_MAC_Binding 0 ip=172.16.0.100 > +check_row_count Advertised_MAC_Binding 2 > + > +AS_BOX(['nat' and 'ip' are independent: dropping 'nat' keeps the LRP > entry]) > +check ovn-nbctl --wait=sb set logical_switch ls-evpn > other_config:dynamic-routing-redistribute=ip > +check_row_count Advertised_MAC_Binding 0 ip=172.16.0.101 > +check_row_count Advertised_MAC_Binding 1 ip=172.16.0.1 > datapath=$ls_evpn_uuid mac='00\:00\:00\:00\:01\:02' > +check_row_count Advertised_MAC_Binding 1 > + > +AS_BOX([Disabling EVPN on the provider LS removes all advertisements]) > +check ovn-nbctl --wait=sb remove logical_switch ls-evpn other_config > dynamic-routing-redistribute > +check_row_count Advertised_MAC_Binding 0 > + > +AS_BOX([Re-enabling redistribute=ip,nat restores both LRP and FIP]) > +check ovn-nbctl --wait=sb set logical_switch ls-evpn > other_config:dynamic-routing-redistribute=ip,nat > +check_row_count Advertised_MAC_Binding 1 ip=172.16.0.101 > datapath=$ls_evpn_uuid mac='00\:11\:22\:33\:44\:66' > +check_row_count Advertised_MAC_Binding 2 > + > +OVN_CLEANUP_NORTHD > +AT_CLEANUP > +]) > + > OVN_FOR_EACH_NORTHD_NO_HV([ > AT_SETUP([IGMP northd crash]) > ovn_start > -- > 2.54.0 (Apple Git-156) > > Regards, Ales _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
