When a Logical Router has a 'dnat_and_snat' NAT entry with both 'external_mac' and 'logical_port' set (a distributed floating IP), its external IP/MAC are not advertised over BGP-EVPN. Only ports directly attached to the EVPN-enabled provider Logical Switch are covered, so floating IPs (a common CMS pattern, e.g. OpenStack Neutron with distributed FIPs) stay unreachable from EVPN peers.
Populate the SB Advertised_MAC_Binding for the provider Logical Switch that carries the NAT's distributed gateway port, using the NAT's 'external_ip' and 'external_mac'. ovn-controller then programs those into the EVPN advertise interface FDB and neighbor table, so the floating IPs are emitted as EVPN Type-2 MAC+IP routes on the chassis that hosts the workload. This is gated by a new 'nat' token of the provider Logical Switch 'dynamic-routing-redistribute' option, independent of the existing 'ip' token: 'ip' keeps advertising VIF and router-port addresses while 'nat' opts in to floating IPs. It mirrors the 'nat' token of the Logical_Router 'dynamic-routing-redistribute' option, which advertises the same NAT entries as EVPN Type-5 routes. The advertised set now depends on per-datapath EVPN settings, so en-advertised-mac-binding-sync recomputes on northd and lr_nat changes; toggling the redistribute tokens therefore takes effect immediately. Reported-by: Chanyeol Yoon <[email protected]> Suggested-by: Ales Musil <[email protected]> Signed-off-by: Chanyeol Yoon <[email protected]> --- .../topics/dynamic-routing/architecture.rst | 12 +++ NEWS | 6 ++ lib/ovn-util.c | 3 + lib/ovn-util.h | 3 +- northd/en-advertised-route-sync.c | 101 ++++++++++++++++-- northd/inc-proc-northd.c | 14 ++- ovn-nb.xml | 14 +++ tests/ovn-inc-proc-graph-dump.at | 3 +- tests/ovn-northd.at | 93 ++++++++++++++++ 9 files changed, 233 insertions(+), 16 deletions(-) diff --git a/Documentation/topics/dynamic-routing/architecture.rst b/Documentation/topics/dynamic-routing/architecture.rst index c517fe085..49af3c8fb 100644 --- a/Documentation/topics/dynamic-routing/architecture.rst +++ b/Documentation/topics/dynamic-routing/architecture.rst @@ -671,6 +671,18 @@ external network fabric. Each record includes: - ``ip`` --- The IP address to announce. - ``mac`` --- The MAC address to announce. +For each EVPN-enabled logical switch, ``ovn-northd`` populates the +table from the following sources, each gated by a token of the +``dynamic-routing-redistribute`` option on the logical switch: + +- With ``ip``: the MAC/IP addresses of regular VIF ports on the logical + switch, and the MAC/IP addresses of router ports peered with it. +- With ``nat``: the ``external_ip``/``external_mac`` of any distributed + ``dnat_and_snat`` NAT entry on a router whose distributed gateway + port is attached to the logical switch. This allows floating IPs + to be advertised as EVPN Type-2 routes alongside directly attached + workloads. The two tokens are independent and may be combined. + ``ovn-controller`` reads these records and installs the corresponding static FDB and neighbor entries on the appropriate kernel interfaces, making them available to the routing daemon for EVPN advertisement. diff --git a/NEWS b/NEWS index 748ae30eb..01e99689f 100644 --- a/NEWS +++ b/NEWS @@ -15,6 +15,12 @@ Post v26.03.0 * Add ECMP/multi-homing support for EVPN FDB entries. FDB entries backed by a kernel nexthop group are load-balanced via OpenFlow select groups with weighted buckets. + * EVPN: distributed dnat_and_snat NAT entries (e.g. floating IPs) + are now populated in the SB Advertised_MAC_Binding table on the + provider logical switch that carries the distributed gateway + port, so that they can be advertised as EVPN Type-2 routes. This + is gated on a new "nat" token of the Logical_Switch + "dynamic-routing-redistribute" option, independent of "ip". - Added "override-connected" option to Logical Router Static Routes to mark static routes as higher-priority than connected routes, which in turn led to changes in administrative distance for specific route types. Please see diff --git a/lib/ovn-util.c b/lib/ovn-util.c index e6143d7a9..4ad5f993c 100644 --- a/lib/ovn-util.c +++ b/lib/ovn-util.c @@ -1754,6 +1754,9 @@ parse_neigh_dynamic_redistribute(const struct smap *options) if (!strcmp(token, "ip")) { mode |= NRM_IP; } + if (!strcmp(token, "nat")) { + mode |= NRM_NAT; + } } free(tokstr); diff --git a/lib/ovn-util.h b/lib/ovn-util.h index bfca178e4..ad1dfa328 100644 --- a/lib/ovn-util.h +++ b/lib/ovn-util.h @@ -796,7 +796,8 @@ char *normalize_addr_str(const char *orig_addr); #define NEIGH_REDISTRIBUTE_MODES \ NEIGH_REDISTRIBUTE_MODE(FDB, 0) \ - NEIGH_REDISTRIBUTE_MODE(IP, 1) + NEIGH_REDISTRIBUTE_MODE(IP, 1) \ + NEIGH_REDISTRIBUTE_MODE(NAT, 2) enum neigh_redistribute_mode_bits { #define NEIGH_REDISTRIBUTE_MODE(PROTOCOL, BIT) NRM_##PROTOCOL##_BIT = BIT, diff --git a/northd/en-advertised-route-sync.c b/northd/en-advertised-route-sync.c index 4a8d13232..3c9d0bcb7 100644 --- a/northd/en-advertised-route-sync.c +++ b/northd/en-advertised-route-sync.c @@ -833,6 +833,18 @@ evpn_ip_redistribution_enabled(const struct ovn_datapath *od) return nrm_mode_IP_is_set(mode); } +static bool +evpn_nat_redistribution_enabled(const struct ovn_datapath *od) +{ + if (!od->has_evpn_vni) { + return false; + } + + enum neigh_redistribute_mode mode = + parse_neigh_dynamic_redistribute(&od->nbs->other_config); + return nrm_mode_NAT_is_set(mode); +} + static uint32_t advertised_mac_binding_get_hash(const struct sbrec_datapath_binding *dp, const struct sbrec_port_binding *sb, @@ -890,22 +902,23 @@ advertised_mac_binding_entry_destroy(struct advertised_mac_binding *e) } static void -advertised_mac_binding_add(struct hmap *map, - const struct sbrec_datapath_binding *dp, - const struct sbrec_port_binding *sb, - struct lport_addresses *addr) +advertised_mac_binding_add_with_mac(struct hmap *map, + const struct sbrec_datapath_binding *dp, + const struct sbrec_port_binding *sb, + const struct lport_addresses *addr, + const char *mac) { - if (!addr) { + if (!addr || !mac) { return; } for (size_t i = 0; i < addr->n_ipv4_addrs; i++) { if (!advertised_mac_binding_entry_find(map, dp, sb, addr->ipv4_addrs[i].addr_s, - addr->ea_s)) { + mac)) { advertised_mac_binding_entry_add(map, dp, sb, addr->ipv4_addrs[i].addr_s, - addr->ea_s); + mac); } } @@ -916,14 +929,27 @@ advertised_mac_binding_add(struct hmap *map, if (!advertised_mac_binding_entry_find(map, dp, sb, addr->ipv6_addrs[i].addr_s, - addr->ea_s)) { + mac)) { advertised_mac_binding_entry_add(map, dp, sb, addr->ipv6_addrs[i].addr_s, - addr->ea_s); + mac); } } } +static void +advertised_mac_binding_add(struct hmap *map, + const struct sbrec_datapath_binding *dp, + const struct sbrec_port_binding *sb, + struct lport_addresses *addr) +{ + if (!addr) { + return; + } + + advertised_mac_binding_add_with_mac(map, dp, sb, addr, addr->ea_s); +} + static void build_advertised_mac_binding(const struct ovn_datapath *od, struct hmap *map) { @@ -956,6 +982,57 @@ build_advertised_mac_binding(const struct ovn_datapath *od, struct hmap *map) } } +/* Advertise distributed dnat_and_snat NAT entries (e.g. floating IPs) over + * EVPN. The advertisement is attached to the provider Logical Switch that + * carries the NAT's distributed gateway port, provided that LS has NAT + * redistribution enabled via 'dynamic-routing-redistribute=nat'. */ +static void +build_advertised_mac_binding_lr(const struct ovn_datapath *od, + const struct lr_nat_table *lr_nats, + struct hmap *map) +{ + ovs_assert(od->nbr); + + const struct lr_nat_record *lrnat_rec = + lr_nat_table_find_by_uuid(lr_nats, od->nbr->header_.uuid); + if (!lrnat_rec) { + return; + } + + for (size_t i = 0; i < lrnat_rec->n_nat_entries; i++) { + const struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; + const struct nbrec_nat *nat = nat_entry->nb; + + if (!nat_entry->is_valid || !nat_entry->is_distributed || + nat_entry->type != DNAT_AND_SNAT) { + continue; + } + + if (!nat->external_mac) { + continue; + } + + if (!smap_get_bool(&nat->options, "dynamic-routing-advertise", true)) { + continue; + } + + const struct ovn_port *dgp = nat_entry->l3dgw_port; + if (!dgp || !dgp->peer || !dgp->peer->sb || !dgp->peer->od) { + continue; + } + + const struct ovn_datapath *peer_od = dgp->peer->od; + if (!peer_od->nbs || !evpn_nat_redistribution_enabled(peer_od)) { + continue; + } + + advertised_mac_binding_add_with_mac(map, peer_od->sdp->sb_dp, + dgp->peer->sb, + &nat_entry->ext_addrs, + nat->external_mac); + } +} + void * en_advertised_mac_binding_sync_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) @@ -970,6 +1047,8 @@ en_advertised_mac_binding_sync_run(struct engine_node *node, struct northd_data *northd_data = engine_get_input_data("northd", node); const struct sbrec_advertised_mac_binding_table *sbrec_adv_mb_table = EN_OVSDB_GET(engine_get_input("SB_advertised_mac_binding", node)); + struct ed_type_lr_nat_data *lr_nat_data = + engine_get_input_data("lr_nat", node); const struct engine_context *eng_ctx = engine_get_context(); struct hmap advertised_mac_binding_map = @@ -979,6 +1058,10 @@ en_advertised_mac_binding_sync_run(struct engine_node *node, HMAP_FOR_EACH (od, key_node, &northd_data->ls_datapaths.datapaths) { build_advertised_mac_binding(od, &advertised_mac_binding_map); } + HMAP_FOR_EACH (od, key_node, &northd_data->lr_datapaths.datapaths) { + build_advertised_mac_binding_lr(od, &lr_nat_data->lr_nats, + &advertised_mac_binding_map); + } struct advertised_mac_binding *e; const struct sbrec_advertised_mac_binding *sb_adv_mb; diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index a2b464411..82160d94b 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -364,11 +364,15 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, NULL); engine_add_input(&en_advertised_mac_binding_sync, &en_sb_advertised_mac_binding, NULL); - /* No need for an explicit handler for northd changes. - * We do need to access en_northd (input) data, i.e., to - * lookup OVN ports. */ - engine_add_input(&en_advertised_mac_binding_sync, &en_northd, - engine_noop_handler); + /* Recompute on northd changes: besides looking up OVN ports, this node + * reads per-datapath EVPN settings (dynamic-routing-vni and the + * dynamic-routing-redistribute 'ip'/'nat' tokens). Toggling those must + * re-evaluate which IPs/MACs and floating IPs are advertised. */ + engine_add_input(&en_advertised_mac_binding_sync, &en_northd, NULL); + /* Distributed dnat_and_snat NAT entries (e.g. floating IPs) are + * advertised via EVPN as well. Trigger a recompute on any lr_nat + * change so that those entries are picked up. */ + engine_add_input(&en_advertised_mac_binding_sync, &en_lr_nat, NULL); engine_add_input(&en_learned_route_sync, &en_sb_learned_route, learned_route_sync_sb_learned_route_change_handler); diff --git a/ovn-nb.xml b/ovn-nb.xml index 15fb1d7e8..8e2248f97 100644 --- a/ovn-nb.xml +++ b/ovn-nb.xml @@ -1032,6 +1032,20 @@ all IPs/MACs bindings that are local to the chassis. This applies to VIFs and router ports. </p> + + <p> + If <code>nat</code> is specified then the external IP/MAC of any + distributed <code>dnat_and_snat</code> NAT entry (for example a + floating IP) whose distributed gateway port is attached to this + logical switch is advertised as an EVPN Type-2 route, using the + NAT's <ref column="external_mac" table="NAT"/>. This is independent + of <code>ip</code>: enabling <code>ip</code> alone does not advertise + floating IPs, and enabling <code>nat</code> alone does not advertise + VIF or router-port addresses. This mirrors the <code>nat</code> + token of <ref column="options" key="dynamic-routing-redistribute" + table="Logical_Router"/>, which advertises the same NAT entries as + EVPN Type-5 routes. + </p> </column> </group> diff --git a/tests/ovn-inc-proc-graph-dump.at b/tests/ovn-inc-proc-graph-dump.at index 3750339d0..c37845188 100644 --- a/tests/ovn-inc-proc-graph-dump.at +++ b/tests/ovn-inc-proc-graph-dump.at @@ -229,7 +229,8 @@ digraph "Incremental-Processing-Engine" { advertised_mac_binding_sync [[style=filled, shape=box, fillcolor=white, label="advertised_mac_binding_sync"]]; SB_port_binding -> advertised_mac_binding_sync [[label=""]]; SB_advertised_mac_binding -> advertised_mac_binding_sync [[label=""]]; - northd -> advertised_mac_binding_sync [[label="engine_noop_handler"]]; + northd -> advertised_mac_binding_sync [[label=""]]; + lr_nat -> advertised_mac_binding_sync [[label=""]]; northd_output [[style=filled, shape=box, fillcolor=white, label="northd_output"]]; acl_id -> northd_output [[label="northd_output_acl_id_handler"]]; sync_from_sb -> northd_output [[label=""]]; diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 7f4a88d4e..1aa483111 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -20264,6 +20264,99 @@ OVN_CLEANUP_NORTHD AT_CLEANUP ]) +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([LR EVPN Advertised_MAC_Binding sync for distributed NAT]) +ovn_start + +check ovn-sbctl chassis-add ch geneve 127.0.0.1 + +AS_BOX([Build topology: tenant LS, LR with DGP, EVPN-enabled provider LS]) +check ovn-nbctl ls-add ls-tenant \ + -- lsp-add ls-tenant vm1 \ + -- lsp-set-addresses vm1 "00:00:00:00:02:00 10.0.0.10" +check ovn-nbctl ls-add ls-evpn \ + -- set logical_switch ls-evpn other_config:dynamic-routing-vni=10 \ + -- set logical_switch ls-evpn other_config:dynamic-routing-redistribute=ip +dnl A provider logical switch has a localnet port; without it the DGP's +dnl peer gets a chassisredirect port and the NAT is not considered +dnl distributed (so floating IPs would never be advertised). +check ovn-nbctl lsp-add ls-evpn ln-evpn \ + -- lsp-set-type ln-evpn localnet \ + -- lsp-set-addresses ln-evpn unknown \ + -- lsp-set-options ln-evpn network_name=phys +check ovn-nbctl lr-add lr \ + -- lrp-add lr lrp-int 00:00:00:00:01:01 10.0.0.1/24 \ + -- lrp-add lr lrp-ext 00:00:00:00:01:02 172.16.0.1/24 \ + -- lrp-set-gateway-chassis lrp-ext ch \ + -- lsp-add ls-tenant ls-tenant-lrp \ + -- lsp-set-type ls-tenant-lrp router \ + -- lsp-set-addresses ls-tenant-lrp 00:00:00:00:01:01 \ + -- lsp-set-options ls-tenant-lrp router-port=lrp-int \ + -- lsp-add ls-evpn ls-evpn-lrp \ + -- lsp-set-type ls-evpn-lrp router \ + -- lsp-set-addresses ls-evpn-lrp 00:00:00:00:01:02 \ + -- lsp-set-options ls-evpn-lrp router-port=lrp-ext +check ovn-nbctl --wait=sb sync + +ls_evpn_uuid=$(fetch_column Datapath_Binding _uuid external_ids:name=ls-evpn) + +AS_BOX([Baseline: LRP MAC/IP advertised, no NAT yet]) +check_row_count Advertised_MAC_Binding 1 ip=172.16.0.1 datapath=$ls_evpn_uuid mac='00\:00\:00\:00\:01\:02' +check_row_count Advertised_MAC_Binding 1 + +AS_BOX([Non-distributed NATs are not advertised]) +check ovn-nbctl --wait=sb lr-nat-add lr snat 172.16.0.50 10.0.0.10 +check ovn-nbctl --wait=sb lr-nat-add lr dnat 172.16.0.51 10.0.0.10 +check_row_count Advertised_MAC_Binding 1 +check ovn-nbctl --wait=sb lr-nat-del lr snat 172.16.0.50 +check ovn-nbctl --wait=sb lr-nat-del lr dnat 172.16.0.51 + +AS_BOX([dnat_and_snat without external_mac is not advertised]) +check ovn-nbctl --wait=sb lr-nat-add lr dnat_and_snat 172.16.0.52 10.0.0.10 +check_row_count Advertised_MAC_Binding 1 +check ovn-nbctl --wait=sb lr-nat-del lr dnat_and_snat 172.16.0.52 + +AS_BOX([Distributed FIP is not advertised with 'ip' alone (needs 'nat')]) +check ovn-nbctl --wait=sb lr-nat-add lr dnat_and_snat 172.16.0.100 10.0.0.10 vm1 00:11:22:33:44:55 +check_row_count Advertised_MAC_Binding 0 ip=172.16.0.100 +check_row_count Advertised_MAC_Binding 1 + +AS_BOX([Adding the 'nat' token advertises the FIP]) +check ovn-nbctl --wait=sb set logical_switch ls-evpn other_config:dynamic-routing-redistribute=ip,nat +check_row_count Advertised_MAC_Binding 1 ip=172.16.0.100 datapath=$ls_evpn_uuid mac='00\:11\:22\:33\:44\:55' +check_row_count Advertised_MAC_Binding 2 + +AS_BOX([Adding a second FIP yields a second advertised entry]) +check ovn-nbctl --wait=sb lsp-add ls-tenant vm2 \ + -- lsp-set-addresses vm2 "00:00:00:00:02:01 10.0.0.11" +check ovn-nbctl --wait=sb lr-nat-add lr dnat_and_snat 172.16.0.101 10.0.0.11 vm2 00:11:22:33:44:66 +check_row_count Advertised_MAC_Binding 1 ip=172.16.0.101 datapath=$ls_evpn_uuid mac='00\:11\:22\:33\:44\:66' +check_row_count Advertised_MAC_Binding 3 + +AS_BOX([Deleting a NAT removes its advertised entry]) +check ovn-nbctl --wait=sb lr-nat-del lr dnat_and_snat 172.16.0.100 +check_row_count Advertised_MAC_Binding 0 ip=172.16.0.100 +check_row_count Advertised_MAC_Binding 2 + +AS_BOX(['nat' and 'ip' are independent: dropping 'nat' keeps the LRP entry]) +check ovn-nbctl --wait=sb set logical_switch ls-evpn other_config:dynamic-routing-redistribute=ip +check_row_count Advertised_MAC_Binding 0 ip=172.16.0.101 +check_row_count Advertised_MAC_Binding 1 ip=172.16.0.1 datapath=$ls_evpn_uuid mac='00\:00\:00\:00\:01\:02' +check_row_count Advertised_MAC_Binding 1 + +AS_BOX([Disabling EVPN on the provider LS removes all advertisements]) +check ovn-nbctl --wait=sb remove logical_switch ls-evpn other_config dynamic-routing-redistribute +check_row_count Advertised_MAC_Binding 0 + +AS_BOX([Re-enabling redistribute=ip,nat restores both LRP and FIP]) +check ovn-nbctl --wait=sb set logical_switch ls-evpn other_config:dynamic-routing-redistribute=ip,nat +check_row_count Advertised_MAC_Binding 1 ip=172.16.0.101 datapath=$ls_evpn_uuid mac='00\:11\:22\:33\:44\:66' +check_row_count Advertised_MAC_Binding 2 + +OVN_CLEANUP_NORTHD +AT_CLEANUP +]) + OVN_FOR_EACH_NORTHD_NO_HV([ AT_SETUP([IGMP northd crash]) ovn_start -- 2.54.0 (Apple Git-156) _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
