For EVPN L3 (type-5) routes, the advertised L3 VNI may differ from the destination logical switch's dynamic-routing-vni. Previously OVN always used the switch's VNI when encapsulating traffic towards a learned route, so it could not honor a per-route VNI (e.g. symmetric IRB).
FRR installs such routes with the L3 VNI attached as lightweight-tunnel (LWT) encapsulation. ovn-controller reads this VNI from the route and stores it in the SB Learned_Route's external_ids:vni. ovn-northd reads it into the parsed route and, in the logical router IP routing stage, loads it into a dedicated register (reg16, \"evpn_l3_vni\") with a validity bit. reg16 is outside the reg0..reg9 range, so it does not alias the routing registers and survives the router-to-switch transition. ovn-controller's EVPN tunnel egress then uses that register to override the tunnel VNI, falling back to the switch's dynamic-routing-vni when it is not set. reg16 requires the running OVS to support 32 registers, so northd only emits the write when all chassis advertise the feature and ovn-controller only installs the override flows when the local OVS supports it. This depends on the OVS route-table library change that parses the LWT tunnel id (VNI) from kernel routes, so that ovn-controller can read it. Assisted-by: Claude Opus 4.8, Cursor Signed-off-by: Han Zhou <[email protected]> --- .../topics/dynamic-routing/architecture.rst | 19 +++ controller/physical.c | 137 +++++++++++++----- controller/route-exchange-netlink.c | 2 + controller/route-exchange-netlink.h | 5 + controller/route-exchange.c | 40 ++++- lib/logical-fields.c | 8 + northd/en-learned-route-sync.c | 15 +- northd/en-lflow.c | 3 +- northd/northd.c | 73 ++++++++-- northd/northd.h | 9 +- ovn-sb.xml | 10 ++ tests/ovn-northd.at | 46 ++++++ tests/system-ovn.at | 18 +++ 13 files changed, 327 insertions(+), 58 deletions(-) diff --git a/Documentation/topics/dynamic-routing/architecture.rst b/Documentation/topics/dynamic-routing/architecture.rst index 0ea1755626de..906b0d6ac2e8 100644 --- a/Documentation/topics/dynamic-routing/architecture.rst +++ b/Documentation/topics/dynamic-routing/architecture.rst @@ -364,6 +364,25 @@ For each qualifying route, ``ovn-controller`` creates a ``Learned_Route`` record in the Southbound database containing the datapath, logical port, IP prefix, and nexthop. +EVPN L3 VNI (Type-5 Routes) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +An EVPN type-5 (IP prefix) route may be advertised with an L3 VNI that +differs from the destination logical switch's ``dynamic-routing-vni``. When +the routing daemon installs such a route it attaches the L3 VNI to the kernel +route as lightweight tunnel (LWT) encapsulation metadata (for example +``ip route add ... encap ip id <vni> ...``, as programmed by FRR for +symmetric-IRB type-5 routes). ``ovn-controller`` reads this VNI from the +route's LWT encapsulation and stores it in the ``Learned_Route`` record's +``external_ids:vni`` key. + +``ovn-northd`` propagates this VNI into the logical router IP routing pipeline +so that, when a packet matches the learned route, the EVPN tunnel egress +encapsulates it with the route's VNI instead of the logical switch's +``dynamic-routing-vni``. If a learned route carries no VNI (the kernel route +has no LWT encapsulation), OVN falls back to the logical switch's +``dynamic-routing-vni``. + Flow Generation by ovn-northd ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/controller/physical.c b/controller/physical.c index 89ff2785e473..8e33ec5cf00e 100644 --- a/controller/physical.c +++ b/controller/physical.c @@ -46,6 +46,7 @@ #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "ovn/actions.h" +#include "ovn/features.h" #include "if-status.h" #include "physical.h" #include "pinctrl.h" @@ -85,6 +86,18 @@ static struct uuid *hc_uuid = NULL; #define CHASSIS_MAC_TO_ROUTER_MAC_CONJID 100 +/* Register (reg16) used to carry the EVPN L3 VNI of a learned (type-5) route + * from the logical router pipeline into the EVPN tunnel egress of the peer + * logical switch pipeline. Must match "evpn_l3_vni" (REG_EVPN_L3_VNI) in + * northd.c. It lives outside the generic logical register range + * (reg0..reg9), so it is not zeroed on the router-to-switch transition and + * survives to the egress without special handling. reg16 requires OVS to + * support 32 registers (OVS_REG32_SUPPORT); the flows using it are only + * installed when that feature is available. Bit 31 marks the VNI as + * valid. */ +#define OVN_EVPN_VNI_LOG_REG 16 +#define OVN_EVPN_VNI_VALID_BIT 31 + void physical_register_ovs_idl(struct ovsdb_idl *ovs_idl) { @@ -3388,6 +3401,72 @@ evpn_local_ip_map_destroy(struct evpn_local_ip_map *map) hmap_destroy(&map->vni_ip6); } +/* Adds the pair of EVPN tunnel egress flows (a normal flow at 'base_prio' and + * a loopback variant at 'base_prio + 5') for 'binding' to + * OFTABLE_REMOTE_VTEP_OUTPUT. All flows set tun_src/tun_dst from the binding. + * + * When 'vni_from_reg' is false the tunnel VNI is the binding's own VNI. When + * true, the flows additionally match on the EVPN L3 VNI validity bit and take + * the VNI from reg16 instead (used for a learned type-5 route advertised with + * a VNI different from the switch's dynamic-routing-vni). */ +static void +add_evpn_binding_egress_flows(const struct evpn_binding *binding, + ovs_be32 local_ip4, + const struct in6_addr *local_ip6, + bool vni_from_reg, uint16_t base_prio, + struct ofpbuf *ofpacts, struct match *match, + struct ovn_desired_flow_table *flow_table) +{ + ofpbuf_clear(ofpacts); + match_init_catchall(match); + match_outport_dp_and_port_keys(match, binding->dp_key, + binding->binding_key); + if (vni_from_reg) { + match_set_reg_masked(match, OVN_EVPN_VNI_LOG_REG, + 1u << OVN_EVPN_VNI_VALID_BIT, + 1u << OVN_EVPN_VNI_VALID_BIT); + } + + if (local_ip4) { + put_load_bytes(&local_ip4, sizeof local_ip4, MFF_TUN_SRC, 0, 32, + ofpacts); + ovs_be32 ip4 = in6_addr_get_mapped_ipv4(&binding->remote_ip); + put_load_bytes(&ip4, sizeof ip4, MFF_TUN_DST, 0, 32, ofpacts); + } else { + put_load_bytes(local_ip6, sizeof *local_ip6, MFF_TUN_IPV6_SRC, + 0, 128, ofpacts); + put_load_bytes(&binding->remote_ip, sizeof binding->remote_ip, + MFF_TUN_IPV6_DST, 0, 128, ofpacts); + } + + if (vni_from_reg) { + put_move(MFF_REG0 + OVN_EVPN_VNI_LOG_REG, 0, MFF_TUN_ID, 0, 24, + ofpacts); + } else { + put_load(binding->vni, MFF_TUN_ID, 0, 24, ofpacts); + } + + size_t ofpacts_size = ofpacts->size; + ofpact_put_OUTPUT(ofpacts)->port = binding->tunnel_ofport; + + ofctrl_add_flow(flow_table, OFTABLE_REMOTE_VTEP_OUTPUT, base_prio, + binding->flow_uuid.parts[0], + match, ofpacts, &binding->flow_uuid); + + /* Loopback variant: match on the LOOPBACK flag and set in_port to none, + * otherwise the hairpin traffic would be rejected by ovs. */ + match_set_reg_masked(match, MFF_LOG_FLAGS - MFF_REG0, + MLF_ALLOW_LOOPBACK, MLF_ALLOW_LOOPBACK); + + ofpbuf_truncate(ofpacts, ofpacts_size); + put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts); + ofpact_put_OUTPUT(ofpacts)->port = binding->tunnel_ofport; + + ofctrl_add_flow(flow_table, OFTABLE_REMOTE_VTEP_OUTPUT, base_prio + 5, + binding->flow_uuid.parts[0], + match, ofpacts, &binding->flow_uuid); +} + static void physical_consider_evpn_binding(const struct evpn_binding *binding, const struct evpn_local_ip_map *vni_ip_map, @@ -3435,46 +3514,28 @@ physical_consider_evpn_binding(const struct evpn_binding *binding, binding->flow_uuid.parts[0], match, ofpacts, &binding->flow_uuid); - /* Egress flows. */ - ofpbuf_clear(ofpacts); - match_init_catchall(match); - - match_outport_dp_and_port_keys(match, binding->dp_key, - binding->binding_key); + /* Egress flows using the binding's own VNI. */ + add_evpn_binding_egress_flows(binding, local_ip4, local_ip6, false, 50, + ofpacts, match, flow_table); - if (local_ip4) { - put_load_bytes(&local_ip4, sizeof local_ip4, MFF_TUN_SRC, 0, 32, - ofpacts); - ovs_be32 ip4 = in6_addr_get_mapped_ipv4(&binding->remote_ip); - put_load_bytes(&ip4, sizeof ip4, MFF_TUN_DST, 0, 32, ofpacts); - } else { - put_load_bytes(local_ip6, sizeof *local_ip6, MFF_TUN_IPV6_SRC, - 0, 128, ofpacts); - put_load_bytes(&binding->remote_ip, sizeof binding->remote_ip, - MFF_TUN_IPV6_DST, 0, 128, ofpacts); + /* EVPN L3 VNI override egress flows. + * + * A learned (type-5) route may need to egress with an L3 VNI that differs + * from this binding's VNI (the destination logical switch's + * dynamic-routing-vni). The logical router pipeline stores the desired + * VNI in reg16 (REG_EVPN_L3_VNI) with the validity bit set; reg16 survives + * the LR->LS transition (it is outside the reg0..reg9 range zeroed there). + * The higher-priority flows added below match on that validity bit and + * take the VNI from the register; tun_src/tun_dst still come from the + * binding (the remote VTEP is the route's nexthop). + * + * reg16 is only usable when the running OVS supports 32 registers. When + * it does not, northd never sets the register, so we skip these flows + * (also avoiding a reg16 reference that ovs-vswitchd would reject). */ + if (ovs_feature_is_supported(OVS_REG32_SUPPORT)) { + add_evpn_binding_egress_flows(binding, local_ip4, local_ip6, true, 60, + ofpacts, match, flow_table); } - put_load(binding->vni, MFF_TUN_ID, 0, 24, ofpacts); - - size_t ofpacts_size = ofpacts->size; - ofpact_put_OUTPUT(ofpacts)->port = binding->tunnel_ofport; - - ofctrl_add_flow(flow_table, OFTABLE_REMOTE_VTEP_OUTPUT, 50, - binding->flow_uuid.parts[0], - match, ofpacts, &binding->flow_uuid); - - /* Add flow that will match on LOOPBACK flag, in that case set - * in_port to none otherwise the hairpin traffic would be rejected - * by ovs. */ - match_set_reg_masked(match, MFF_LOG_FLAGS - MFF_REG0, - MLF_ALLOW_LOOPBACK, MLF_ALLOW_LOOPBACK); - - ofpbuf_truncate(ofpacts, ofpacts_size); - put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts); - ofpact_put_OUTPUT(ofpacts)->port = binding->tunnel_ofport; - - ofctrl_add_flow(flow_table, OFTABLE_REMOTE_VTEP_OUTPUT, 55, - binding->flow_uuid.parts[0], - match, ofpacts, &binding->flow_uuid); /* Dynamic FDB learn flows. */ ofpbuf_clear(ofpacts); diff --git a/controller/route-exchange-netlink.c b/controller/route-exchange-netlink.c index 8f1615c4ecaf..00b9ea4dd2b1 100644 --- a/controller/route-exchange-netlink.c +++ b/controller/route-exchange-netlink.c @@ -248,6 +248,8 @@ handle_route_msg(const struct route_table_msg *msg, .prefix = rd->rta_dst, .plen = rd->rtm_dst_len, .nexthop = nexthop->addr, + .vni_present = rd->vni_present, + .vni = rd->vni, }; memcpy(rr.ifname, nexthop->ifname, IFNAMSIZ); rr.ifname[IFNAMSIZ] = '\0'; diff --git a/controller/route-exchange-netlink.h b/controller/route-exchange-netlink.h index c137b5119b36..2e00daf25646 100644 --- a/controller/route-exchange-netlink.h +++ b/controller/route-exchange-netlink.h @@ -18,6 +18,7 @@ #ifndef ROUTE_EXCHANGE_NETLINK_H #define ROUTE_EXCHANGE_NETLINK_H 1 +#include <stdbool.h> #include <stdint.h> #include <linux/rtnetlink.h> #include <netinet/in.h> @@ -45,6 +46,10 @@ struct re_nl_received_route_node { struct in6_addr nexthop; /* Adding 1 to this to be sure we actually have a terminating '\0' */ char ifname[IFNAMSIZ + 1]; + /* EVPN L3 VNI learned from the route's LWT encapsulation. Valid only if + * 'vni_present'. */ + bool vni_present; + uint32_t vni; }; int re_nl_create_vrf(const char *ifname, uint32_t table_id); diff --git a/controller/route-exchange.c b/controller/route-exchange.c index b86eb43bf259..ad0a7b3a6da0 100644 --- a/controller/route-exchange.c +++ b/controller/route-exchange.c @@ -18,6 +18,8 @@ #include <config.h> #include <errno.h> +#include <inttypes.h> +#include <limits.h> #include <net/if.h> #include <stdbool.h> @@ -104,11 +106,31 @@ route_add_entry(struct hmap *routes, hmap_insert(routes, &route_e->hmap_node, hash); } +/* Returns true if the VNI stored in 'sb_route' (external_ids:vni) matches the + * '(vni_present, vni)' pair learned from the system. A learned route without + * a VNI matches only an SB route without a VNI, so that a route which gains or + * loses its VNI is detected as changed. */ +static bool +learned_route_vni_matches(const struct sbrec_learned_route *sb_route, + bool vni_present, uint32_t vni) +{ + /* A valid VNI is at most 24 bits, so UINT_MAX is a safe marker for a + * missing (or unparseable) external_ids:vni. */ + unsigned int existing_vni = + smap_get_uint(&sb_route->external_ids, "vni", UINT_MAX); + + if (!vni_present) { + return existing_vni == UINT_MAX; + } + return existing_vni == vni; +} + static struct route_entry * route_lookup(struct hmap *route_map, const struct sbrec_datapath_binding *sb_db, const struct sbrec_port_binding *logical_port, - const char *ip_prefix, const char *nexthop) + const char *ip_prefix, const char *nexthop, + bool vni_present, uint32_t vni) { struct route_entry *route_e; uint32_t hash; @@ -130,6 +152,9 @@ route_lookup(struct hmap *route_map, if (strcmp(route_e->sb_route->nexthop, nexthop)) { continue; } + if (!learned_route_vni_matches(route_e->sb_route, vni_present, vni)) { + continue; + } return route_e; } @@ -196,7 +221,9 @@ sb_sync_learned_routes(const struct vector *learned_routes, } route_e = route_lookup(&sync_routes, datapath, - logical_port, ip_prefix, nexthop); + logical_port, ip_prefix, nexthop, + learned_route->vni_present, + learned_route->vni); if (route_e) { route_e->stale = false; } else { @@ -209,6 +236,15 @@ sb_sync_learned_routes(const struct vector *learned_routes, sbrec_learned_route_set_logical_port(sb_route, logical_port); sbrec_learned_route_set_ip_prefix(sb_route, ip_prefix); sbrec_learned_route_set_nexthop(sb_route, nexthop); + if (learned_route->vni_present) { + struct smap external_ids = + SMAP_INITIALIZER(&external_ids); + smap_add_format(&external_ids, "vni", "%"PRIu32, + learned_route->vni); + sbrec_learned_route_set_external_ids(sb_route, + &external_ids); + smap_destroy(&external_ids); + } route_add_entry(&sync_routes, sb_route, false); } diff --git a/lib/logical-fields.c b/lib/logical-fields.c index 35d9a2e229c6..bfa8e745f576 100644 --- a/lib/logical-fields.c +++ b/lib/logical-fields.c @@ -135,6 +135,14 @@ ovn_init_symtab(struct shash *symtab) free(name); } + /* EVPN L3 VNI register. Carries the L3 VNI of a learned (type-5) route + * from the logical router pipeline to the EVPN tunnel egress in the peer + * logical switch pipeline. Backed by reg16, which lives outside the + * generic logical register range (reg0..reg9) so it survives the + * router-to-switch transition, and is only used when the running OVS + * supports 32 registers (see the OVS_REG32_SUPPORT feature). */ + expr_symtab_add_field(symtab, "evpn_l3_vni", MFF_REG16, NULL, false); + /* Flags used in logical to physical transformation. */ expr_symtab_add_field(symtab, "flags", MFF_LOG_FLAGS, NULL, false); char flags_str[16]; diff --git a/northd/en-learned-route-sync.c b/northd/en-learned-route-sync.c index cbd516b6874b..ffba8390a858 100644 --- a/northd/en-learned-route-sync.c +++ b/northd/en-learned-route-sync.c @@ -15,6 +15,7 @@ */ #include <config.h> +#include <limits.h> #include <stdbool.h> #include "openvswitch/vlog.h" @@ -199,10 +200,20 @@ parse_route_from_sbrec_route(struct hmap *parsed_routes_out, return NULL; } + /* An EVPN (type-5) route may be learned with an L3 VNI that differs from + * the local switch's dynamic-routing-vni. When present, it is stored in + * the SB Learned_Route's external_ids:vni. A valid VNI is at most 24 + * bits, so UINT_MAX marks a missing (or, an unexpectedly malformed) + * value. */ + unsigned int raw_vni = smap_get_uint(&route->external_ids, "vni", + UINT_MAX); + bool vni_present = raw_vni != UINT_MAX; + uint32_t vni = vni_present ? raw_vni : 0; + return parsed_route_add(od, nexthop, &prefix, plen, false, lrp_addr_s, out_port, 0, false, false, false, NULL, - ROUTE_SOURCE_LEARNED, true, &route->header_, NULL, - parsed_routes_out); + ROUTE_SOURCE_LEARNED, vni_present, vni, true, + &route->header_, NULL, parsed_routes_out); } static void diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 8cb987777714..f5b9295d934b 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -315,7 +315,8 @@ lflow_group_ecmp_route_change_handler(struct engine_node *node, lflow_ref_unlink_lflows(route_node->lflow_ref); build_route_data_flows_for_lrouter( route_node->od, lflow_data->lflow_table, - route_node, lflow_input.bfd_ports); + route_node, lflow_input.bfd_ports, + lflow_input.features->reg32); bool handled = lflow_ref_sync_lflows( route_node->lflow_ref, lflow_data->lflow_table, diff --git a/northd/northd.c b/northd/northd.c index 22108dd823f6..c2af425efe4f 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -231,6 +231,20 @@ BUILD_ASSERT_DECL(ACL_OBS_STAGE_MAX < (1 << 2)); #define REG_POLICY_CHAIN_ID "reg9[16..31]" #define REG_ROUTE_TABLE_ID "reg7" +/* Register carrying the EVPN L3 VNI of a learned (type-5) route towards the + * EVPN tunnel egress in the peer logical switch pipeline. This is used when + * the route was advertised with an L3 VNI different from the destination + * logical switch's dynamic-routing-vni. It is backed by reg16 (see + * "evpn_l3_vni" in lib/logical-fields.c), which lives outside the generic + * logical register range (reg0..reg9), so it does not alias the routing + * registers (xxreg0/xxreg1) and survives the LR->LS transition without any + * special handling. reg16 is only available when the running OVS supports 32 + * registers, so these flows are only emitted when all chassis advertise the + * OVS_REG32_SUPPORT feature; otherwise OVN falls back to the logical switch's + * dynamic-routing-vni. Bit 31 marks the VNI as valid. */ +#define REG_EVPN_L3_VNI "evpn_l3_vni[0..23]" +#define REG_EVPN_L3_VNI_VALID "evpn_l3_vni[31]" + /* Registers used for pasing observability information for switches: * domain and point ID. */ #define REG_OBS_POINT_ID_NEW "reg3" @@ -12353,6 +12367,11 @@ parsed_route_lookup(struct hmap *routes, size_t hash, continue; } + if (pr->vni_present != new_pr->vni_present || + pr->vni != new_pr->vni) { + continue; + } + return pr; } @@ -12373,6 +12392,8 @@ parsed_route_init(const struct ovn_datapath *od, bool override_connected, const struct sset *ecmp_selection_fields, enum route_source source, + bool vni_present, + uint32_t vni, bool dynamic_routing_advertise, const struct ovn_port *tracked_port, const struct ovsdb_idl_row *source_hint) @@ -12384,6 +12405,8 @@ parsed_route_init(const struct ovn_datapath *od, new_pr->plen = plen; new_pr->nexthop = nexthop; new_pr->route_table_id = route_table_id; + new_pr->vni_present = vni_present; + new_pr->vni = vni; new_pr->is_src_route = is_src_route; new_pr->od = od; new_pr->ecmp_symmetric_reply = ecmp_symmetric_reply; @@ -12418,8 +12441,8 @@ parsed_route_clone(const struct parsed_route *pr) pr->od, nexthop, pr->prefix, pr->plen, pr->is_discard_route, pr->lrp_addr_s, pr->out_port, pr->route_table_id, pr->is_src_route, pr->ecmp_symmetric_reply, pr->override_connected, - &pr->ecmp_selection_fields, pr->source, pr->dynamic_routing_advertise, - pr->tracked_port, pr->source_hint); + &pr->ecmp_selection_fields, pr->source, pr->vni_present, pr->vni, + pr->dynamic_routing_advertise, pr->tracked_port, pr->source_hint); new_pr->hash = pr->hash; return new_pr; @@ -12481,6 +12504,8 @@ parsed_route_add(const struct ovn_datapath *od, bool override_connected, const struct sset *ecmp_selection_fields, enum route_source source, + bool vni_present, + uint32_t vni, bool dynamic_routing_advertise, const struct ovsdb_idl_row *source_hint, const struct ovn_port *tracked_port, @@ -12492,7 +12517,8 @@ parsed_route_add(const struct ovn_datapath *od, lrp_addr_s, out_port, route_table_id, is_src_route, ecmp_symmetric_reply, override_connected, ecmp_selection_fields, - source, dynamic_routing_advertise, + source, vni_present, vni, + dynamic_routing_advertise, tracked_port, source_hint); new_pr->hash = route_hash(new_pr); @@ -12643,6 +12669,7 @@ parsed_routes_add_static(const struct ovn_datapath *od, ecmp_symmetric_reply, override_connected, &ecmp_selection_fields, source, + false, 0, dynamic_routing_advertise, &route->header_, NULL, routes); sset_destroy(&ecmp_selection_fields); @@ -12662,7 +12689,7 @@ parsed_routes_add_connected(const struct ovn_datapath *od, parsed_route_add(od, NULL, &prefix, addr->plen, false, addr->addr_s, op, 0, false, false, false, NULL, ROUTE_SOURCE_CONNECTED, - true, &op->nbrp->header_, NULL, routes); + false, 0, true, &op->nbrp->header_, NULL, routes); } for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { @@ -12670,7 +12697,7 @@ parsed_routes_add_connected(const struct ovn_datapath *od, parsed_route_add(od, NULL, &addr->network, addr->plen, false, addr->addr_s, op, 0, false, false, false, - NULL, ROUTE_SOURCE_CONNECTED, true, + NULL, ROUTE_SOURCE_CONNECTED, false, 0, true, &op->nbrp->header_, NULL, routes); } } @@ -13088,7 +13115,8 @@ add_route(struct lflow_table *lflows, const struct ovn_datapath *od, const struct ovsdb_idl_row *stage_hint, bool is_discard_route, enum route_source source, struct lflow_ref *lflow_ref, bool is_ipv4_prefix, bool is_ipv4_nexthop, - bool override_connected) + bool override_connected, + bool evpn_vni_present, uint32_t evpn_vni, bool reg32_supported) { struct ds match = DS_EMPTY_INITIALIZER; uint16_t priority = calc_priority(plen, source, override_connected, @@ -13123,6 +13151,16 @@ add_route(struct lflow_table *lflows, const struct ovn_datapath *od, ds_put_cstr(&actions, debug_drop_action()); } else { ds_put_format(&common_actions, REG_ECMP_GROUP_ID" = 0; "); + if (evpn_vni_present && reg32_supported) { + /* Carry the route's EVPN L3 VNI towards the EVPN tunnel egress so + * that it is used (instead of the destination logical switch's + * dynamic-routing-vni) when encapsulating this traffic. This + * requires reg16, hence the OVS 32-register capability; without + * it we fall back to the switch's dynamic-routing-vni. */ + ds_put_format(&common_actions, + REG_EVPN_L3_VNI" = %"PRIu32"; " + REG_EVPN_L3_VNI_VALID" = 1; ", evpn_vni); + } if (gateway) { ds_put_format(&common_actions, "%s = ", is_ipv4_nexthop ? REG_NEXT_HOP_IPV4 : @@ -13168,7 +13206,8 @@ add_route(struct lflow_table *lflows, const struct ovn_datapath *od, static void build_route_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct parsed_route *route, - const struct sset *bfd_ports, struct lflow_ref *lflow_ref) + const struct sset *bfd_ports, struct lflow_ref *lflow_ref, + bool reg32_supported) { bool is_ipv4_prefix = IN6_IS_ADDR_V4MAPPED(&route->prefix); bool is_ipv4_nexthop = route->nexthop @@ -13182,7 +13221,8 @@ build_route_flow(struct lflow_table *lflows, const struct ovn_datapath *od, route->route_table_id, bfd_ports, route->source_hint, route->is_discard_route, route->source, lflow_ref, - is_ipv4_prefix, is_ipv4_nexthop, route->override_connected); + is_ipv4_prefix, is_ipv4_nexthop, route->override_connected, + route->vni_present, route->vni, reg32_supported); free(prefix_s); } @@ -15251,7 +15291,7 @@ void build_route_data_flows_for_lrouter( const struct ovn_datapath *od, struct lflow_table *lflows, const struct group_ecmp_datapath *route_node, - const struct sset *bfd_ports) + const struct sset *bfd_ports, bool reg32_supported) { struct ecmp_groups_node *group; HMAP_FOR_EACH (group, hmap_node, &route_node->ecmp_groups) { @@ -15274,7 +15314,7 @@ build_route_data_flows_for_lrouter( const struct unique_routes_node *ur; HMAP_FOR_EACH (ur, hmap_node, &route_node->unique_routes) { build_route_flow(lflows, od, ur->route, bfd_ports, - route_node->lflow_ref); + route_node->lflow_ref, reg32_supported); } } @@ -15282,7 +15322,8 @@ static void build_route_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, const struct group_ecmp_route_data *route_data, - struct simap *route_tables, const struct sset *bfd_ports) + struct simap *route_tables, const struct sset *bfd_ports, + bool reg32_supported) { ovs_assert(od->nbr); build_default_route_flows_for_lrouter(od, lflows, route_tables); @@ -15292,7 +15333,8 @@ build_route_flows_for_lrouter( if (!datapath_node) { return; } - build_route_data_flows_for_lrouter(od, lflows, datapath_node, bfd_ports); + build_route_data_flows_for_lrouter(od, lflows, datapath_node, bfd_ports, + reg32_supported); } static void @@ -18919,7 +18961,8 @@ build_routable_flows_for_router_port( bfd_ports, &router_port->nbrp->header_, false, ROUTE_SOURCE_CONNECTED, lrp->stateful_lflow_ref, - true, is_ipv4_nexthop ? true : false, false); + true, is_ipv4_nexthop ? true : false, false, + false, 0, false); } } } @@ -19708,7 +19751,8 @@ build_lswitch_and_lrouter_iterate_by_lr(struct ovn_datapath *od, od->datapath_lflows); build_route_flows_for_lrouter(od, lsi->lflows, lsi->route_data, lsi->route_tables, - lsi->bfd_ports); + lsi->bfd_ports, + lsi->features->reg32); build_mcast_lookup_flows_for_lrouter(od, lsi->lflows, &lsi->match, od->datapath_lflows); build_ingress_policy_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports, @@ -20404,6 +20448,7 @@ lflow_handle_northd_lr_changes(struct ovsdb_idl_txn *ovnsb_txn, .route_data = lflow_input->route_data, .route_tables = lflow_input->route_tables, .route_policies = lflow_input->route_policies, + .features = lflow_input->features, .match = DS_EMPTY_INITIALIZER, .actions = DS_EMPTY_INITIALIZER, }; diff --git a/northd/northd.h b/northd/northd.h index d27f519d6e33..b415b18e3753 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -878,6 +878,11 @@ struct parsed_route { const struct ovn_port *out_port; const struct ovn_port *tracked_port; /* May be NULL. */ bool dynamic_routing_advertise; + /* EVPN L3 VNI for this route (e.g. a type-5 route learned with a + * different VNI than the local switch's dynamic-routing-vni). Valid only + * if 'vni_present'. */ + bool vni_present; + uint32_t vni; }; struct parsed_route *parsed_route_clone(const struct parsed_route *); @@ -901,6 +906,8 @@ struct parsed_route *parsed_route_add( bool override_connected, const struct sset *ecmp_selection_fields, enum route_source source, + bool vni_present, + uint32_t vni, bool dynamic_routing_advertise, const struct ovsdb_idl_row *source_hint, const struct ovn_port *tracked_port, @@ -984,7 +991,7 @@ void lflow_reset_northd_refs(struct lflow_input *); void build_route_data_flows_for_lrouter( const struct ovn_datapath *od, struct lflow_table *lflows, const struct group_ecmp_datapath *route_node, - const struct sset *bfd_ports); + const struct sset *bfd_ports, bool reg32_supported); bool lflow_handle_northd_lr_changes(struct ovsdb_idl_txn *ovnsh_txn, struct tracked_dps *, diff --git a/ovn-sb.xml b/ovn-sb.xml index e403eb36031f..a8834963f6a8 100644 --- a/ovn-sb.xml +++ b/ovn-sb.xml @@ -5518,6 +5518,16 @@ tcp.flags = RST; This is the nexthop ip we learned from outside of OVN. </column> + <column name="external_ids" key="vni"> + For routes learned on an EVPN (type-5) enabled datapath, this is the + L3 VNI associated with the route, as learned from the route's + lightweight tunnel (LWT) encapsulation. When set, + <code>ovn-controller</code> uses this VNI (rather than the logical + switch's <code>dynamic-routing-vni</code>) when encapsulating traffic + that egresses towards this route's nexthop. When not set, the local + <code>dynamic-routing-vni</code> is used. + </column> + <column name="external_ids"> See <em>External IDs</em> at the beginning of this document. </column> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 562e6a0e88b3..77b8262ae471 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -16629,6 +16629,52 @@ OVN_CLEANUP_NORTHD AT_CLEANUP ]) +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([dynamic-routing - learned route with EVPN L3 VNI]) +AT_KEYWORDS([dynamic-routing]) +ovn_start + +check ovn-nbctl lr-add lr0 +check ovn-nbctl --wait=sb set Logical_Router lr0 option:dynamic-routing=true \ + option:dynamic-routing-redistribute="connected,static" +check ovn-nbctl --wait=sb lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 +sw0=$(fetch_column port_binding _uuid logical_port=lr0-sw0) +datapath=$(fetch_column datapath_binding _uuid external_ids:name=lr0) + +# A type-5 route learned with an L3 VNI (external_ids:vni) that differs from +# the local switch's dynamic-routing-vni. northd should carry the VNI towards +# the EVPN tunnel egress via reg16 (REG_EVPN_L3_VNI) with the validity bit set. +check_uuid ovn-sbctl create Learned_Route \ + datapath=$datapath \ + logical_port=$sw0 \ + ip_prefix=172.16.0.0/24 \ + nexthop=10.0.0.11 \ + external_ids:vni=100 +check ovn-nbctl --wait=sb sync +check_row_count Learned_Route 1 +ovn-sbctl dump-flows lr0 > lr0flows +AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | grep 'ip4.dst == 172.16.0.0/24' | ovn_strip_lflows], [0], [dnl + table=??(lr_in_ip_routing ), priority=1838 , match=(reg7 == 0 && ip4.dst == 172.16.0.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; evpn_l3_vni[[0..23]] = 100; evpn_l3_vni[[31]] = 1; reg0 = 10.0.0.11; reg5 = 10.0.0.1; eth.src = 00:00:00:00:ff:01; outport = "lr0-sw0"; flags.loopback = 1; reg9[[9]] = 1; next;) +]) + +# A learned route without a VNI must not set the EVPN VNI register (fallback +# to the destination switch's dynamic-routing-vni at egress). +check_uuid ovn-sbctl create Learned_Route \ + datapath=$datapath \ + logical_port=$sw0 \ + ip_prefix=172.16.1.0/24 \ + nexthop=10.0.0.12 +check ovn-nbctl --wait=sb sync +check_row_count Learned_Route 2 +ovn-sbctl dump-flows lr0 > lr0flows +AT_CHECK([grep -w "lr_in_ip_routing" lr0flows | grep 'ip4.dst == 172.16.1.0/24' | ovn_strip_lflows], [0], [dnl + table=??(lr_in_ip_routing ), priority=1838 , match=(reg7 == 0 && ip4.dst == 172.16.1.0/24), action=(ip.ttl--; reg8[[0..15]] = 0; reg0 = 10.0.0.12; reg5 = 10.0.0.1; eth.src = 00:00:00:00:ff:01; outport = "lr0-sw0"; flags.loopback = 1; reg9[[9]] = 1; next;) +]) + +OVN_CLEANUP_NORTHD +AT_CLEANUP +]) + OVN_FOR_EACH_NORTHD_NO_HV([ AT_SETUP([dynamic-routing - route learning ecmp]) AT_KEYWORDS([dynamic-routing]) diff --git a/tests/system-ovn.at b/tests/system-ovn.at index 17b4dcb3dd05..54e942de8087 100644 --- a/tests/system-ovn.at +++ b/tests/system-ovn.at @@ -20521,6 +20521,24 @@ AT_CHECK([ip route del 10.10.3.1 via 20.0.0.25 vrf vrf-$vni]) OVS_WAIT_FOR_OUTPUT([ovn-sbctl list Learned_Route | grep ip_prefix | sort], [0], [dnl ]) +# A route learned with a lightweight-tunnel (LWT) encap ID - as FRR installs +# for EVPN type-5 routes carrying an L3 VNI - records that VNI in the +# Learned_Route's external_ids:vni. A route without an encap ID has no VNI. +AS_BOX([$(date +%H:%M:%S.%03N) Learned route with EVPN L3 VNI]) +AT_CHECK([ip route add 10.10.5.1 encap ip id 5000 dst 20.0.0.25 \ + via 20.0.0.25 vrf vrf-$vni proto zebra]) +wait_row_count Learned_Route 1 ip_prefix=10.10.5.1 external_ids:vni=5000 + +AT_CHECK([ip route add 10.10.6.1 via 20.0.0.25 vrf vrf-$vni proto zebra]) +wait_row_count Learned_Route 1 ip_prefix=10.10.6.1 +# Only the encap route carries a VNI; the plain route does not. +check_row_count Learned_Route 1 external_ids:vni=5000 + +AT_CHECK([ip route del 10.10.5.1 encap ip id 5000 dst 20.0.0.25 \ + via 20.0.0.25 vrf vrf-$vni]) +AT_CHECK([ip route del 10.10.6.1 via 20.0.0.25 vrf vrf-$vni]) +wait_row_count Learned_Route 0 + # Disable learning on router AS_BOX([$(date +%H:%M:%S.%03N) Disable dynamic-route learning]) -- 2.38.1 _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
