Hi,
I would like to discuss the OVN dynamic-routing model required to
support workload-local EVPN Type-5 advertisement for OpenStack
distributed Floating IPs.
The intended OpenStack use case is:
- Neutron ML2/OVN creates a distributed dnat_and_snat entry for a
Floating IP.
- DNAT/SNAT processing takes place on the compute chassis hosting
the VM.
- The Floating IP /32 is advertised as an EVPN Type-5 route from
the same compute chassis.
- When the VM migrates, the old compute withdraws the /32 and the
new compute advertises it.
In other words, the desired ownership relationship is:
NAT.logical_port locality
= VM owner chassis
= FIP DNAT/SNAT chassis
= FIP Type-5 advertising chassis
This follows the existing OpenStack distributed Floating IP model,
where FIP traffic is processed on the compute hosting the VM. The
difference is that external reachability would be provided through an
EVPN Type-5 host route instead of relying only on direct provider
network L2 adjacency.
This work is related to the previous discussion about advertising
distributed NAT external IP/MAC information through EVPN Type-2:
https://mail.openvswitch.org/pipermail/ovs-dev/2026-May/432228.html
The Type-2 work provides the FIP external MAC/IP binding and mobility.
This RFC discusses the separate Type-5 route ownership decision.
== Scope relative to the OpenStack Type-5 proposal ==
The current Neutron 2026.2 EVPN Type-5 specification explicitly uses
centralized routing:
https://specs.openstack.org/openstack/neutron-specs/specs/2026.2/bgp_evpn_type_5_route_support.html
Therefore, I am not treating distributed FIP Type-5 advertisement as
functionality that the current Neutron specification already promises.
My question is whether OVN dynamic routing could provide the generic
route-selection primitive required for a possible future OpenStack
extension, without changing the existing behavior of centralized NAT
or ordinary routes.
Neutron API, capability negotiation and deployment policy could be
handled separately after the OVN ownership and dataplane models are
agreed upon.
== Test topology ==
The issue was reproduced with the following topology:
Provider network:
flat network connected to the Neutron router
Tenant network:
Geneve network containing the VM
Compute1:
hosts the VM LSP
NAT.logical_port is bound here
Advertised_Route.tracked_port is local here
Compute2:
hosts the active gateway/redirect port
the advertising/route-exchange LRP is local here
The relevant distributed NAT entry contains:
type = dnat_and_snat
external_ip = <FIP>
external_mac = <FIP external MAC>
logical_port = <VM LSP>
gateway_port = <gateway LRP>
ovn-northd correctly generates an Advertised_Route similar to:
ip_prefix = <FIP>/32
logical_port = <advertising/route-exchange LRP>
tracked_port = <VM LSP on Compute1>
The tracked_port therefore already contains the correct source of
workload ownership.
== Neutron topology constraint observed in this environment ==
Although enable_distributed_floating_ip=true is configured, this
router contains both a flat provider network and a Geneve tenant
network.
Neutron consequently configures the gateway LRP with:
reside-on-redirect-chassis=true
As a result, the route-exchange/advertising LRP is local to the active
redirect chassis rather than to every workload chassis.
I understand this to be an intentional Neutron safeguard for the
existing mixed provider/overlay routing model. I am not proposing to
remove this behavior globally.
It seems that supporting this OpenStack use case may require
distinguishing between:
- gateway redirect ownership used for centralized gateway and SNAT
behavior; and
- workload ownership used for distributed FIP Type-5 advertisement.
== Current ovn-controller behavior ==
In controller/route.c, route_run() first checks whether the
Advertised_Route.logical_port is local:
if (!route_advertising_port_is_local(...)) {
continue;
}
Only after this check does the controller evaluate tracked_port
locality and assign PRIORITY_LOCAL_BOUND.
On Compute1, the state is:
advertising logical_port = remote
tracked VM port = local
The route is therefore skipped before tracked_port locality is
evaluated.
Consequently, Compute1 does not install the FIP /32 into its local VRF
table, and FRR cannot advertise the Type-5 route from the VM owner
chassis.
Compute2 can satisfy advertising-port locality because it owns the
active redirect/route-exchange LRP, but it does not own the VM.
Based on my current understanding, advertising LRP locality, rather
than distributed NAT workload locality, determines where the FIP
Type-5 route can be exported.
== Control-plane PoC result ==
For testing only, I made the distributed parent LRP eligible for route
exchange on Compute1 and disabled the redirect-chassis restriction.
After ovn-controller recomputed:
- the FIP /32 appeared in the local Linux VRF table on Compute1;
- FRR advertised the Type-5 route from Compute1;
- the EVPN next hop changed to the Compute1 VTEP; and
- the non-owner compute did not advertise the local route.
This seems to indicate that:
- northd selects the correct VM LSP as tracked_port;
- tracked-port locality can drive the expected route priority; and
- the mandatory advertising logical_port locality check prevents the
VM owner controller from reaching the tracked_port decision.
These manual changes were used only to isolate the control-plane
behavior and are not proposed as an operational configuration.
== Possible ownership model for discussion ==
For normal routes and centralized NAT routes, the existing rule could
remain unchanged:
advertising logical_port must be local
For an explicitly identified tracked-port distributed NAT route, a
possible alternative would be to require both:
1. the chassis has the required local EVPN advertising datapath,
VRF/VNI and route-exchange capability; and
2. Advertised_Route.tracked_port is local and up on that chassis.
Conceptually:
normal route or centralized NAT:
selected by advertising logical_port locality
distributed NAT host route:
selected by local advertising datapath capability
AND tracked VM port locality
The advertising LRP would still identify the logical router, datapath,
VRF/VNI and route-exchange context. Only its direct chassis locality
would no longer be a mandatory ownership condition for this specific
route class.
I am not suggesting that route_advertising_port_is_local() should
simply be removed. That could allow a chassis without the required
VRF/VNI or route-exchange configuration to advertise the prefix and
create a blackhole.
I may be missing an existing design assumption around advertising LRP
locality, so I would appreciate guidance from the maintainers before
proceeding with an implementation.
== Identifying distributed NAT routes ==
northd currently knows that the route source is ROUTE_SOURCE_NAT and
sets tracked_port to NAT.logical_port for distributed NAT.
However, the SB Advertised_Route does not retain an explicit
distinction between centralized NAT and tracked-port distributed NAT.
Distributed LB routes already use external_ids such as:
source=lb
distributed-lb=true
Would it be appropriate for northd to publish equivalent internal
metadata for distributed NAT, for example:
source=nat
distribution=tracked-port
ovn-controller could then apply the alternative locality rule only to
that route class.
Alternatively, is there an existing property or another preferred
representation that should be used instead of adding metadata?
== Expected migration behavior ==
The desired result during an OpenStack VM migration would be:
Before migration:
Compute1 owns the VM and advertises the FIP /32.
Other chassis do not advertise it.
During migration:
Compute1 observes the tracked VM port becoming remote or down and
withdraws the route.
Compute2 advertises the route only after the VM port is local/up
and the local EVPN VRF/VNI datapath is ready.
After migration:
only Compute2 advertises the FIP /32, with Compute2 as the EVPN
next hop.
At minimum, the final state should contain one owner advertisement and
no stale route on the previous owner.
Would independent reconciliation by each ovn-controller be considered
sufficient for this transition?
Should tracked-port distributed NAT routes publish
Advertised_Route_Status, similar to distributed LB routes, so that
migration and stale-route state can be observed?
I understand that Advertised_Route_Status reports the OVN/kernel route
state and does not confirm completion of the external FRR/BGP
advertisement.
== Separate dataplane observation ==
After forcing the Type-5 route to the VM-owning chassis, I observed a
separate dataplane issue: Type-5 ingress uses the per-VNI Router MAC,
while the existing distributed NAT path expects NAT.external_mac.
I plan to discuss this separately after the route ownership model is
agreed upon. I mention it here only to clarify that route-locality
support is necessary but not sufficient for complete distributed FIP
Type-5 support.
== Questions ==
1. Would workload-local Type-5 advertisement for an OpenStack
distributed dnat_and_snat entry be considered a reasonable
extension of the OVN dynamic-routing model?
2. For this route class, would it be reasonable to select the
advertising chassis using:
local EVPN datapath/VRF/VNI capability
+ local/up tracked_port
instead of requiring direct advertising LRP locality?
3. Should northd explicitly identify tracked-port distributed NAT
routes in Advertised_Route, or is there an existing representation
that should be used?
4. If this direction is reasonable, would a failing system test for
the remote-advertising-LRP/local-tracked-port case be the preferred
first step?
If the general direction is considered reasonable, I can start with a
failing system test that reproduces:
- advertising LRP remote;
- distributed NAT tracked VM LSP local;
- local EVPN advertising datapath available; and
- no FIP /32 installed on the VM owner chassis.
The first patch series could then address route identification,
tracked-port chassis selection, withdrawal and migration tests.
The Router MAC ingress issue could be proposed separately after the
route ownership model is agreed upon.
Thank you for your time and guidance.
Best regards,
Chanyeol Yoon
KT Cloud
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
