jball-resetdata opened a new issue, #13757:
URL: https://github.com/apache/cloudstack/issues/13757
### problem
**Title:** `routingmode=Dynamic` is silently ignored on NATTED network
offerings — no BGP session is created for IPv6 on dual-stack NATTED isolated
networks
---
## problem
A network offering's `networkmode` (NATTED/ROUTED) applies to **both address
families at once**, and the BGP dynamic-routing machinery is gated on `ROUTED`.
This makes the normal dual-stack posture — **IPv4 NATted** (RFC1918 guest CIDR,
source NAT on the VR, because public IPv4 is scarce) + **IPv6 routed with the
guest /64 announced via BGP** — impossible in a single isolated network.
IPv6 NAT does not exist in CloudStack (correctly — the guest /64 is always
globally routed), so a `networkmode=NATTED internetprotocol=DualStack
routingmode=Dynamic` offering has exactly one sane meaning: v4 NAT + v6
dynamically routed. CloudStack accepts that offering, and then silently ignores
the Dynamic part:
- `createNetworkOffering` accepts the combination — there is no
cross-validation between `networkmode` and `routingmode`
(`ConfigurationManagerImpl`).
- The UI actively offers it: `AddNetworkOffering.vue` shows the routing-mode
selector for any IPv6/dual-stack offering regardless of NATTED/ROUTED.
- Creating a network from the offering **allocates a zone AS number** to it
— `NetworkServiceImpl` keys ASN allocation on `RoutingMode.Dynamic` alone.
- But the push to the VR is gated by
`RoutedIpv4ManagerImpl.isDynamicRoutedNetwork()`, which requires
`NetworkMode.ROUTED && RoutingMode.Dynamic`:
```java
@Override
public boolean isDynamicRoutedNetwork(NetworkOffering networkOffering) {
return
NetworkOffering.NetworkMode.ROUTED.equals(networkOffering.getNetworkMode())
&&
NetworkOffering.RoutingMode.Dynamic.equals(networkOffering.getRoutingMode());
}
```
For a NATTED offering,
`VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork` therefore
never sends `SetBgpPeersCommand`: `bgpd` is never enabled on the VR, no
`frr.conf` is written, and no BGP session is established for v4 **or** v6 —
with no error and no log line explaining why. `changeBgpPeersForNetwork` is
rejected with *"The network does not support Dynamic routing"*, and the BGP/ASN
fields are suppressed from `listNetworks` responses, so the network looks as if
dynamic routing was never configured.
Net effect: the operator configures Dynamic routing, the system consumes an
ASN for it, and silently delivers a static-routed network whose /64 is only
reachable if the operator hand-maintains upstream static routes (`ip6routes`).
The only workaround is two networks per tenant (one NATTED IPv4-only + one
ROUTED-Dynamic IPv6) with dual-NIC guests — two VRs, two ASNs, and asymmetric
routing.
### Origin — design gap, not a regression
`git blame` traces both the predicate and the `SetBgpPeersCommand` gate in
`VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork` to the
original dynamic-routing feature commit (`679ce1a639`, "feature: Dynamic and
Static Routing", #9470, merged Sep 2024 / 4.20). The `ROUTED && Dynamic`
restriction has been in place since the feature's inception, and neither the PR
nor the code records a deliberate decision to exclude NATTED dual-stack
offerings — while the offering-validation and ASN-allocation paths were left
accepting the combination. It appears the NATTED+Dynamic case was simply never
wired through, rather than intentionally rejected.
## versions
Observed on 4.22.0.0 and 4.22.1.0 (stock packages, KVM, Ubuntu 24.04). The
gating predicate is unchanged on current main — present since 4.20 (#9470);
this is configuration-flow behaviour, not version-specific.
---
*Related but distinct: #13745 (`routed.network.vpc.enabled` gates the
BGP/ASN APIs and hides them when disabled).*
### versions
4.22.0.0 and 4.22.1.0 (stock packages, KVM, Ubuntu 24.04).
### The steps to reproduce the bug
1. Advanced zone with: an IPv6 range on the public VLAN, a guest IPv6 prefix
(`createGuestNetworkIpv6Prefix`), an ASN range (`createASNRange`), and a BGP
peer (`createBgpPeer`).
2. Create a network offering: `networkmode=NATTED routingmode=Dynamic
internetprotocol=DualStack` with the usual VR services (SourceNat, Dhcp, Dns,
Firewall, ...) — accepted without complaint.
3. Create an isolated network from it — succeeds; an AS number is allocated
to the network (visible in the `as_number` table).
4. On the network's VR: `/etc/frr/daemons` still shows `bgpd=no`; the
bgppeers databag is empty; no BGP session, no v6 announcement.
5. `changeBgpPeersForNetwork` on the network →
`InvalidParameterValueException: The network does not support Dynamic routing`.
### What to do about it?
1. Advanced zone with: an IPv6 range on the public VLAN, a guest IPv6 prefix
(`createGuestNetworkIpv6Prefix`), an ASN range (`createASNRange`), and a BGP
peer (`createBgpPeer`).
2. Create a network offering: `networkmode=NATTED routingmode=Dynamic
internetprotocol=DualStack` with the usual VR services (SourceNat, Dhcp, Dns,
Firewall, ...) — accepted without complaint.
3. Create an isolated network from it — succeeds; an AS number is allocated
to the network (visible in the `as_number` table).
4. On the network's VR: `/etc/frr/daemons` still shows `bgpd=no`; the
bgppeers databag is empty; no BGP session, no v6 announcement.
5. `changeBgpPeersForNetwork` on the network →
`InvalidParameterValueException: The network does not support Dynamic routing`.
Expected: either the offering combination is honoured, or it is rejected at
validation time — not accepted and silently ignored.
The machinery already exists end-to-end: the VR's FRR config generation
(`CsBgpPeers.py`) is databag-driven and mode-agnostic, and the guest /64 is
already carried in `BgpPeerTO.guestIp6Cidr`. Honouring the combination appears
to be a very small change — relax `isDynamicRoutedNetwork()` to also accept
`NATTED && Dynamic`, and for NATTED networks suppress the IPv4 prefix (pass
`guestIp4Cidr=null` so the RFC1918 guest CIDR is never announced; the VR then
only configures the v6 address family). We have a draft patch along these lines
and are happy to raise a PR if maintainers agree with the direction. If instead
the view is that NATTED+Dynamic should be invalid, `createNetworkOffering` and
the UI should reject it explicitly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
