hulucc opened a new issue, #14445:
URL: https://github.com/apache/cloudstack/issues/14445
##### ISSUE TYPE
* Bug Report
##### COMPONENT NAME
~~~
VR (systemvm scripts, cs/CsAddress.py), ROUTED network mode
~~~
##### CLOUDSTACK VERSION
~~~
4.20.3.1 (systemvm template 4.20.2, scripts from cloudstack-common 4.20.3.1)
The same code is on the 4.20 and 4.22 branches and main.
~~~
##### CONFIGURATION
Advanced zone, KVM. Isolated networks and VPCs with network mode ROUTED
(Static and Dynamic routing) using a redundant router offering.
##### OS / ENVIRONMENT
Management server 4.20.3.1, KVM hosts on CentOS 8 (libvirt 8.0), VR on
Debian 12.
##### SUMMARY
When a ROUTED isolated network or VPC uses a redundant router offering, both
VRs become PRIMARY: both hold the guest gateway VIP and the public IP.
In ROUTED mode the VR flushes iptables and uses nftables (table
`ip4_firewall` for isolated networks, `ip4_acl` for VPCs) with an INPUT policy
of drop. The INPUT whitelist built in `CsAddress.py` (`fw_router_routing` /
`fw_vpcrouter_routing`) accepts DHCP, DNS, userdata and SSH, but not VRRP
(224.0.0.18; keepalived uses `auth_type AH`, so these are IP protocol 51) or
conntrackd sync (225.0.0.50). The non-routed iptables path accepts both (`-A
INPUT -d 224.0.0.18/32 -j ACCEPT`, `-A INPUT -d 225.0.0.50/32 -j ACCEPT` in
`fw_router` / `fw_vpcrouter`). Each VR drops the other's advertisements and
promotes itself.
Related: #11565 asks for redundant VR support in routed networks. Part of
this is already in place. The Dynamic and Static Routing feature (#9470) added
handling of the RedundantRouter capability on the Gateway service for ROUTED
offerings: in `ConfigurationManagerImpl` and `VpcManagerImpl`, in the
capabilities of `VirtualRouterElement`, and in the UI offering forms, which
send it when the redundant router switch is on. So redundant ROUTED networks
and VPCs can be created (via the API), but the VRs come up dual PRIMARY because
of the missing rules above.
As a side note, the UI switch is shown only when SourceNat is selected, so
it never appears for ROUTED offerings.
I am filing this as a bug against 4.20 because the configuration can be
created on 4.20 and leaves the network with duplicate gateway and public IPs.
If you would rather treat it as part of #11565 on main, I can retarget the PR;
the fix applies cleanly to main as well. With the two missing accept rules
added, redundancy worked in our tests (see the linked PR).
##### STEPS TO REPRODUCE
~~~
# 1. A redundant ROUTED offering (API only, see above)
cmk create networkoffering name=Isolated-Routed-Static-HA
displaytext=Isolated-Routed-Static-HA \
guestiptype=Isolated traffictype=GUEST networkmode=ROUTED
routingmode=Static \
supportedservices=Dhcp,Dns,UserData,Firewall specifyvlan=false \
serviceproviderlist[0].service=Dhcp
serviceproviderlist[0].provider=VirtualRouter \
serviceproviderlist[1].service=Dns
serviceproviderlist[1].provider=VirtualRouter \
serviceproviderlist[2].service=UserData
serviceproviderlist[2].provider=VirtualRouter \
serviceproviderlist[3].service=Firewall
serviceproviderlist[3].provider=VirtualRouter \
servicecapabilitylist[0].service=Gateway
servicecapabilitylist[0].capabilitytype=RedundantRouter \
servicecapabilitylist[0].capabilityvalue=true
cmk update networkoffering id=<id> state=Enabled
# 2. Add an IPv4 subnet for routed networks to the zone, create a network
with this
# offering (cidrsize=24) and deploy a VM in it.
# 3. On both VRs
/opt/cloud/bin/checkrouter.sh
ip -4 -br addr
nft list chain ip ip4_firewall INPUT
tcpdump -ni eth0 host 224.0.0.18
~~~
##### EXPECTED RESULTS
~~~
One VR PRIMARY, the other BACKUP (public interface down, no guest gateway
VIP).
~~~
##### ACTUAL RESULTS
~~~
r-563-VM Status: PRIMARY eth0 10.105.0.73/24 10.105.0.1/24 eth2 UP
10.73.3.208/24
r-564-VM Status: PRIMARY eth0 10.105.0.169/24 10.105.0.1/24 eth2 UP
10.73.3.208/24
keepalived: (inside_network) Entering BACKUP STATE
keepalived: (inside_network) Entering MASTER STATE <- 4 s later, on
both routers
tcpdump -ni eth0 host 224.0.0.18 (on either VR) shows the advertisements of
both routers:
IP 10.105.0.169 > 224.0.0.18: AH(spi=...): VRRPv2, Advertisement, vrid 51,
prio 100, authtype ah
IP 10.105.0.73 > 224.0.0.18: AH(spi=...): VRRPv2, Advertisement, vrid 51,
prio 100, authtype ah
table ip ip4_firewall { chain INPUT { type filter hook input priority
filter; policy drop;
... no rule accepting 224.0.0.18 / 225.0.0.50 ...
~~~
The same happens for a ROUTED VPC with a redundant VPC offering (`ip4_acl`
table).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
