adestis-mf commented on issue #14146: URL: https://github.com/apache/cloudstack/issues/14146#issuecomment-6802225858
Root cause, a reproduction without CKS, and a fix in #14681. I ran into the same problem with a CKS cluster and opened #14669 before I found this issue; it describes the same case, so I collect the details here. The order in the description (firewall rule first, then the port forwarding rule) is one form of the problem. The trigger is a firewall rule on **any** public IP of the tier, not only on the new IP, and it affects load balancing rules and the removal of IPs as well. `IpAddressManagerImpl.applyIpAssociations(network, postApplyRules, continueOnError, publicIps)` passes `VpcVirtualRouterElement.applyIps` the union of the services of the public IPs it hands over. For port forwarding and load balancing rules (also when they are re-applied on a network restart) these are all public IPs associated with the tier (`listByAssociatedNetwork`). `NetworkModelImpl.getIpToServices(..., includingFirewall = true)` adds `Service.Firewall` for every IP that has a firewall rule. `applyIps` checks `canHandle(network, service)` for every service in that union; if one check fails, it associates no IP and returns `true`, without a log line. Since #12706 the Firewall service for public IPs is provided by the VPC offering and the tier's network offering does not list it, so `canHandle(tier, Firewall)` is `false`. As a result, once any public IP of a tier has a firewall rule, no `IpAssocVpcCommand` is sent for that tier any more, neither to add a public IP nor to remove one. `applyStaticNats` hands over only the static NAT IPs, so static NAT is not affected by firewall rules on other IPs. Reading the code, it would be affected when the static NAT IP itself already has a firewall rule; that case was not tested. Reproduction on 4.23.0.0 without patches: KVM, a VPC offering with the Firewall service provided by the VPC virtual router and conserve mode on, one tier whose network offering (conserve mode on) has no Firewall service, one VM. All rules use port 22; the firewall rules allow one client address. `IpAssocVpcCommand` is taken from the management server log, the router state from `getDiagnosticsData` (`files=ipaddr,/etc/cloudstack/ips.json`). | # | Action | Firewall rules in the tier | `IpAssocVpcCommand` | Result | |---|---|---|---|---| | 1 | IP A: port forwarding | none | A | A on the router | | 2 | IP B: port forwarding | none | A, B | B on the router | | 3 | Firewall rule on A | | | A reachable | | 4 | IP C: port forwarding, then a firewall rule on C | on A | none | C not on the router, not reachable | | 5 | IP D: static NAT to the VM | on A, C | D | D on the router | | 6 | IP E: load balancing rule | on A, C | none | E not on the router | | 7 | `restartNetwork cleanup=false` | on A, C | D only | C and E still not on the router | | 8 | Delete the port forwarding rule of B | on A, C | none | B stays on the router; `listPublicIpAddresses` no longer shows B as associated with the tier | | 9 | Delete the firewall rules on A and C, then `restartNetwork cleanup=false` | none | A, C, D, E | C and E on the router; B still on the router | | 10 | Firewall rules on C and E | | | C and E reachable | Step 9 is the counter-check: same tier, same router, same port forwarding, load balancing and static NAT rules; only the firewall rules are gone. Step 8 shows the other direction: an IP whose last rule is removed stays configured on the router. CloudStack no longer associates it with the tier, so a later restart does not remove it either. Reading the code, releasing such an IP does not remove it from the router either; that was not tested. The fix in #14681 is in `VpcVirtualRouterElement.applyIps`: when the tier does not provide the Firewall service, the service is checked on the VPC, as `applyFWRulesInVPC` does. Other services and `canHandle(network, service)` are unchanged. I saw that this issue is assigned; I hope the PR helps, and I am happy to adjust it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
