adestis-mf opened a new issue, #14669:
URL: https://github.com/apache/cloudstack/issues/14669
### problem
With the Firewall service for VPC public IPs (#12706), a firewall rule can
be created on a
public IP of a VPC before the IP has any port forwarding, load balancing or
static NAT rule. In
that order the IP is never configured on the VPC virtual router:
1. `IpAddressManagerImpl.applyRules` resolves only the VPC for a VPC
firewall rule
(`networkId` is null), skips the IP association and logs
`Unable to resolve network for firewall rules (networkId=null,
vpcId=...); skipping IP association`.
The firewall rules are sent to the VR anyway and the `FIREWALL_<ip>`
chain is created.
2. The first LB or PF rule on the IP is applied afterwards, but no
`IpAssocVpcCommand` is sent
for it. The IP never appears in the `ips` data bag of the VR and is not
added to the public
interface. Clients on the same public subnet get `No route to host`,
others time out.
3. When the LB/PF rules are applied, the VR also deletes the previously
added firewall rules
from `FIREWALL_<ip>` (`Delete rule -D FIREWALL_<ip> ...` in
`/var/log/cloud.log`), so only the
final DROP remains. Re-applying the firewall rules restores them, but the
IP is still not on
the VR. (On a correctly associated IP, re-applying PF rules during a CKS
scale kept the
firewall rules.)
With the opposite order (LB/PF first, firewall rules afterwards) everything
works: the LB/PF
apply sends `IpAssocVpcCommand`, the VR runs `ip addr add dev eth1
<ip>/<mask>`, and later
firewall rules take effect immediately. The skip in step 1 is harmless then,
because the IP is
already associated.
Observed on 4.23.0.0 with a VPC offering that has the Firewall service
(provider
VpcVirtualRouter), same VPC, same tier, same steps except for the order:
| | firewall rules before LB/PF | firewall rules after LB/PF |
|---|---|---|
| `IpAssocVpcCommand` in the job that applied LB/PF | 0 | 8 |
| IP on the VR public interface (`ip addr`) | no | yes (`secondary eth1`) |
| `FIREWALL_<ip>` after the LB/PF apply | rules deleted, only DROP | rules
present |
| TCP from an allowed source | timeout / `No route to host` | open |
In the "before" case the management server also logged
`Merovingian2: Was unable to find lock for the key user_ip_address<id>`
while applying the LB
rule on that IP.
A side observation: after an IP with firewall rules is released, its
`FIREWALL_<ip>` chain stays
on the VR (with RELATED/ESTABLISHED and DROP), and the `PREROUTING -d <ip>
-j FIREWALL_<ip>`
jump stays as well. When the IP is acquired again in the same VPC the chain
is reused.
### versions
- Apache CloudStack 4.23.0.0; `IpAddressManagerImpl.applyRules` is unchanged
on `main`
(a7be64af7a)
- KVM, advanced zone, VPC offering with the Firewall service (provider
VpcVirtualRouter),
SystemVM template for 4.23 on the VR
### The steps to reproduce the bug
1. In a VPC with the Firewall service, get a new public IP that is
associated with the VPC and
with a tier but has no rules yet. We saw it with the IP that CKS acquires
for a cluster
(`associateIPToVpc` + `associateIPToGuestNetwork`); a plain
`associateIpAddress vpcid=...`
was not tested.
2. Create a firewall rule on that IP, for example TCP 22 from your address.
3. Create a port forwarding (or load balancing) rule on the same IP to a VM
in the tier.
4. The port is not reachable. `ip addr` on the VR (or `getDiagnosticsData
files=ipaddr`) does not
show the IP, and the job that applied the port forwarding rule sent no
`IpAssocVpcCommand`.
Doing step 3 before step 2 works.
### What to do about it?
When firewall rules are applied for a VPC public IP that is not yet
associated with the VR,
associate it first, as `applyRules` does for network-scoped rules (resolve
the network from
the IP's associated network or use the VPC-level IP association). The VR
should not drop
firewall rules of an IP when PF/LB rules for that IP are applied. Released
IPs should have their
firewall chain removed.
Found while fixing CKS for VPCs with the Firewall service (#14668), whose
fix creates the
firewall rules after the LB/PF rules to avoid this.
Related, but a different problem in the same feature: #14068 (NPE when
destroying a VPC tier
with the Firewall service, PR #14133).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
