adestis-mf opened a new issue, #114:
URL: https://github.com/apache/cloudstack-kubernetes-provider/issues/114
Since apache/cloudstack#12706 (CloudStack 4.23.0.0) a VPC offering can
include the Firewall
service for public IPs. The VPC virtual router then drops ingress traffic to
a public IP that
has a port forwarding, load balancing or static NAT rule unless a firewall
rule allows it (a
`FIREWALL_<ip>` chain ending in DROP).
`applyLoadBalancerRules` (`cloudstack_loadbalancer.go`, `main` at
363cd035ea, same in v1.3.0-rc1)
decides per port between a firewall rule and a network ACL item from the
services of the
cluster's **network**:
```go
if isFirewallSupported(network.Service) {
lb.updateFirewallRule(lbRule.Publicipid, ...)
} else if isNetworkACLSupported(network.Service) {
lb.updateNetworkACL(..., network.Id)
}
```
In a VPC the Firewall service belongs to the VPC, not to the tier. A VPC
tier reports
`NetworkACL` but not `Firewall`, so the CCM only adds an ACL item, and the
service port stays
behind the DROP of the VPC firewall. Removal (`pruneFirewallRule` /
`pruneNetworkACLRule`) uses
the same switch.
### Reproduced
CloudStack 4.23.0.0 (KVM), VPC offering with `Firewall:VpcVirtualRouter`,
tier offering with
`Dhcp, Dns, Lb, NetworkACL, PortForwarding, SourceNat, StaticNat, UserData,
Vpn` (no Firewall),
CKS cluster v1.35.4 (1 control, 1 worker), CCM `v1.3.0-rc1`, nginx behind a
`LoadBalancer`
service on port 80.
Service on the cluster's public IP (`spec.loadBalancerIP` set to the IP that
CKS uses for the
API, so the IP is already configured on the VR):
- The CCM created the load balancer rule (port 80, Active) and a network ACL
item for TCP 80
("Managed by the CloudStack Kubernetes Provider"), but no firewall rule.
- TCP 80 on that IP timed out from two different clients, while TCP 6443 on
the same IP, which
has a firewall rule, answered.
- After adding a firewall rule for TCP 80 on that IP by hand, the service
answered at once
(HTTP 200 from both clients).
- On `kubectl delete svc` the CCM removed the load balancer rule and the ACL
item. The firewall
rule stayed, as expected, since the CCM does not know about it; a fix has
to remove the rule it
created, because the IP is shared with the cluster API.
Service on a new public IP (no `loadBalancerIP`):
- Same rules as above: load balancer rule and ACL item, no firewall rule.
- In addition, the new IP was never configured on the VPC virtual router: no
`IpAssocVpcCommand`
was sent when the load balancer rule was applied, the IP was missing on
the VR's public
interface, and a firewall rule added by hand did not change that. This
part is a CloudStack
issue (apache/cloudstack#14146): once a public IP of a tier has a firewall
rule (here the
cluster IP, which has rules for the API), no further public IP of that
tier is added to the
router. Fix: apache/cloudstack#14681. Until then the CCM change alone is
not enough for this case.
In a VPC without the Firewall service the same kind of service on its own
IP works.
### Expected
When the cluster network is a VPC tier and the VPC has the Firewall service
(the VPC's `service`
list from `listVPCs` contains `Firewall`), create a firewall rule for each
service port on the
service's public IP in addition to the ACL item, honouring
`loadBalancerSourceRanges`, and remove
it together with the service. The order in the code is already right (load
balancer rule first,
then the firewall rule): on CloudStack versions without the fix for
apache/cloudstack#14146 the
opposite order keeps a new public IP off the router.
### Related
- apache/cloudstack#14668: the same gap in CKS itself (cluster API and SSH),
fix in
apache/cloudstack#14670.
- apache/cloudstack#14146 (and #14669): once a public IP of a VPC tier has a
firewall rule,
further public IPs of the tier are not added to or removed from the
router; fix in
apache/cloudstack#14681.
- Not covered here: a tier whose own offering includes the Firewall service
takes the firewall
branch and gets no ACL item; whether that matters depends on the ACL of
the tier.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
