permyakovsv opened a new issue, #13970:
URL: https://github.com/apache/cloudstack/issues/13970
As an Operator I would like to be able to restrict the source CIDR of the
firewall rules that
CKS provisions for node SSH access, instead of having them permanently
opened to `0.0.0.0/0`.
### Current behaviour
When a Kubernetes cluster is created on an isolated network, CKS provisions
ingress firewall
rules on the network's source NAT IP for the node SSH ports (`2222` .. `2222
+ nodes - 1`,
plus one extra rule per external node). The source CIDR is hard-coded:
```java
//
plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/
// actionworkers/KubernetesClusterActionWorker.java
protected void provisionFirewallRules(final IpAddress publicIp, final
Account account,
int startPort, int endPort) throws ...
{
List<String> sourceCidrList = new ArrayList<String>();
sourceCidrList.add("0.0.0.0/0"); // <-- hard-coded
...
cidrField.set(rule, sourceCidrList);
firewallService.createIngressFirewallRule(rule);
firewallService.applyIngressFwRules(publicIp.getId(), account);
}
```
It is reached from `addFirewallRulesForNodes()`. There is no way to override
the value:
`KubernetesClusterManagerImpl.getConfigKeys()` exposes 15 keys (timeouts,
network offering,
max cluster size, etcd start port, ...) and none of them relates to network
access, and
`CreateKubernetesClusterCmd` has no corresponding parameter.
The net effect is that on every CKS cluster deployed on an isolated network,
SSH of every
control and worker node is reachable from the whole internet, and the
operator has no
supported way to change that.
### Why narrowing the rule by hand is not a workaround
Editing the rule does not survive normal lifecycle operations.
`KubernetesClusterScaleWorker.scaleKubernetesClusterIsolatedNetworkRules()`
calls
`removeSshFirewallRule()`, which matches the rule **by port number only**:
```java
if (Objects.equals(firewallRule.getSourcePortStart(),
CLUSTER_NODES_DEFAULT_START_SSH_PORT)
|| (Objects.nonNull(pfRule) && pfRule.getDestinationPortStart() ==
DEFAULT_SSH_PORT)) {
rule = firewallRule;
firewallService.revokeIngressFwRule(firewallRule.getId(), true);
break;
}
```
The narrowed rule is therefore revoked and then recreated with `0.0.0.0/0` by
`setupKubernetesClusterIsolatedNetworkRules()`. The same happens on
`addNodesToKubernetesCluster`. Because the public ports of the SSH
port-forwarding rules are
fixed by CKS itself, there is no way to express a narrowed rule that this
matcher would not
pick up. `cidrlist` of an existing firewall rule is immutable, so it cannot
be edited in place
either.
### Prior discussion
This has been raised before, inside bug reports about something else:
* [#11779](https://github.com/apache/cloudstack/issues/11779) describes
exactly this scenario —
the reporter removed the default wide-open rules for security reasons,
after which cluster
scaling failed with `ManagementServerException: Firewall rule for node SSH
access can't be
provisioned`. Closed as a duplicate of #11758.
* In [#11758](https://github.com/apache/cloudstack/issues/11758),
@weizhouapache confirmed
(2025-10-01) that *"several code lines are based on the assumption that
the firewall rules
and port forwarding rules for SSH (to control/worker nodes) start from
port 2222"*, and when
asked specifically about the security risk of opening 6443 and 2222–22xx
to `0.0.0.0/0`,
answered (2025-10-03): *"I understand your concerns. I agree we should
improve it. it is not
a simple fix, please keep eye on this issue"*.
[#12806](https://github.com/apache/cloudstack/pull/12806) then closed
#11758. That PR fixed the
robustness side of the problem — a missing or NULL-ported rule no longer
throws — but
intentionally left the hard-coded CIDR untouched. As a result the security
aspect is currently
not tracked by any open issue, which is why I am opening this one.
### Proposed feature
Add a configuration key, for example
`cloud.kubernetes.cluster.ssh.allowed.cidr`, scoped to
Account or Domain, defaulting to `0.0.0.0/0` so that existing behaviour is
preserved, and use
its value in `provisionFirewallRules()` in place of the constant. Optionally
expose the same
value as a parameter of `createKubernetesCluster` so it can be set per
cluster.
Two related points worth covering by the same setting:
* The API-port rule (6443) provisioned for the load balancer / port
forwarding has the same
issue.
* For VPC-based clusters the equivalent path is `createVpcTierAclRules()` →
`provisionVpcTierAllowPortACLRule()`, which builds a `CreateNetworkACLCmd`
without setting
`cidrlist`, so `getSourceCidrList()` falls back to `0.0.0.0/0` and `::/0`.
A more thorough alternative would be to stop relying on the public IP for
management-server →
node SSH altogether (the shared-network path already uses the node's private
address via
`getKubernetesClusterServerIpSshPortForSharedNetwork()`), but a configurable
CIDR would already
remove the immediate exposure with a very small change.
### Versions
Observed on 4.22.1.0; the code paths above are unchanged in `main`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
