bhouse-nexthop commented on PR #14611:
URL: https://github.com/apache/cloudstack/pull/14611#issuecomment-6751737673
Added a commit, 1370152b9d ("CKS: drop a removed node's VM map entry even if
its VM can't be expunged"), for a related scale-down bug in the node-removal
step this PR already touches.
When a node had been deleted from Kubernetes but its VM then failed to
destroy or expunge, the exception skipped the deletion of the VM's
`kubernetes_cluster_vm_map` entry. That stale entry never matches a Kubernetes
node again, so the state scanner kept the cluster in Alert for good (and Alert
blocks the next scale). The next scale-down's node count and node choice also
worked from it.
Now the entry is deleted once the node is out of Kubernetes, whatever
happens to the VM:
- an expunge failure is a warning event and the scale-down goes on; the
expunge task retries the VM;
- a destroy failure still fails the scale-down, with an error naming the VM
to destroy by hand;
- if the scale-down stops part way, the cluster's node count, cores and
memory are brought back in line with the map.
The commit message has the details, and there are unit tests for the happy
path, expunge failure, destroy failure and partial failure. As before, I have
only run unit tests, not tested against a live cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
