bhouse-nexthop commented on PR #14637:
URL: https://github.com/apache/cloudstack/pull/14637#issuecomment-6752449180
### Test report: CKS upgrade with the binaries ISO already attached to a node
**Summary:** with the upgrade's binaries ISO already attached to the control
node, as an operation that didn't finish would leave it,
`upgradeKubernetesCluster` no longer fails at the attach step. It leaves the
ISO in place on that node, attaches it to the other node, upgrades both, and
detaches it from both at the end.
**Environment**
- A CloudStack 24.0 (main) build with this PR, alongside some of my other
open PRs, including #14634 and #14635 in the same plugin. The PR is for main
only.
- Three KVM hosts, Ceph RBD primary storage, one management server.
- Cluster B: 1 control node and 1 worker, Kubernetes v1.33.1, isolated
network. It was created on 4.20 and upgraded to 24.0.
- Upgrade target: v1.33.13, from an ISO built with main's
`create-kubernetes-binaries-iso.sh`.
**Method**
1. `attachIso` of the v1.33.13 binaries ISO to the control node.
`listVirtualMachines` then showed that ISO on the control node and no ISO
anywhere else.
2. `upgradeKubernetesCluster` to v1.33.13.
3. Check the log, the cluster, the nodes and which VMs hold an ISO.
**Results**
- The upgrade succeeded in 334 s: `Running` -> `Upgrading` -> `Running`.
- The attach step, trimmed:
```
20:30:56,899 INFO KubernetesClusterUpgradeWorker Upgrading Kubernetes
cluster: cluster B
20:30:56,984 INFO KubernetesClusterUpgradeWorker Binaries ISO is already
attached to VM: ... control-1 ...
20:30:57,260 INFO KubernetesClusterUpgradeWorker Attached binaries ISO
for VM: ... worker-1 ...
20:30:57,261 INFO KubernetesClusterUpgradeWorker Upgrading node
control-1 ...
20:35:14,855 INFO KubernetesClusterUpgradeWorker Successfully upgraded
node on VM ... control-1 ...
20:36:26,576 INFO KubernetesClusterUpgradeWorker Successfully upgraded
node on VM ... worker-1 ...
20:36:28,844 INFO KubernetesClusterUpgradeWorker Detached Kubernetes
binaries from VM: ... control-1 ...
20:36:29,755 INFO KubernetesClusterUpgradeWorker Detached Kubernetes
binaries from VM: ... worker-1 ...
```
- The log has no `The specified ISO is already attached to this Instance`,
which is the stock failure in #14630.
- `kubectl get nodes` afterwards: both nodes Ready, on v1.33.13. Every pod
was Running or Completed.
- No VM held an ISO afterwards.
**Result:** pass. A node already holding the binaries ISO no longer blocks
the upgrade. The ISO is kept on that node for the upgrade, and detached from
every node at the end, as for a node that had none.
**Not covered**
- A node holding a *different* ISO, which the PR detaches before attaching
the binaries ISO. That path is covered by the unit tests only.
- The same case in a scale-up or a start. A scale-up attaches the ISO only
to the new nodes, so a leftover ISO on an existing node doesn't reach this code
there.
- A leftover ISO in a second CD-ROM: the lab VMs have one CD-ROM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
