tromob opened a new issue, #13634:
URL: https://github.com/apache/cloudstack/issues/13634
### problem
This is a bug in the **CloudStack CSI driver** (repo
`cloudstack/cloudstack-csi-driver`, image
`ghcr.io/cloudstack/cloudstack-csi-driver`). Filing it here because Issues are
disabled on that repository. Related open PR:
cloudstack/cloudstack-csi-driver#7.
**Symptom:** Expanding a PVC (StorageClass with `allowVolumeExpansion:
true`) never completes when the backing CloudStack volume is owned by a
**project**. The `external-resizer` retries indefinitely and the PVC stays in
`Resizing` forever; `status.capacity` never updates.
On every attempt the `cloudstack-csi-controller` logs:
```
ControllerExpandVolume ... volume_id="<volume-uuid>"
rpc error: code = Internal desc = Could not resize volume "<volume-uuid>" to
size N:
failed to retrieve volume '<volume-uuid>': No match found for <volume-uuid>:
&{Count:0 Volumes:[]}
```
**Root cause:** In `pkg/cloud/volumes.go`, `ExpandVolume()` looks up the
volume with the raw cloudstack-go SDK call `c.Volume.GetVolumeByID(volumeID)`,
which does **not** include `projectid`. For a project-owned volume, a
project-scoped API key then gets `Count:0` ("No match found") — even though
`project-id` is correctly set in the cloud-config. The driver's own
project-aware wrapper `c.GetVolumeByID(ctx, volumeID)` (which calls
`SetProjectid`) is not used in this path.
**Secondary issue:** `ExpandVolume` calls `ResizeVolume(newSize)`
unconditionally, without checking `currentSize >= newSize`, so it also cannot
converge if the CloudStack volume is already at the requested size.
### versions
- Apache CloudStack: 4.22.0.0
- cloudstack-csi-driver: v3.0.0 (same code in release `cloudstack-csi-3.0.1`
and current `main`); image `ghcr.io/cloudstack/cloudstack-csi-driver`
- Kubernetes: v1.33; sidecar `external-resizer` v1.11.1
- Hypervisor: KVM
- Storage backend: Linstor/DRBD (note: the bug is at the CloudStack API
layer and is storage-agnostic)
### The steps to reproduce the bug
1. A Kubernetes cluster whose CSI volumes live in a CloudStack **project**,
with a project-scoped API key in `cloudstack-secret` (cloud-config has
`project-id` set).
2. Provision a PVC via the `cloudstack-csi` StorageClass (works fine).
3. Increase the PVC's `spec.resources.requests.storage` (e.g. 20Gi -> 40Gi).
4. `external-resizer` loops with `... No match found for <uuid>: &{Count:0
Volumes:[]}`; the PVC never leaves `Resizing`.
### What to do about it?
Make `ExpandVolume` use a project-aware lookup (the existing
`c.GetVolumeByID(ctx, ...)` wrapper), and/or set the project id globally on the
cloudstack-go client so every SDK call includes it.
Open PR cloudstack/cloudstack-csi-driver#7 already adds the global approach
via `csClient.DefaultOptions(cloudstack.WithProject(config.ProjectID))`, which
would also cover `ExpandVolume` — but it is still open, and its description
only mentions the `getVMByName` / node-metadata symptom, not volume resize.
Merging that (and noting it also fixes resize) would resolve this.
Additionally, guard `ExpandVolume` to skip the resize when the volume is
already at the requested size (idempotency).
**Workaround for others hitting this:** grow the CloudStack volume, then
online-grow the filesystem on the node (`resize2fs` on the mounted ext4 volume
— non-disruptive), and reconcile the Kubernetes objects (`PV .spec.capacity`
and `PVC .status.capacity` to the new size, and clear the `Resizing` condition)
so the resizer sees `requests == capacity` and stops.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
