rp- opened a new pull request, #13910:
URL: https://github.com/apache/cloudstack/pull/13910
### Description
This PR adds support for LINSTOR **shared storage pools** (thick LVM on a
LUN that all
hypervisors can access, SAN-style) to the Linstor volume plugin.
Until now the plugin assumed every resource is DRBD replicated and thin
provisioned. On a
shared storage pool the data exists once on the LUN, a resource is active on
at most one
node at a time, there is no DRBD layer to make dual-primary, and the backing
LVM is thick.
Functionally this changes:
* **Live migration** uses the LINSTOR 1.29 API: `make-available` with
`auto_manage_dual_primary` prepares the destination (DRBD dual-primary, or
activating the
resource on both nodes for shared pools) and `unmake-available` reverts it
on the source.
The controller REST API version is probed once per controller; against
controllers older
than 1.29 the previous manual `allow-two-primaries` handling is used
unchanged, so
existing DRBD/thin deployments behave exactly as before.
* **Resources of stopped VMs** are INACTIVE on a shared pool and expose no
device. They are
now activated on demand (through the controller, so the shared-space lock
is honored) for
VM start, snapshot backup, snapshot revert and template cloning, and
deactivated again
afterwards.
* **Snapshots on thick LVM** work: the snapshot path is resolved for thick
pools, the
snapshot LV is temporarily activated for the copy to secondary storage
(without
registering it with `dmeventd`, which could block the node wide LVM lock),
and the copy is
routed to the node the volume is active on, because dm-snapshot is not
cluster aware.
* **Capacity reporting** counts a shared space once instead of once per node
(a 100 GiB LUN
on three nodes was reported as 300 GiB), and thickly provisioned pools get
a per pool
`storage.overprovisioning.factor` of 1.0 at registration, since they
cannot be over
provisioned. Thin pools keep the current default.
* Resources and clone targets are created with the layer stack configured on
the resource
group instead of the implicit default, which otherwise produced DRBD
resources in a
STORAGE-only resource group.
One change is outside the plugin:
`KVMStorageProcessor.copyVolumeFromPrimaryToSecondary()`
now connects the source volume before reading it and disconnects it
afterwards, mirroring
what the VM start and attach flows do. Migrating a **detached** volume
between pools
otherwise reads a device that was never connected, which fails for any
storage driver that
exposes devices on demand.
Requires `java-linstor` 0.8.0 (LINSTOR REST API 1.29.0, shipped with
linstor-server 1.35).
<!-- Fixes: # -->
### Types of changes
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] Build/CI
- [ ] Test (unit or integration test code)
### Feature/Enhancement Scale or Bug Severity
#### Feature/Enhancement Scale
- [x] Major
- [ ] Minor
#### Bug Severity
- [ ] BLOCKER
- [ ] Critical
- [ ] Major
- [ ] Minor
- [ ] Trivial
### Screenshots (if appropriate):
n/a - no UI or API changes.
### How Has This Been Tested?
Two 3-node KVM clusters (Ubuntu 24.04, CloudStack 4.22, one management
server each), so
both storage models and both controller API versions are covered:
| cluster | primary storage | LINSTOR REST API | code path |
| --- | --- | --- | --- |
| A | shared thick LVM: one 100 GiB LUN attached to all hosts, LVM VG with a
LINSTOR shared space, resource group `STORAGE` layer only, place-count 1 |
1.29.0 (linstor-server 1.35) | new make-available/unmake-available |
| B | thin LVM (`lvmthin`) + DRBD, 2 replicas | 1.28.0 | version fallback to
the previous handling |
Marvin, `test/integration/plugins/linstor`:
* `test_linstor_volumes.py` (15 tests): **15/15 pass on both clusters** -
includes volume
attach/detach/reboot, snapshot create, template from snapshot, volume
migration to the
same and to a distinct pool, and VM snapshot create/revert/delete.
* `test_linstor_encrypted_snapshots.py` (3 tests): **3/3 pass on cluster A**
(revert of an encrypted root snapshot,
create-volume-from-encrypted-snapshot is rejected,
and the qcow2 on secondary storage is verified to be LUKS encrypted at
rest). On cluster B
the third test skips itself because it needs DB access from the test
runner to locate the
backed-up file.
* 9 plugin unit tests, incl. new ones for the shared-space capacity
deduplication, thick
provider-kind detection, diskful copy ordering and the controller version
gate.
Manually verified on cluster A, checking the LINSTOR side after each step:
VM deploy from
template, stop/start of a VM whose resource is INACTIVE, live migration
(controller log
shows make-available on the destination and unmake-available on the source;
the resource
ends up active on the destination only), snapshot of a running and of a
stopped VM,
template from snapshot, volume migration between pools, and capacity (the
100 GiB LUN is
now reported as 100 GB instead of 599.98 GB).
#### How did you try to break this feature and the system with this change?
* **Old controller**: ran the full suite against a 1.28.0 controller. This
surfaced that the
regenerated client serialized `auto_manage_dual_primary` on *every*
make-available (the
generated field is initialized with its schema default), and LINSTOR
rejects unknown
properties - so template copies and VM starts failed even though the
feature is version
gated. Fixed by only setting the flag for migrations (and, in
java-linstor, by serializing
only explicitly set properties). Full suite passes on 1.28.0 afterwards,
i.e. no
regression for existing deployments.
* **Cold start**: rebooted all nodes, so every shared resource came back
INACTIVE. That
broke template cloning, VM start and snapshots, which all assumed an
active resource;
each path now activates on demand and was re-tested from a rebooted
cluster.
* **Concurrency**: parallel deploys from the same template - deactivating
the template right
after a clone raced other clones still using it, so the template is left
active.
* **Host loss**: a diskful copy whose host is down/disabled no longer forces
a fallback to a
temporary resource; all diskful copies are considered, in preference order.
* **Thin pools**: verified capacity, over-provisioning and snapshot behavior
are unchanged
for `lvmthin` (dedup keys on the shared space, thick detection requires
all pools of the
resource group to be thick, mixed setups keep the current default).
* **Encrypted (LUKS) volumes** on thick pools, snapshot on a stopped VM
(resource inactive)
and on a running VM, and repeated snapshot/delete cycles to check nothing
is left behind
in LINSTOR.
* **LVM edge cases**: `lvchange` registering a snapshot LV with `dmeventd`
could hang and
block the node wide LVM lock, wedging all storage operations on that host
- the temporary
activation now uses `--monitor n`.
Testing this also uncovered several LINSTOR server side issues (satellite
startup probe
racing on a shared VG, snapshot rollback with an inactive copy, an
io-suspend hang on
encrypted devices, a clone that never left `CLONING`); those are fixed in
linstor-server
1.35 and are not CloudStack changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
