rosi-shapeblue commented on PR #11789:
URL: https://github.com/apache/cloudstack/pull/11789#issuecomment-3408192292
**WIP**
### 1.Prepare & Cancel Maintenance on Primary Storage
**Action:** Put pool in maintenance → cancel.
**Expected Result:** Pool transitions Up → PrepareForMaintenance → Up
cleanly, no errors.
**Actual Result**
- Smooth transition: Up → PrepareForMaintenance → Up.
- No ErrorInMaintenance.
- No exceptions in management-server.log.
- Pool fully usable after cancel.
**Evidence**
- Pools are in Up state
<details><summary><code>(localcloud) 🐱 > list storagepools </code></summary>
```json
{
"count": 2,
"storagepool": [
{
"clusterid": "46169705-e70d-46ff-ad6d-1cb31fc6423c",
"clustername": "p1-c1",
"created": "2025-10-15T13:38:55+0000",
"disksizeallocated": 5243076688,
"disksizetotal": 2898029182976,
"disksizeused": 2158164443136,
"hasannotations": false,
"hypervisor": "KVM",
"id": "cca423c8-b72e-36f2-8931-60434242a9ca",
"ipaddress": "10.0.32.4",
"istagarule": false,
"managed": false,
"name": "ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri1",
"overprovisionfactor": "2.0",
"path":
"/acs/primary/ref-trl-9699-k-Mol8-rositsa-kyuchukova/ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri1",
"podid": "0e13ddcc-d5ef-444a-a358-0c6ac6e9c744",
"podname": "Pod1",
"provider": "DefaultPrimary",
"scope": "CLUSTER",
"state": "Up",
"storagecapabilities": {
"VOLUME_SNAPSHOT_QUIESCEVM": "false"
},
"type": "NetworkFilesystem",
"zoneid": "36c95913-6c6d-400c-9d7e-12d8ac72fc35",
"zonename": "ref-trl-9699-k-Mol8-rositsa-kyuchukova"
},
{
"clusterid": "46169705-e70d-46ff-ad6d-1cb31fc6423c",
"clustername": "p1-c1",
"created": "2025-10-15T13:38:56+0000",
"disksizeallocated": 5243076688,
"disksizetotal": 2898029182976,
"disksizeused": 2158164443136,
"hasannotations": false,
"hypervisor": "KVM",
"id": "5e33de91-2078-34b9-b954-c6646408bf99",
"ipaddress": "10.0.32.4",
"istagarule": false,
"managed": false,
"name": "ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2",
"overprovisionfactor": "2.0",
"path":
"/acs/primary/ref-trl-9699-k-Mol8-rositsa-kyuchukova/ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2",
"podid": "0e13ddcc-d5ef-444a-a358-0c6ac6e9c744",
"podname": "Pod1",
"provider": "DefaultPrimary",
"scope": "CLUSTER",
"state": "Up",
"storagecapabilities": {
"VOLUME_SNAPSHOT_QUIESCEVM": "false"
},
"type": "NetworkFilesystem",
"zoneid": "36c95913-6c6d-400c-9d7e-12d8ac72fc35",
"zonename": "ref-trl-9699-k-Mol8-rositsa-kyuchukova"
}
]
}
```
</details>
- Trigger prepare for maintenance -> Pool transitions to
PrepareForMaintenance
<details><summary><code>(localcloud) 🐱 > enable storagemaintenance
id=5e33de91-2078-34b9-b954-c6646408bf99 </code></summary>
```json
{
"storagepool": {
"clusterid": "46169705-e70d-46ff-ad6d-1cb31fc6423c",
"clustername": "p1-c1",
"created": "2025-10-15T13:38:56+0000",
"disksizeallocated": 5243076688,
"disksizetotal": 2898029182976,
"disksizeused": 2161114087424,
"hasannotations": false,
"hypervisor": "KVM",
"id": "5e33de91-2078-34b9-b954-c6646408bf99",
"ipaddress": "10.0.32.4",
"istagarule": false,
"jobid": "6f934724-025d-44a5-9ad6-0c43f6fd8c2b",
"jobstatus": 0,
"managed": false,
"name": "ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2",
"overprovisionfactor": "2.0",
"path":
"/acs/primary/ref-trl-9699-k-Mol8-rositsa-kyuchukova/ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2",
"podid": "0e13ddcc-d5ef-444a-a358-0c6ac6e9c744",
"podname": "Pod1",
"provider": "DefaultPrimary",
"scope": "CLUSTER",
"state": "Maintenance",
"type": "NetworkFilesystem",
"zoneid": "36c95913-6c6d-400c-9d7e-12d8ac72fc35",
"zonename": "ref-trl-9699-k-Mol8-rositsa-kyuchukova"
}
}
```
</details>
- Verify Pool state after command is triggered
```
(localcloud) 🐱 > list storagepools id=5e33de91-2078-34b9-b954-c6646408bf99
filter=state
{
"count": 1,
"storagepool": [
{
"state": "Maintenance"
}
]
}
```
<img width="2522" height="883" alt="Screenshot from 2025-10-15 17-04-33"
src="https://github.com/user-attachments/assets/f249b8d1-3cb5-4bee-800a-b7b7ca1deec0";
/>
- Cancel maintenance
<details><summary><code>(localcloud) 🐱 > cancel storagemaintenance
id=5e33de91-2078-34b9-b954-c6646408bf99 </code></summary>
```json
{
"storagepool": {
"clusterid": "46169705-e70d-46ff-ad6d-1cb31fc6423c",
"clustername": "p1-c1",
"created": "2025-10-15T13:38:56+0000",
"disksizeallocated": 196688,
"disksizetotal": 2898029182976,
"disksizeused": 2168474042368,
"hasannotations": false,
"hypervisor": "KVM",
"id": "5e33de91-2078-34b9-b954-c6646408bf99",
"ipaddress": "10.0.32.4",
"istagarule": false,
"jobid": "ff7026cb-ee1f-4520-bfc4-2a854452d05b",
"jobstatus": 0,
"managed": false,
"name": "ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2",
"overprovisionfactor": "2.0",
"path":
"/acs/primary/ref-trl-9699-k-Mol8-rositsa-kyuchukova/ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2",
"podid": "0e13ddcc-d5ef-444a-a358-0c6ac6e9c744",
"podname": "Pod1",
"provider": "DefaultPrimary",
"scope": "CLUSTER",
"state": "Up",
"type": "NetworkFilesystem",
"zoneid": "36c95913-6c6d-400c-9d7e-12d8ac72fc35",
"zonename": "ref-trl-9699-k-Mol8-rositsa-kyuchukova"
}
}
```
</details>
- Verify Pool transitions back to Up
```
(localcloud) 🐱 > list storagepools id=5e33de91-2078-34b9-b954-c6646408bf99
filter=state
{
"count": 1,
"storagepool": [
{
"state": "Up"
}
]
}
```
<img width="2522" height="883" alt="image"
src="https://github.com/user-attachments/assets/df654bd4-3ff3-400c-b4ca-eb8dcab5ee42";
/>
- Destroy & recreate the CPVM
<details><summary><code>(localcloud) 🐱 > destroy systemvm
id=4fc85d47-094f-47f5-8eb9-ecd4fb0753a3 </code></summary>
```json
{
"systemvm": {
"arch": "x86_64",
"created": "2025-10-15T13:40:26+0000",
"dns1": "10.0.32.1",
"dns2": "8.8.8.8",
"hasannotations": false,
"hostcontrolstate": "Enabled",
"hostid": "21507531-193b-4f7d-a34c-e4d7f1e764e2",
"hostname": "ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm2",
"hypervisor": "KVM",
"id": "4fc85d47-094f-47f5-8eb9-ecd4fb0753a3",
"isdynamicallyscalable": false,
"name": "v-1-VM",
"podid": "0e13ddcc-d5ef-444a-a358-0c6ac6e9c744",
"podname": "Pod1",
"serviceofferingid": "0e9d510e-86ec-4b8d-971f-856c7a0ecbce",
"serviceofferingname": "System Offering For Console Proxy",
"state": "Running",
"systemvmtype": "consoleproxy",
"templateid": "33e2baf6-a9c9-11f0-9687-1e0035000318",
"templatename": "SystemVM Template (KVM)",
"zoneid": "36c95913-6c6d-400c-9d7e-12d8ac72fc35",
"zonename": "ref-trl-9699-k-Mol8-rositsa-kyuchukova"
}
}
```
</details>
<img width="2547" height="814" alt="Screenshot from 2025-10-15 17-38-30"
src="https://github.com/user-attachments/assets/dc93c0b8-f729-4627-8d72-18571bc4dc42";
/>
<img width="2547" height="814" alt="Screenshot from 2025-10-15 17-39-23"
src="https://github.com/user-attachments/assets/56c3685c-aafd-4b7e-8ea5-1413a0b0ca78";
/>
- CPVM volume confirmed on
`ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2`. CPVM successfully recreated
and reached `Running` state after maintenance cancel - pool operational.
```
[root@ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm1 ~]# virsh dumpxml v-3-VM |
grep "source file"
<source
file='/mnt/5e33de91-2078-34b9-b954-c6646408bf99/02095b34-2f4a-4665-b4fe-c29373325dfc'
index='2'/>
<source
file='/mnt/5e33de91-2078-34b9-b954-c6646408bf99/33e2baf6-a9c9-11f0-9687-1e0035000318'/>
[root@ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm1 ~]#
```
### 2.Host Heartbeat Removal / Re-Add
**Action:** Prepare & cancel maintenance on Pool, while monitoring logs.
**Expected Result:** Heartbeats removed on prepare
(ModifyStoragePoolCommand(false)) and re-added on cancel (true), no failures.
**Actual Result**
- add:false → heartbeat removal on prepare.
- add:true → heartbeat re-add on cancel.
- Commands sent successfully to both hosts (kvm1, kvm2).
- No errors or failures.
**Evidence**
```
2025-10-15 13:38:57,006 DEBUG [c.c.a.t.Request] (pool-11-thread-1:[])
(logid:) Seq 1-6179783113682452493: Sending { Cmd , MgmtId: 32986238026520,
via: 1(ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm1), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":"true","pool":{"id":"2","uuid":"5e33de91-2078-34b9-b954-c6646408bf99","host":"10.0.32.4","path":"/acs/primary/ref-trl-9699-k-Mol8-rositsa-kyuchukova/ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2","port":"2049","type":"NetworkFilesystem"},"localPath":"/mnt//5e33de91-2078-34b9-b954-c6646408bf99","wait":"300","bypassHostMaintenance":"false"}}]
}
2025-10-15 13:38:57,155 DEBUG [c.c.a.t.Request] (pool-11-thread-1:[])
(logid:) Seq 2-1762596304162127882: Sending { Cmd , MgmtId: 32986238026520,
via: 2(ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm2), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":"true","pool":{"id":"2","uuid":"5e33de91-2078-34b9-b954-c6646408bf99","host":"10.0.32.4","path":"/acs/primary/ref-trl-9699-k-Mol8-rositsa-kyuchukova/ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2","port":"2049","type":"NetworkFilesystem"},"localPath":"/mnt//5e33de91-2078-34b9-b954-c6646408bf99","wait":"300","bypassHostMaintenance":"false"}}]
}
2025-10-15 14:03:38,642 DEBUG [c.c.a.t.Request]
(API-Job-Executor-29:[ctx-1a84fa5f, job-35, ctx-80386708]) (logid:6f934724) Seq
1-6179783113682452572: Sending { Cmd , MgmtId: 32986238026520, via:
1(ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm1), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":"false","pool":{"id":"2","uuid":"5e33de91-2078-34b9-b954-c6646408bf99","host":"10.0.32.4","path":"/acs/primary/ref-trl-9699-k-Mol8-rositsa-kyuchukova/ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2","port":"2049","type":"NetworkFilesystem"},"localPath":"/mnt//5e33de91-2078-34b9-b954-c6646408bf99","wait":"0","bypassHostMaintenance":"false"}}]
}
2025-10-15 14:03:38,682 DEBUG [c.c.a.t.Request]
(API-Job-Executor-29:[ctx-1a84fa5f, job-35, ctx-80386708]) (logid:6f934724) Seq
2-1762596304162127959: Sending { Cmd , MgmtId: 32986238026520, via:
2(ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm2), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":"false","pool":{"id":"2","uuid":"5e33de91-2078-34b9-b954-c6646408bf99","host":"10.0.32.4","path":"/acs/primary/ref-trl-9699-k-Mol8-rositsa-kyuchukova/ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2","port":"2049","type":"NetworkFilesystem"},"localPath":"/mnt//5e33de91-2078-34b9-b954-c6646408bf99","wait":"0","bypassHostMaintenance":"false"}}]
}
2025-10-15 14:11:19,134 DEBUG [c.c.a.t.Request]
(API-Job-Executor-30:[ctx-f4e08cc2, job-38, ctx-8c9a2448]) (logid:ff7026cb) Seq
1-6179783113682452599: Sending { Cmd , MgmtId: 32986238026520, via:
1(ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm1), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":"true","pool":{"id":"2","uuid":"5e33de91-2078-34b9-b954-c6646408bf99","host":"10.0.32.4","path":"/acs/primary/ref-trl-9699-k-Mol8-rositsa-kyuchukova/ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2","port":"2049","type":"NetworkFilesystem"},"localPath":"/mnt//5e33de91-2078-34b9-b954-c6646408bf99","wait":"0","bypassHostMaintenance":"false"}}]
}
2025-10-15 14:11:19,225 DEBUG [c.c.a.t.Request]
(API-Job-Executor-30:[ctx-f4e08cc2, job-38, ctx-8c9a2448]) (logid:ff7026cb) Seq
2-1762596304162127986: Sending { Cmd , MgmtId: 32986238026520, via:
2(ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm2), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.ModifyStoragePoolCommand":{"add":"true","pool":{"id":"2","uuid":"5e33de91-2078-34b9-b954-c6646408bf99","host":"10.0.32.4","path":"/acs/primary/ref-trl-9699-k-Mol8-rositsa-kyuchukova/ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2","port":"2049","type":"NetworkFilesystem"},"localPath":"/mnt//5e33de91-2078-34b9-b954-c6646408bf99","wait":"0","bypassHostMaintenance":"false"}}]
}
```
### 3.VM Stop/Start Behavior (System & User VMs)
**Action:**
- Deploy a user VM while Primary Storage 1(pri1) is in maintenance, ensuring
CPVM, SSVM, and the user VM are running on Primary Storage 2 (pri2).
- Trigger maintenance on pri2 while no other pool is active to force system
and user VMs to stop.
- Re-enable pri1 and verify automatic redeployment of system VMs and manual
start of the user VM on pri1.
- Cancel maintenance on pri2 to restore normal pool availability.
**Expected Result:** VMs stop or migrate as expected, system VMs restart
automatically, no orphaned resources.
- All system and user VMs initially run on pri2.
- When pri2 enters maintenance with no alternate pool active, CPVM, SSVM,
and the user VM stop gracefully.
- After pri1 is re-enabled, system VMs are automatically redeployed on pri1,
and the user VM can be manually started without errors.
- All volumes now reside on pri1.
- No failures or stuck states are observed in management-server.log.
**Actual Result**
- With Primary Storage 1 (pri1) disabled, all system and user VMs were
confirmed to be using Primary Storage 2 (pri2) as their storage pool.
- When pri2 was put into maintenance, both system VMs (CPVM and SSVM) and
the user VM stopped cleanly without errors.
- After re-enabling pri1, system VMs were automatically redeployed on pri1
(verified via virsh dumpxml disk paths).
- The user VM was successfully started manually on pri1 while pri2 remained
in maintenance.
- Canceling maintenance on pri2 restored the pool to Up state without issues.
- No failures or unexpected behavior were observed in the management server
logs or UI.
- CPVM and SSVM were fully functional after redeploy and recreation,
confirming successful recovery flow.
**Evidence**
- only **pri2** is available
```
(localcloud) 🐱 > list storagepools filter=state,name,id
{
"count": 2,
"storagepool": [
{
"id": "cca423c8-b72e-36f2-8931-60434242a9ca",
"name": "ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri1",
"state": "Disabled"
},
{
"id": "5e33de91-2078-34b9-b954-c6646408bf99",
"name": "ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2",
"state": "Up"
}
]
}
```
- Storage Pool for CPVM is on **pri2** (**pri2 ID:**
5e33de91-2078-34b9-b954-c6646408bf99)
```
[root@ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm1 ~]# virsh dumpxml v-3-VM |
grep "source"
<source
file='/mnt/5e33de91-2078-34b9-b954-c6646408bf99/02095b34-2f4a-4665-b4fe-c29373325dfc'
index='2'/>
<source
file='/mnt/5e33de91-2078-34b9-b954-c6646408bf99/33e2baf6-a9c9-11f0-9687-1e0035000318'/>
```
- Storage Pool for SSVM is on **pri2** (**pri2 ID:**
5e33de91-2078-34b9-b954-c6646408bf99)
```
[root@ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm2 ~]# virsh dumpxml s-4-VM |
grep source
<source
file='/mnt/5e33de91-2078-34b9-b954-c6646408bf99/c7e1a2d2-0cf7-4f64-92dc-44085a2b81ee'
index='2'/>
<source
file='/mnt/5e33de91-2078-34b9-b954-c6646408bf99/33e2baf6-a9c9-11f0-9687-1e0035000318'/>
```
- Storage placement on the deployed VM is on **pri2** (**pri2 ID:**
5e33de91-2078-34b9-b954-c6646408bf99):
```
[root@ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm2 ~]# virsh dumpxml i-2-5-VM
| grep "source"
<source
file='/mnt/5e33de91-2078-34b9-b954-c6646408bf99/4093c1d5-7b51-4f31-b16c-465398d41b44'
index='2'/>
<source
file='/mnt/5e33de91-2078-34b9-b954-c6646408bf99/33e3a5c0-a9c9-11f0-9687-1e0035000318'/>
```
- Trigger maintenance on **pri2**
<details><summary><code>(localcloud) 🐱 > enable storagemaintenance
id=5e33de91-2078-34b9-b954-c6646408bf99 </code></summary>
```json
{
"storagepool": {
"clusterid": "46169705-e70d-46ff-ad6d-1cb31fc6423c",
"clustername": "p1-c1",
"created": "2025-10-15T13:38:56+0000",
"disksizeallocated": 24318968016,
"disksizetotal": 2898029182976,
"disksizeused": 2201851265024,
"hasannotations": false,
"hypervisor": "KVM",
"id": "5e33de91-2078-34b9-b954-c6646408bf99",
"ipaddress": "10.0.32.4",
"istagarule": false,
"jobid": "42925036-5491-47d1-819b-1388fb62c8e3",
"jobstatus": 0,
"managed": false,
"name": "ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2",
"overprovisionfactor": "2.0",
"path":
"/acs/primary/ref-trl-9699-k-Mol8-rositsa-kyuchukova/ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2",
"podid": "0e13ddcc-d5ef-444a-a358-0c6ac6e9c744",
"podname": "Pod1",
"provider": "DefaultPrimary",
"scope": "CLUSTER",
"state": "Maintenance",
"type": "NetworkFilesystem",
"zoneid": "36c95913-6c6d-400c-9d7e-12d8ac72fc35",
"zonename": "ref-trl-9699-k-Mol8-rositsa-kyuchukova"
}
}
```
</details>
<img width="2589" height="754" alt="image"
src="https://github.com/user-attachments/assets/30b943b2-6f18-4b96-8ffc-dff3e9936f3c";
/>
- User VM & System VMs are Stopped
<img width="2589" height="754" alt="Screenshot from 2025-10-15 22-15-50"
src="https://github.com/user-attachments/assets/4a29f689-d89c-4e97-8a77-e795a401c588";
/>
<img width="2559" height="509" alt="Screenshot from 2025-10-15 22-15-39"
src="https://github.com/user-attachments/assets/f7906d4a-cb23-45f1-9215-472d51638032";
/>
- Re-enable **pri1** so system VMs can redeploy
<img width="2591" height="787" alt="image"
src="https://github.com/user-attachments/assets/0c2b6fec-107e-42f1-8590-b000a4c15d0d";
/>
- System VMs start redeploying on **pri1**
**NOTE** Destroy and re-create them, if needed
<img width="2589" height="849" alt="image"
src="https://github.com/user-attachments/assets/b2431aa4-190c-418b-8e5a-348ab7846b46";
/>
- Verify Storage Pool for both System VMs is now **pri1**, since **pri2** is
in Maintenance (**pri1 ID:** cca423c8-b72e-36f2-8931-60434242a9ca)
```
[root@ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm2 ~]# virsh dumpxml s-15-VM
| grep source
<source
file='/mnt/cca423c8-b72e-36f2-8931-60434242a9ca/58a37398-2021-439f-8d27-36a7cac8b99b'
index='2'/>
<source
file='/mnt/cca423c8-b72e-36f2-8931-60434242a9ca/33e2baf6-a9c9-11f0-9687-1e0035000318'/>
```
and
```
[root@ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm1 ~]# virsh dumpxml v-14-VM
| grep source
<resource>
</resource>
<source
file='/mnt/cca423c8-b72e-36f2-8931-60434242a9ca/2cb750a3-8005-4998-ac5b-24913dab493f'
index='2'/>
<source
file='/mnt/cca423c8-b72e-36f2-8931-60434242a9ca/33e2baf6-a9c9-11f0-9687-1e0035000318'/>
```
- Manually start the User VM (test-vm-pri2)
<img width="2580" height="666" alt="image"
src="https://github.com/user-attachments/assets/97ed9de9-cfa1-4157-8edd-904759e73bb7";
/>
- VM starts on **pri1** (while **pri2** is still in Maintenance)
<img width="2580" height="666" alt="image"
src="https://github.com/user-attachments/assets/6bea1aba-819a-4cb3-b198-0196e0702a39";
/>
<img width="2324" height="918" alt="image"
src="https://github.com/user-attachments/assets/c183ef58-b217-4aa1-916d-1a4a146c0293";
/>
- Cancel maintenance on **pri2** (to restore baseline):
<details><summary><code>(localcloud) 🐱 > cancel storagemaintenance
id=5e33de91-2078-34b9-b954-c6646408bf99 </code></summary>
```json
{
"storagepool": {
"clusterid": "46169705-e70d-46ff-ad6d-1cb31fc6423c",
"clustername": "p1-c1",
"created": "2025-10-15T13:38:56+0000",
"disksizeallocated": 13833208016,
"disksizetotal": 2898029182976,
"disksizeused": 2208384942080,
"hasannotations": false,
"hypervisor": "KVM",
"id": "5e33de91-2078-34b9-b954-c6646408bf99",
"ipaddress": "10.0.32.4",
"istagarule": false,
"jobid": "17330882-91f9-4b90-a339-20b6fe15c80b",
"jobstatus": 0,
"managed": false,
"name": "ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2",
"overprovisionfactor": "2.0",
"path":
"/acs/primary/ref-trl-9699-k-Mol8-rositsa-kyuchukova/ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2",
"podid": "0e13ddcc-d5ef-444a-a358-0c6ac6e9c744",
"podname": "Pod1",
"provider": "DefaultPrimary",
"scope": "CLUSTER",
"state": "Up",
"type": "NetworkFilesystem",
"zoneid": "36c95913-6c6d-400c-9d7e-12d8ac72fc35",
"zonename": "ref-trl-9699-k-Mol8-rositsa-kyuchukova"
}
}
```
</details>
<img width="2563" height="819" alt="image"
src="https://github.com/user-attachments/assets/9f4ff2b7-e61b-4f32-afaf-39be361f31f8";
/>
### 4.VM-HA Scenario
**Action:** Enable HA on a user VM → prepare pool for maintenance.
**Expected Result:** VM automatically restarts on the other pool, stable
after cancel.
**Actual Result**
**Evidence**
1. Baseline:
- User VM is in Running state
```
(localcloud) 🐱 > list virtualmachines
id=4848b36a-bb8c-438b-a93e-82f613a8d6d0 filter=name,instancename,state,hostname
{
"count": 1,
"virtualmachine": [
{
"hostname": "ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm2",
"instancename": "i-2-16-VM",
"name": "test-vm-ha",
"state": "Running"
}
]
}
```
- HA enabled
<img width="1785" height="182" alt="image"
src="https://github.com/user-attachments/assets/5fe3614d-5a3f-4aec-87df-d834ad3a232c";
/>
- Primary Storage 1 and Primary Storage 2 are in Up state
```
(localcloud) 🐱 > list storagepools filter=name,id,state
{
"count": 2,
"storagepool": [
{
"id": "cca423c8-b72e-36f2-8931-60434242a9ca",
"name": "ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri1",
"state": "Up"
},
{
"id": "5e33de91-2078-34b9-b954-c6646408bf99",
"name": "ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm-pri2",
"state": "Up"
}
]
}
```
- User VM hosted on Primary Storage 1 (ID:
cca423c8-b72e-36f2-8931-60434242a9ca)
```
[root@ref-trl-9699-k-Mol8-rositsa-kyuchukova-kvm2 ~]# virsh dumpxml
i-2-16-VM | grep source
<source
file='/mnt/cca423c8-b72e-36f2-8931-60434242a9ca/a82f054a-8449-4e41-9dab-7aed00033820'
index='2'/>
<source
file='/mnt/cca423c8-b72e-36f2-8931-60434242a9ca/33e3a5c0-a9c9-11f0-9687-1e0035000318'/>
```
2. Enable Maintenance on the Host where the VM is running (KVM2) (?)
<img width="2625" height="975" alt="image"
src="https://github.com/user-attachments/assets/c19e8829-b82b-4a8b-9c25-7a204ac79389";
/>
3. Verify that the User VM without HA enabled - is Stopped, while the VM
with HA enabled - automatically starts on the other pool (pri2)
<img width="2594" height="900" alt="image"
src="https://github.com/user-attachments/assets/3c8645af-6905-46ef-94c3-1c57cbe6cb1f";
/>
<img width="2603" height="876" alt="image"
src="https://github.com/user-attachments/assets/169e8897-8c26-4557-bca1-d6184bb89460";
/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
