slavkap opened a new issue, #11764:
URL: https://github.com/apache/cloudstack/issues/11764
### problem
Zone-wide storage:
If there are virtual machines on specific primary storage and that storage
is set to Maintenance mode it is not possible to cancel the maintenance mode if
the agents in the zone were restarted.
Cluster-wide storage pool:
If there are virtual machines on such pool, and the pool is set to
maintenance and after that if we try to cancel the maintenance mode it's not
possible
### versions
4.21
Zone-wide primary storage - StorPool and NFS
Cluster-wide primary storage - NFS
### The steps to reproduce the bug
All VMs are created with service offering that have storage tags. I haven't
tested it without tags
Zone-wide pool:
- deploy a VM
- enable maintenance mode of the storage pool
- force reconnect all host in the zone from the UI (or restart the
cloudstack-agent service)
- cancle maintenance mode of the storage pool
Cluster-wide
- deploy a VM
- enable maintenance mode
- cancle maintenance mode
```
2025-09-30 14:09:42,578 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) Found pools [[]] that match with tags [[nfs-c
luster2]].
2025-09-30 14:09:42,578 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) No storage pools available for [shared] volum
e allocation.
2025-09-30 14:09:42,578 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) Using volume allocation algorithm random to r
eorder pools.
2025-09-30 14:09:42,578 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) Trying to find suitable pools to allocate volume [Vo
lume
{"id":562,"instanceId":380,"name":"ROOT-380","uuid":"bb7f3c0b-19f8-4be1-a6ce-bf29cff9a0ff","volumeType":"ROOT"}]
necessary to deploy VM [VM instance
{"id":380,"instanceName":"i-2-380-VM","state":"Startin
g","type":"User","uuid":"f4f66160-a169-4b4a-8729-df528c516c33"}], using
StoragePoolAllocator: [ZoneWideStoragePoolAllocator].
2025-09-30 14:09:42,580 DEBUG [o.a.c.s.a.ZoneWideStoragePoolAllocator]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) Could not find any zone wide storage pool that ma
tched with any of the following tags [[nfs-cluster2]].
2025-09-30 14:09:42,581 DEBUG [o.a.c.s.a.ZoneWideStoragePoolAllocator]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) ZoneWideStoragePoolAllocator is returning [0] sui
table storage pools [[]].
2025-09-30 14:09:42,581 DEBUG [o.a.c.s.a.ZoneWideStoragePoolAllocator]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) Using volume allocation algorithm random to reord
er pools.
2025-09-30 14:09:42,581 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) No suitable pools found for volume [Volume {"id":562
,"instanceId":380,"name":"ROOT-380","uuid":"bb7f3c0b-19f8-4be1-a6ce-bf29cff9a0ff","volumeType":"ROOT"}]
used by VM [VM instance
{"id":380,"instanceName":"i-2-380-VM","state":"Starting","type":"User","uuid":"f
4f66160-a169-4b4a-8729-df528c516c33"}] under cluster: [3].
2025-09-30 14:09:42,581 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) No suitable pools found
2025-09-30 14:09:42,581 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) No suitable storagePools found under this Cluster: C
luster {id: "3", name: "Cluster-A-2", uuid:
"bd8f0dce-1699-4879-8804-04713a1a87ee"}
2025-09-30 14:09:42,583 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) Could not find suitable Deployment Destination for t
his VM under any clusters, returning.
2025-09-30 14:09:42,584 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) Searching resources only under specified Pod: HostPod
{"id":1,"nam
e":"Pod-A","uuid":"f1cb86a7-ea4f-4211-a104-dd501578d7e4"}
2025-09-30 14:09:42,584 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) Listing clusters in order of aggregate capacity, that have (at
lea
st one host with) enough CPU and RAM capacity under this Pod: 1
2025-09-30 14:09:42,585 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) CapacityType: CPU is used for Cluster ordering
2025-09-30 14:09:42,586 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) Removing from the clusterId list these clusters from avoid
set: [1
, 3]
2025-09-30 14:09:42,587 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) No clusters found after removing disabled clusters and
clusters in
avoid list, returning.
2025-09-30 14:09:42,594 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) VM instance {"id":380,"instanceName":"i-2-380-VM","state":"Sto
pped","type":"User","uuid":"f4f66160-a169-4b4a-8729-df528c516c33"} state
transited from [Starting] to [Stopped] with event [OperationFailed]. VM's
original host: Host {"id":7,"name":"cs-dev-kvm-3-4","type":"R
outing","uuid":"e078cdb7-8819-43b2-a2c9-262788ad6099"}, new host: null, host
before state transition: null
2025-09-30 14:09:42,598 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) Invocation exception, caused by: com.cloud.exception.Insuffi
cientServerCapacityException: Unable to create a deployment for VM instance
{"id":380,"instanceName":"i-2-380-VM","state":"Starting","type":"User","uuid":"f4f66160-a169-4b4a-8729-df528c516c33"}Scope=interface
com.cloud.dc.DataCenter; id=1
2025-09-30 14:09:42,599 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445, ctx-59178917])
(logid:51ee9166) Rethrow exception com.cloud.exception.InsufficientServerCapa
cityException: Unable to create a deployment for VM instance
{"id":380,"instanceName":"i-2-380-VM","state":"Starting","type":"User","uuid":"f4f66160-a169-4b4a-8729-df528c516c33"}Scope=interface
com.cloud.dc.D
ataCenter; id=1
2025-09-30 14:09:42,599 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445]) (logid:51ee9166) Done
with run of VM work job: com.cloud.vm.VmWorkStart for VM 380, job origi
n: 5444
2025-09-30 14:09:42,599 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-15:[ctx-6b4bcd04, job-5444/job-5445]) (logid:51ee9166)
Unable to complete AsyncJob {"accountId":2,"cmd":"com.cloud.vm.VmWorkStart",
"cmdInfo":"rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zd
ElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkc
QB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAABfHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwc
HA","cmdVersion":0,"completeMsid":null,"created":"Tue Sep 30 14:09:41 UTC
2025","id":5445,"initMsid":2199191191555,"instanceId":null,"instanceType":null,"lastPolled":null,"lastUpdated":null,"processStatus":0,
"removed":null,"result":null,"resultCode":0,"status":"IN_PROGRESS","userId":2,"uuid":"68e14887-3529-4f40-a73d-1960007af720"},
job origin: 5444 com.cloud.exception.InsufficientServerCapacityException:
Unable t
o create a deployment for VM instance
{"id":380,"instanceName":"i-2-380-VM","state":"Starting","type":"User","uuid":"f4f66160-a169-4b4a-8729-df528c516c33"}Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1451)
```
### What to do about it?
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
