mbertolina opened a new issue, #14204: URL: https://github.com/apache/cloudstack/issues/14204
### problem **Description:** Not able to delete CLVM_NG primary storage. I am getting "Failed to delete storage pool on host" error message. All VMs and associated volumes, templates, etc were deleted from primary storage and it was pit under maintenance state. This may be related to: [https://github.com/apache/cloudstack/discussions/13392](url) <img width="580" height="254" alt="Image" src="https://github.com/user-attachments/assets/fb4dc6a0-6b91-4409-a99d-c39b2e68759e"; /> ``` # vgs VG #PV #LV #SN Attr VSize VFree hdd_raid6_vg 1 1 0 wz--n- <4.37t 0 rhel_vg 1 6 0 wz--n- <444.61g <100.61g ssd_raid1_vg 1 1 0 wz--n- 3.49t 0 ds_test_01_vg 1 0 0 wz--ns <4.00t <4.00t ds_test_02_vg 1 0 0 wz--ns <4.00t <4.00t ds_test_03_vg 1 0 0 wz--ns <4.00t <4.00t ds_test_04_vg 1 0 0 wz--ns <4.00t <4.00t ds_test_05_vg 1 0 0 wz--ns <4.00t <4.00t # ``` **Workaround:** Removed all shared VGs and restarted agent service on all nodes ``` # vgs VG #PV #LV #SN Attr VSize VFree hdd_raid6_vg 1 1 0 wz--n- <4.37t 0 rhel_vg 1 6 0 wz--n- <444.61g <100.61g ssd_raid1_vg 1 1 0 wz--n- 3.49t 0 # /usr/bin/systemctl restart cloudstack-agent.service # ``` **Logs from ACS server:** ``` 2026-09-18 13:40:56,985 DEBUG [c.c.a.ApiServlet] (qtp1390913202-47871:[ctx-90bdde62]) (logid:ac4f0fba) ===START=== 108.145.117.211 -- POST command=deleteStoragePool response=json id=0b35ce6c-7826-3188-b0bb-1150c455a5a3 forced=true sessionkey=Ln4wx0lPZhcXCec2jVuN6FNC5Sk 2026-09-18 13:40:56,985 DEBUG [c.c.a.ApiServlet] (qtp1390913202-47871:[ctx-90bdde62]) (logid:ac4f0fba) Two factor authentication is already verified for the user 2, so skipping 2026-09-18 13:40:56,988 DEBUG [c.c.a.ApiServer] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) CIDRs from which account 'Account [{"accountName":"admin","id":2,"uuid":"1b4fa8b0-8f51-11f1-ba8c-525400083ffc"}]' is allowed to perform API calls: 0.0.0.0/0,::/0 2026-09-18 13:40:56,989 INFO [o.a.c.a.DynamicRoleBasedAPIAccessChecker] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Account [Account [{"accountName":"admin","id":2,"uuid":"1b4fa8b0-8f51-11f1-ba8c-525400083ffc"}]] is Root Admin and there aren't any API key pair permissions involved, thus, all APIs are allowed. 2026-09-18 13:40:56,989 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) RoleService is enabled. We will use it instead of StaticRoleBasedAPIAccessChecker. 2026-09-18 13:40:56,989 DEBUG [o.a.c.r.ApiRateLimitServiceImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) API rate limiting is disabled. We will not use ApiRateLimitService. 2026-09-18 13:40:57,003 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Wait time setting on com.cloud.agent.api.DeleteStoragePoolCommand is 1800 seconds 2026-09-18 13:40:57,004 DEBUG [c.c.a.m.ClusteredAgentAttache] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 61-5210101818914280197: Routed from 90520734213782 2026-09-18 13:40:57,004 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 21-5210101818914280197: Sending { Cmd , MgmtId: 90520734213782, via: 21(esx0101), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.DeleteStoragePoolCommand":{"_pool":{"id":"74","uuid":"0b35ce6c-7826-3188-b0bb-1150c455a5a3","host":"localhost","path":"ds_test_01_vg","port":"0","type":"CLVM_NG"},"_localPath":"/mnt//4df4b702-17ad-35ae-8003-1286a2ffb702","_removeDatastore":"false","wait":"0","bypassHostMaintenance":"false"}}] } 2026-09-18 13:40:57,047 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 21-5210101818914280197: Received: { Ans: , MgmtId: 90520734213782, via: 21(esx0101), Ver: v1, Flags: 10, { Answer } } 2026-09-18 13:40:57,047 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Details from executing class com.cloud.agent.api.DeleteStoragePoolCommand: com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: Requested operation is not valid: storage pool 'ds_test_01_vg' is not active 2026-09-18 13:40:57,047 DEBUG [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Failed to delete storage pool: false 2026-09-18 13:40:57,047 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Wait time setting on com.cloud.agent.api.DeleteStoragePoolCommand is 1800 seconds 2026-09-18 13:40:57,048 DEBUG [c.c.a.m.ClusteredAgentAttache] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 61-1204994375298338227: Routed from 90520734213782 2026-09-18 13:40:57,048 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 22-1204994375298338227: Sending { Cmd , MgmtId: 90520734213782, via: 22(esx0102), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.DeleteStoragePoolCommand":{"_pool":{"id":"74","uuid":"0b35ce6c-7826-3188-b0bb-1150c455a5a3","host":"localhost","path":"ds_test_01_vg","port":"0","type":"CLVM_NG"},"_localPath":"/mnt//4df4b702-17ad-35ae-8003-1286a2ffb702","_removeDatastore":"false","wait":"0","bypassHostMaintenance":"false"}}] } 2026-09-18 13:40:57,050 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 22-1204994375298338227: Received: { Ans: , MgmtId: 90520734213782, via: 22(esx0102), Ver: v1, Flags: 10, { Answer } } 2026-09-18 13:40:57,050 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Details from executing class com.cloud.agent.api.DeleteStoragePoolCommand: com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: Requested operation is not valid: storage pool 'ds_test_01_vg' is not active 2026-09-18 13:40:57,050 DEBUG [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Failed to delete storage pool: false 2026-09-18 13:40:57,051 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Wait time setting on com.cloud.agent.api.DeleteStoragePoolCommand is 1800 seconds 2026-09-18 13:40:57,051 DEBUG [c.c.a.m.ClusteredAgentAttache] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 61-6906832978526096897: Routed from 90520734213782 2026-09-18 13:40:57,051 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 23-6906832978526096897: Sending { Cmd , MgmtId: 90520734213782, via: 23(esx0103), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.DeleteStoragePoolCommand":{"_pool":{"id":"74","uuid":"0b35ce6c-7826-3188-b0bb-1150c455a5a3","host":"localhost","path":"ds_test_01_vg","port":"0","type":"CLVM_NG"},"_localPath":"/mnt//4df4b702-17ad-35ae-8003-1286a2ffb702","_removeDatastore":"false","wait":"0","bypassHostMaintenance":"false"}}] } 2026-09-18 13:40:57,054 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 23-6906832978526096897: Received: { Ans: , MgmtId: 90520734213782, via: 23(esx0103), Ver: v1, Flags: 10, { Answer } } 2026-09-18 13:40:57,054 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Details from executing class com.cloud.agent.api.DeleteStoragePoolCommand: com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: Requested operation is not valid: storage pool 'ds_test_01_vg' is not active 2026-09-18 13:40:57,054 DEBUG [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Failed to delete storage pool: false 2026-09-18 13:40:57,054 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Wait time setting on com.cloud.agent.api.DeleteStoragePoolCommand is 1800 seconds 2026-09-18 13:40:57,055 DEBUG [c.c.a.m.ClusteredAgentAttache] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 61-5033617008516681232: Routed from 90520734213782 2026-09-18 13:40:57,055 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 24-5033617008516681232: Sending { Cmd , MgmtId: 90520734213782, via: 24(esx0104), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.DeleteStoragePoolCommand":{"_pool":{"id":"74","uuid":"0b35ce6c-7826-3188-b0bb-1150c455a5a3","host":"localhost","path":"ds_test_01_vg","port":"0","type":"CLVM_NG"},"_localPath":"/mnt//4df4b702-17ad-35ae-8003-1286a2ffb702","_removeDatastore":"false","wait":"0","bypassHostMaintenance":"false"}}] } 2026-09-18 13:40:57,057 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 24-5033617008516681232: Received: { Ans: , MgmtId: 90520734213782, via: 24(esx0104), Ver: v1, Flags: 10, { Answer } } 2026-09-18 13:40:57,057 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Details from executing class com.cloud.agent.api.DeleteStoragePoolCommand: com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: Requested operation is not valid: storage pool 'ds_test_01_vg' is not active 2026-09-18 13:40:57,057 DEBUG [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Failed to delete storage pool: false 2026-09-18 13:40:57,058 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Wait time setting on com.cloud.agent.api.DeleteStoragePoolCommand is 1800 seconds 2026-09-18 13:40:57,058 DEBUG [c.c.a.m.ClusteredAgentAttache] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 61-1866460570568379857: Routed from 90520734213782 2026-09-18 13:40:57,058 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 25-1866460570568379857: Sending { Cmd , MgmtId: 90520734213782, via: 25(esx0307), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.DeleteStoragePoolCommand":{"_pool":{"id":"74","uuid":"0b35ce6c-7826-3188-b0bb-1150c455a5a3","host":"localhost","path":"ds_test_01_vg","port":"0","type":"CLVM_NG"},"_localPath":"/mnt//4df4b702-17ad-35ae-8003-1286a2ffb702","_removeDatastore":"false","wait":"0","bypassHostMaintenance":"false"}}] } 2026-09-18 13:40:57,101 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 25-1866460570568379857: Received: { Ans: , MgmtId: 90520734213782, via: 25(esx0307), Ver: v1, Flags: 10, { Answer } } 2026-09-18 13:40:57,101 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Details from executing class com.cloud.agent.api.DeleteStoragePoolCommand: com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: Requested operation is not valid: storage pool 'ds_test_01_vg' is not active 2026-09-18 13:40:57,101 DEBUG [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Failed to delete storage pool: false 2026-09-18 13:40:57,101 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Wait time setting on com.cloud.agent.api.DeleteStoragePoolCommand is 1800 seconds 2026-09-18 13:40:57,101 DEBUG [c.c.a.m.ClusteredAgentAttache] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 61-971088669651783173: Routed from 90520734213782 2026-09-18 13:40:57,102 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 26-971088669651783173: Sending { Cmd , MgmtId: 90520734213782, via: 26(esx0308), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.DeleteStoragePoolCommand":{"_pool":{"id":"74","uuid":"0b35ce6c-7826-3188-b0bb-1150c455a5a3","host":"localhost","path":"ds_test_01_vg","port":"0","type":"CLVM_NG"},"_localPath":"/mnt//4df4b702-17ad-35ae-8003-1286a2ffb702","_removeDatastore":"false","wait":"0","bypassHostMaintenance":"false"}}] } 2026-09-18 13:40:57,145 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 26-971088669651783173: Received: { Ans: , MgmtId: 90520734213782, via: 26(esx0308), Ver: v1, Flags: 10, { Answer } } 2026-09-18 13:40:57,145 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Details from executing class com.cloud.agent.api.DeleteStoragePoolCommand: com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: Requested operation is not valid: storage pool 'ds_test_01_vg' is not active 2026-09-18 13:40:57,145 DEBUG [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Failed to delete storage pool: false 2026-09-18 13:40:57,145 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Wait time setting on com.cloud.agent.api.DeleteStoragePoolCommand is 1800 seconds 2026-09-18 13:40:57,146 DEBUG [c.c.a.m.ClusteredAgentAttache] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 61-3703084793605410297: Routed from 90520734213782 2026-09-18 13:40:57,146 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 27-3703084793605410297: Sending { Cmd , MgmtId: 90520734213782, via: 27(esx0401), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.DeleteStoragePoolCommand":{"_pool":{"id":"74","uuid":"0b35ce6c-7826-3188-b0bb-1150c455a5a3","host":"localhost","path":"ds_test_01_vg","port":"0","type":"CLVM_NG"},"_localPath":"/mnt//4df4b702-17ad-35ae-8003-1286a2ffb702","_removeDatastore":"false","wait":"0","bypassHostMaintenance":"false"}}] } 2026-09-18 13:40:57,148 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 27-3703084793605410297: Received: { Ans: , MgmtId: 90520734213782, via: 27(esx0401), Ver: v1, Flags: 10, { Answer } } 2026-09-18 13:40:57,148 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Details from executing class com.cloud.agent.api.DeleteStoragePoolCommand: com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: Requested operation is not valid: storage pool 'ds_test_01_vg' is not active 2026-09-18 13:40:57,148 DEBUG [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Failed to delete storage pool: false 2026-09-18 13:40:57,149 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Wait time setting on com.cloud.agent.api.DeleteStoragePoolCommand is 1800 seconds 2026-09-18 13:40:57,149 DEBUG [c.c.a.m.ClusteredAgentAttache] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 61-385902193070329372: Routed from 90520734213782 2026-09-18 13:40:57,149 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 28-385902193070329372: Sending { Cmd , MgmtId: 90520734213782, via: 28(esx0402), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.DeleteStoragePoolCommand":{"_pool":{"id":"74","uuid":"0b35ce6c-7826-3188-b0bb-1150c455a5a3","host":"localhost","path":"ds_test_01_vg","port":"0","type":"CLVM_NG"},"_localPath":"/mnt//4df4b702-17ad-35ae-8003-1286a2ffb702","_removeDatastore":"false","wait":"0","bypassHostMaintenance":"false"}}] } 2026-09-18 13:40:57,193 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 28-385902193070329372: Received: { Ans: , MgmtId: 90520734213782, via: 28(esx0402), Ver: v1, Flags: 10, { Answer } } 2026-09-18 13:40:57,193 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Details from executing class com.cloud.agent.api.DeleteStoragePoolCommand: com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: Requested operation is not valid: storage pool 'ds_test_01_vg' is not active 2026-09-18 13:40:57,193 DEBUG [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Failed to delete storage pool: false 2026-09-18 13:40:57,193 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Wait time setting on com.cloud.agent.api.DeleteStoragePoolCommand is 1800 seconds 2026-09-18 13:40:57,194 DEBUG [c.c.a.m.ClusteredAgentAttache] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 61-4814910951612501578: Routed from 90520734213782 2026-09-18 13:40:57,194 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 29-4814910951612501578: Sending { Cmd , MgmtId: 90520734213782, via: 29(esx0403), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.DeleteStoragePoolCommand":{"_pool":{"id":"74","uuid":"0b35ce6c-7826-3188-b0bb-1150c455a5a3","host":"localhost","path":"ds_test_01_vg","port":"0","type":"CLVM_NG"},"_localPath":"/mnt//4df4b702-17ad-35ae-8003-1286a2ffb702","_removeDatastore":"false","wait":"0","bypassHostMaintenance":"false"}}] } 2026-09-18 13:40:57,237 DEBUG [c.c.a.t.Request] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Seq 29-4814910951612501578: Received: { Ans: , MgmtId: 90520734213782, via: 29(esx0403), Ver: v1, Flags: 10, { Answer } } 2026-09-18 13:40:57,237 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Details from executing class com.cloud.agent.api.DeleteStoragePoolCommand: com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: Requested operation is not valid: storage pool 'ds_test_01_vg' is not active 2026-09-18 13:40:57,237 DEBUG [o.a.c.s.d.l.CloudStackPrimaryDataStoreLifeCycleImpl] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) Failed to delete storage pool: false 2026-09-18 13:40:57,237 ERROR [c.c.a.ApiServer] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) unhandled exception executing api command: [Ljava.lang.String;@112cb1cd com.cloud.utils.exception.CloudRuntimeException: Failed to delete storage pool on host at org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.deleteDataStore(CloudStackPrimaryDataStoreLifeCycleImpl.java:520) at com.cloud.storage.StorageManagerImpl.deleteDataStoreInternal(StorageManagerImpl.java:1808) at com.cloud.storage.StorageManagerImpl.deletePool(StorageManagerImpl.java:1711) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) at jdk.proxy3/jdk.proxy3.$Proxy140.deletePool(Unknown Source) at org.apache.cloudstack.api.command.admin.storage.DeletePoolCmd.execute(DeletePoolCmd.java:71) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:173) at com.cloud.api.ApiServer.queueCommand(ApiServer.java:820) at com.cloud.api.ApiServer.handleRequest(ApiServer.java:705) at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:414) at com.cloud.api.ApiServlet$1.run(ApiServlet.java:191) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:188) at com.cloud.api.ApiServlet.doPost(ApiServlet.java:147) at javax.servlet.http.HttpServlet.service(HttpServlet.java:665) at javax.servlet.http.HttpServlet.service(HttpServlet.java:750) at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1450) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:554) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:516) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base/java.lang.Thread.run(Thread.java:840) 2026-09-18 13:40:57,238 DEBUG [c.c.a.ApiServlet] (qtp1390913202-47871:[ctx-90bdde62, ctx-9aa46af6]) (logid:ac4f0fba) ===END=== 108.145.117.211 -- POST command=deleteStoragePool response=json id=0b35ce6c-7826-3188-b0bb-1150c455a5a3 forced=true sessionkey=Ln4wx0lPZhcXCec2jVuN6FNC5Sk 2026-09-18 13:40:57,701 INFO [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:[ctx-5d77e301]) (logid:c0016586) Found 1 inactive management server node based on timestamp 2026-09-18 14:08:38,906 DEBUG [cloud.agent.Agent] (AgentRequest-Handler-5:[]) (logid:f7ddf00d) Request:Seq 21-5210101818914280260: { Cmd , MgmtId: 90520734213782, via: 21, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.DeleteStoragePoolCommand":{"_pool":{"id":"74","uuid":"0b35ce6c-7826-3188-b0bb-1150c455a5a3","host":"localhost","path":"ds_test_01_vg","port":"0","type":"CLVM_NG"},"_localPath":"/mnt//4df4b702-17ad-35ae-8003-1286a2ffb702","_removeDatastore":"false","wait":"0","bypassHostMaintenance":"false"}}] } 2026-09-18 14:08:38,906 DEBUG [cloud.agent.Agent] (AgentRequest-Handler-5:[]) (logid:f7ddf00d) Processing command: com.cloud.agent.api.DeleteStoragePoolCommand 2026-09-18 14:08:38,906 INFO [kvm.storage.ClvmStorageAdaptor] (AgentRequest-Handler-5:[]) (logid:f7ddf00d) Attempting to remove storage pool 0b35ce6c-7826-3188-b0bb-1150c455a5a3 from libvirt 2026-09-18 14:08:38,906 DEBUG [kvm.resource.LibvirtConnection] (AgentRequest-Handler-5:[]) (logid:f7ddf00d) Looking for libvirtd connection at: qemu:///system 2026-09-18 14:08:38,907 INFO [kvm.storage.ClvmStorageAdaptor] (AgentRequest-Handler-5:[]) (logid:f7ddf00d) Storage pool 0b35ce6c-7826-3188-b0bb-1150c455a5a3 has no corresponding secret. Not removing any secret. ``` ### versions The versions of ACS, hypervisors, storage, network etc.. ### The steps to reproduce the bug 1. 2. 3. ... ### What to do about it? _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
