winterhazel commented on issue #9538:
URL: https://github.com/apache/cloudstack/issues/9538#issuecomment-3378922646
@abh1sar looking at the agent's logs, it seems that the `cp` in
`managesnapshot.sh` fails because the directory
`/mnt/29506a35-fd13-3d28-9ea5-f38d05dbfe5a/snapshots/2/75` does not exist,
which is weird as there is a `mkdir` right before.
```
2025-10-07 21:59:49,730 DEBUG
[storage.resource.StorageSubsystemCommandHandlerBase]
(agentRequest-Handler-3:[]) (logid:119c7f19) Executing command CopyCommand:
[{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"/mnt/8f989a84-e691-3a8f-bbcf-14ec8b839ce5/snapshots/fc6bec25-28ff-483a-99da-f6917542c4da","volume":{"uuid":"b568488d-1b5c-49e2-b3e6-19a1a764cbb8","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"8f989a84-e691-3a8f-bbcf-14ec8b839ce5","name":"pri","id":1,"poolType":"NetworkFilesystem","host":"192.168.122.10","path":"/export/primary","port":2049,"url":"NetworkFilesystem://192.168.122.10/export/primary/?ROLE=Primary&STOREUUID=8f989a84-e691-3a8f-bbcf-14ec8b839ce5","isManaged":false}},"name":"ROOT-74","size":52428800,"path":"b568488d-1b5c-49e2-b3e6-19a1a764cbb8","volumeId":75,"vmName":"i-2-74-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","poolId":1,"id":75,"deviceId":0,"hypervisorType":"KVM","direc
tDownload":false,"deployAsIs":false,"followRedirects":false},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"8f989a84-e691-3a8f-bbcf-14ec8b839ce5","name":"pri","id":1,"poolType":"NetworkFilesystem","host":"192.168.122.10","path":"/export/primary","port":2049,"url":"NetworkFilesystem://192.168.122.10/export/primary/?ROLE=Primary&STOREUUID=8f989a84-e691-3a8f-bbcf-14ec8b839ce5","isManaged":false}},"vmName":"i-2-74-VM","name":"a","hypervisorType":"KVM","id":2,"quiescevm":false,"physicalSize":0,"accountId":2,"followRedirects":false}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/2/75","volume":{"uuid":"b568488d-1b5c-49e2-b3e6-19a1a764cbb8","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"8f989a84-e691-3a8f-bbcf-14ec8b839ce5","name":"pri","id":1,"poolType":"NetworkFilesystem","host":"192.168.122.10","path":"/export/primary","port":2049,"url":"NetworkFilesystem://192.168.122.10/exp
ort/primary/?ROLE=Primary&STOREUUID=8f989a84-e691-3a8f-bbcf-14ec8b839ce5","isManaged":false}},"name":"ROOT-74","size":52428800,"path":"b568488d-1b5c-49e2-b3e6-19a1a764cbb8","volumeId":75,"vmName":"i-2-74-VM","accountId":2,"format":"QCOW2","provisioningType":"THIN","poolId":1,"id":75,"deviceId":0,"hypervisorType":"KVM","directDownload":false,"deployAsIs":false,"followRedirects":false},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://192.168.122.10/export/secondary","_role":"Image"}},"vmName":"i-2-74-VM","name":"a","hypervisorType":"KVM","id":2,"quiescevm":false,"physicalSize":0,"accountId":2,"followRedirects":false}},"executeInSequence":false,"options":{"snapshot.backup.to.secondary":"false","fullSnapshot":"true"},"options2":{},"wait":21600,"bypassHostMaintenance":false}].
2025-10-07 21:59:49,731 DEBUG [agent.properties.AgentPropertiesFileHandler]
(agentRequest-Handler-3:[]) (logid:119c7f19) Property [hypervisor.uri] has
empty or null value. Using default value [null].
2025-10-07 21:59:49,731 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-3:[]) (logid:119c7f19) Looking for libvirtd connection
at: qemu:///system
2025-10-07 21:59:49,739 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-3:[]) (logid:119c7f19) Can not find KVM connection for
Instance: i-2-74-VM, continuing.
2025-10-07 21:59:49,740 DEBUG [agent.properties.AgentPropertiesFileHandler]
(agentRequest-Handler-3:[]) (logid:119c7f19) Property [hypervisor.uri] has
empty or null value. Using default value [null].
2025-10-07 21:59:49,740 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-3:[]) (logid:119c7f19) Looking for libvirtd connection
at: lxc:///
2025-10-07 21:59:49,740 INFO [kvm.resource.LibvirtConnection]
(agentRequest-Handler-3:[]) (logid:119c7f19) No existing libvirtd connection
found. Opening a new one
2025-10-07 21:59:49,741 DEBUG [agent.properties.AgentPropertiesFileHandler]
(agentRequest-Handler-3:[]) (logid:119c7f19) Property [libvirt.events.enabled]
has empty or null value. Using default value [false].
2025-10-07 21:59:49,741 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-3:[]) (logid:119c7f19) Libvirt event listening is
disabled, not setting up event loop
2025-10-07 21:59:49,745 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-3:[]) (logid:119c7f19) Can not find LXC connection for
Instance: i-2-74-VM, continuing.
2025-10-07 21:59:49,745 WARN [kvm.resource.LibvirtConnection]
(agentRequest-Handler-3:[]) (logid:119c7f19) Can not find a connection for
Instance i-2-74-VM. Assuming the default connection.
2025-10-07 21:59:49,745 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-3:[]) (logid:119c7f19) Looking for libvirtd connection
at: qemu:///system
2025-10-07 21:59:49,748 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Attempting to create storage pool
29506a35-fd13-3d28-9ea5-f38d05dbfe5a (NetworkFilesystem) in libvirt
2025-10-07 21:59:49,751 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-3:[]) (logid:119c7f19) Looking for libvirtd connection
at: qemu:///system
2025-10-07 21:59:49,755 WARN [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Storage pool
29506a35-fd13-3d28-9ea5-f38d05dbfe5a was not found running in libvirt. Need to
create it.
2025-10-07 21:59:49,755 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Didn't find an existing storage
pool 29506a35-fd13-3d28-9ea5-f38d05dbfe5a by UUID, checking for pools with
duplicate paths
2025-10-07 21:59:49,758 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Checking path of existing pool
e409d730-8af7-4815-a35a-64a477e5aedf against pool we want to create
2025-10-07 21:59:49,772 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Checking path of existing pool
8f989a84-e691-3a8f-bbcf-14ec8b839ce5 against pool we want to create
2025-10-07 21:59:49,796 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Checking path of existing pool
c08f4f65-7793-3fe4-bb4a-8207dbb34b8b against pool we want to create
2025-10-07 21:59:49,836 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Checking path of existing pool
cc9d9a67-4b15-32b1-b401-f432ffcd85d1 against pool we want to create
2025-10-07 21:59:49,852 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Attempting to create storage pool
29506a35-fd13-3d28-9ea5-f38d05dbfe5a
2025-10-07 21:59:49,852 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:[]) (logid:119c7f19) <pool type='netfs'
xmlns:fs='http://libvirt.org/schemas/storagepool/fs/1.0'>
2025-10-07 21:59:49,853 DEBUG [utils.script.Script]
(agentRequest-Handler-3:[]) (logid:119c7f19) Executing command [/bin/bash -c
mountpoint -q /mnt/29506a35-fd13-3d28-9ea5-f38d05dbfe5a ].
2025-10-07 21:59:49,871 WARN [utils.script.Script]
(agentRequest-Handler-3:[]) (logid:119c7f19) Execution of process [13173] for
command [/bin/bash -c mountpoint -q /mnt/29506a35-fd13-3d28-9ea5-f38d05dbfe5a ]
failed.
2025-10-07 21:59:49,871 DEBUG [utils.script.Script]
(agentRequest-Handler-3:[]) (logid:119c7f19) Exit value of process [13173] for
command [/bin/bash -c mountpoint -q /mnt/29506a35-fd13-3d28-9ea5-f38d05dbfe5a ]
is [32].
2025-10-07 21:59:49,871 WARN [utils.script.Script]
(agentRequest-Handler-3:[]) (logid:119c7f19) Process [13173] for command
[/bin/bash -c mountpoint -q /mnt/29506a35-fd13-3d28-9ea5-f38d05dbfe5a ]
encountered the error: [32].
2025-10-07 21:59:49,947 DEBUG [utils.script.Script]
(agentRequest-Handler-3:[]) (logid:119c7f19) Executing command [/bin/bash -c
mountpoint -q /mnt/29506a35-fd13-3d28-9ea5-f38d05dbfe5a ].
2025-10-07 21:59:49,963 DEBUG [utils.script.Script]
(agentRequest-Handler-3:[]) (logid:119c7f19) Successfully executed process
[13180] for command [/bin/bash -c mountpoint -q
/mnt/29506a35-fd13-3d28-9ea5-f38d05dbfe5a ].
2025-10-07 21:59:49,964 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Trying to fetch storage pool
29506a35-fd13-3d28-9ea5-f38d05dbfe5a from libvirt
2025-10-07 21:59:49,964 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-3:[]) (logid:119c7f19) Looking for libvirtd connection
at: qemu:///system
2025-10-07 21:59:49,984 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Successfully refreshed pool
29506a35-fd13-3d28-9ea5-f38d05dbfe5a Capacity: (96.88 GB) 104028176384 Used:
(44.57 GB) 47854911488 Available: (52.32 GB) 56173264896
2025-10-07 21:59:49,985 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Trying to fetch storage pool
8f989a84-e691-3a8f-bbcf-14ec8b839ce5 from libvirt
2025-10-07 21:59:49,985 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-3:[]) (logid:119c7f19) Looking for libvirtd connection
at: qemu:///system
2025-10-07 21:59:50,012 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Successfully refreshed pool
8f989a84-e691-3a8f-bbcf-14ec8b839ce5 Capacity: (96.88 GB) 104028176384 Used:
(44.52 GB) 47801434112 Available: (52.37 GB) 56226742272
2025-10-07 21:59:50,029 DEBUG [kvm.storage.KVMStorageProcessor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Executing command
[/usr/share/cloudstack-common/scripts/storage/qcow2/managesnapshot.sh -b
/mnt/8f989a84-e691-3a8f-bbcf-14ec8b839ce5/snapshots/fc6bec25-28ff-483a-99da-f6917542c4da
-n fc6bec25-28ff-483a-99da-f6917542c4da -p
/mnt/29506a35-fd13-3d28-9ea5-f38d05dbfe5a/snapshots/2/75 -t
2c7300fd-c557-4923-a8cc-480ad733c612 ].
2025-10-07 21:59:50,077 WARN [kvm.storage.KVMStorageProcessor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Execution of process [13181] for
command [/usr/share/cloudstack-common/scripts/storage/qcow2/managesnapshot.sh
-b
/mnt/8f989a84-e691-3a8f-bbcf-14ec8b839ce5/snapshots/fc6bec25-28ff-483a-99da-f6917542c4da
-n fc6bec25-28ff-483a-99da-f6917542c4da -p
/mnt/29506a35-fd13-3d28-9ea5-f38d05dbfe5a/snapshots/2/75 -t
2c7300fd-c557-4923-a8cc-480ad733c612 ] failed.
2025-10-07 21:59:50,077 DEBUG [kvm.storage.KVMStorageProcessor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Exit value of process [13181] for
command [/usr/share/cloudstack-common/scripts/storage/qcow2/managesnapshot.sh
-b
/mnt/8f989a84-e691-3a8f-bbcf-14ec8b839ce5/snapshots/fc6bec25-28ff-483a-99da-f6917542c4da
-n fc6bec25-28ff-483a-99da-f6917542c4da -p
/mnt/29506a35-fd13-3d28-9ea5-f38d05dbfe5a/snapshots/2/75 -t
2c7300fd-c557-4923-a8cc-480ad733c612 ] is [2].
2025-10-07 21:59:50,078 WARN [kvm.storage.KVMStorageProcessor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Process [13181] for command
[/usr/share/cloudstack-common/scripts/storage/qcow2/managesnapshot.sh -b
/mnt/8f989a84-e691-3a8f-bbcf-14ec8b839ce5/snapshots/fc6bec25-28ff-483a-99da-f6917542c4da
-n fc6bec25-28ff-483a-99da-f6917542c4da -p
/mnt/29506a35-fd13-3d28-9ea5-f38d05dbfe5a/snapshots/2/75 -t
2c7300fd-c557-4923-a8cc-480ad733c612 ] encountered the error: [cp: cannot
create regular file
'/mnt/29506a35-fd13-3d28-9ea5-f38d05dbfe5a/snapshots/2/75/2c7300fd-c557-4923-a8cc-480ad733c612':
No such file or directoryFailed to backup fc6bec25-28ff-483a-99da-f6917542c4da
for disk
/mnt/8f989a84-e691-3a8f-bbcf-14ec8b839ce5/snapshots/fc6bec25-28ff-483a-99da-f6917542c4da
to /mnt/29506a35-fd13-3d28-9ea5-f38d05dbfe5a/snapshots/2/75].
2025-10-07 21:59:50,078 DEBUG [kvm.storage.KVMStorageProcessor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Failed to backup snaptshot: cp:
cannot create regular file
'/mnt/29506a35-fd13-3d28-9ea5-f38d05dbfe5a/snapshots/2/75/2c7300fd-c557-4923-a8cc-480ad733c612':
No such file or directoryFailed to backup fc6bec25-28ff-483a-99da-f6917542c4da
for disk
/mnt/8f989a84-e691-3a8f-bbcf-14ec8b839ce5/snapshots/fc6bec25-28ff-483a-99da-f6917542c4da
to /mnt/29506a35-fd13-3d28-9ea5-f38d05dbfe5a/snapshots/2/75
2025-10-07 21:59:50,079 DEBUG [kvm.storage.KVMStorageProcessor]
(agentRequest-Handler-3:[]) (logid:119c7f19) This backup is temporary, not
deleting snapshot
[/mnt/8f989a84-e691-3a8f-bbcf-14ec8b839ce5/snapshots/fc6bec25-28ff-483a-99da-f6917542c4da]
on primary storage [8f989a84-e691-3a8f-bbcf-14ec8b839ce5]
2025-10-07 21:59:50,079 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Attempting to remove storage pool
29506a35-fd13-3d28-9ea5-f38d05dbfe5a from libvirt
2025-10-07 21:59:50,079 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-3:[]) (logid:119c7f19) Looking for libvirtd connection
at: qemu:///system
2025-10-07 21:59:50,085 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Storage pool
29506a35-fd13-3d28-9ea5-f38d05dbfe5a has no corresponding secret. Not removing
any secret.
2025-10-07 21:59:50,110 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-3:[]) (logid:119c7f19) Storage pool
29506a35-fd13-3d28-9ea5-f38d05dbfe5a was successfully removed from libvirt.
2025-10-07 21:59:50,122 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-3:[]) (logid:119c7f19) Seq 1-224617031415103660: { Ans:
, MgmtId: 117375486344555, via: 1, Ver: v1, Flags: 10,
[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":"false","details":"cp:
cannot create regular file
'/mnt/29506a35-fd13-3d28-9ea5-f38d05dbfe5a/snapshots/2/75/2c7300fd-c557-4923-a8cc-480ad733c612':
No such file or directoryFailed to backup fc6bec25-28ff-483a-99da-f6917542c4da
for disk
/mnt/8f989a84-e691-3a8f-bbcf-14ec8b839ce5/snapshots/fc6bec25-28ff-483a-99da-f6917542c4da
to
/mnt/29506a35-fd13-3d28-9ea5-f38d05dbfe5a/snapshots/2/75","wait":"0","bypassHostMaintenance":"false"}}]
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
