MilanHofmann commented on issue #7171:
URL: https://github.com/apache/cloudstack/issues/7171#issuecomment-1421253794
@slavkap
I actually find this line from the logs interesting:
`com.cloud.exception.StorageUnavailableException: Resource [StoragePool:7]
is unreachable: Unable to create Vol[164|vm=164|ROOT]:Failed to get template
from pool: 73d1d51a-7a3a-3643-833c-a828041be576`
If I mount the secondary storage manually the image is available.
But once it's being mounted by cloudstack itself the directory is empty.
Here are the agent logs:
`
2023-02-07 19:18:12,457 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-4:null) (logid:40b6260a) Request:Seq
7-1562186120744141630: { Cmd , MgmtId: 2934858251578, via: 7, Ver: v1, Flags:
100111,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"template/tmpl/1/3/","origUrl":"https://download.cloudstack.org/systemvm/4.17/systemvmtemplate-4.17.2-kvm.qcow2.bz2","uuid":"df635688-a32e-11ed-a71b-02ab532efd3a","id":"3","format":"QCOW2","accountId":"1","checksum":"0e44e12e0a6a370b85d5d7f2ad541354","hvm":"false","displayText":"SystemVM
Template
(KVM)","imageDataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://3.70.229.74/export/secondary","_role":"Image"}},"name":"routing-3","size":"(0
bytes)
0","hypervisorType":"KVM","bootable":"false","uniqueName":"routing-3","directDownload":"false","deployAsIs":"false"}},"destTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"origUrl":"https://download.cloudstack.o
rg/systemvm/4.17/systemvmtemplate-4.17.2-kvm.qcow2.bz2","uuid":"df635688-a32e-11ed-a71b-02ab532efd3a","id":"3","format":"QCOW2","accountId":"1","checksum":"0e44e12e0a6a370b85d5d7f2ad541354","hvm":"false","displayText":"SystemVM
Template
(KVM)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"19986357-feca-4161-830a-2a147fc59d9f","name":"m0995.contaboserver.net-local-19986357","id":"7","poolType":"Filesystem","host":"79.143.179.221","path":"/var/lib/libvirt/images","port":"0","url":"Filesystem://79.143.179.221/var/lib/libvirt/images/?ROLE=Primary&STOREUUID=19986357-feca-4161-830a-2a147fc59d9f","isManaged":"false"}},"name":"routing-3","size":"(0
bytes)
0","hypervisorType":"KVM","bootable":"false","uniqueName":"routing-3","directDownload":"false","deployAsIs":"false"}},"executeInSequence":"true","options":{},"options2":{},"wait":"10800","bypassHostMaintenance":"false"}}]
}
2023-02-07 19:18:12,457 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-4:null) (logid:40b6260a) Processing command:
org.apache.cloudstack.storage.command.CopyCommand
2023-02-07 19:18:12,457 DEBUG
[storage.resource.StorageSubsystemCommandHandlerBase]
(agentRequest-Handler-4:null) (logid:40b6260a) Executing command CopyCommand:
[{"srcTO":{"path":"template/tmpl/1/3/","origUrl":"https://download.cloudstack.org/systemvm/4.17/systemvmtemplate-4.17.2-kvm.qcow2.bz2","uuid":"df635688-a32e-11ed-a71b-02ab532efd3a","id":3,"format":"QCOW2","accountId":1,"checksum":"0e44e12e0a6a370b85d5d7f2ad541354","hvm":false,"displayText":"SystemVM
Template
(KVM)","imageDataStore":{"_url":"nfs://3.70.229.74/export/secondary","_role":"Image"},"name":"routing-3","size":0,"hypervisorType":"KVM","bootable":false,"uniqueName":"routing-3","directDownload":false,"deployAsIs":false},"destTO":{"origUrl":"https://download.cloudstack.org/systemvm/4.17/systemvmtemplate-4.17.2-kvm.qcow2.bz2","uuid":"df635688-a32e-11ed-a71b-02ab532efd3a","id":3,"format":"QCOW2","accountId":1,"checksum":"0e44e12e0a6a370b85d5d7f2ad541354","hvm":false,"displayText":"SystemVM
Template (KVM)","imageDataSto
re":{"uuid":"19986357-feca-4161-830a-2a147fc59d9f","name":"m0995.contaboserver.net-local-19986357","id":7,"poolType":"Filesystem","host":"79.143.179.221","path":"/var/lib/libvirt/images","port":0,"url":"Filesystem://79.143.179.221/var/lib/libvirt/images/?ROLE\u003dPrimary\u0026STOREUUID\u003d19986357-feca-4161-830a-2a147fc59d9f","isManaged":false},"name":"routing-3","size":0,"hypervisorType":"KVM","bootable":false,"uniqueName":"routing-3","directDownload":false,"deployAsIs":false},"executeInSequence":true,"options":{},"options2":{},"contextMap":{"logid":"40b6260a","job":"job-440/job-548"},"wait":10800,"bypassHostMaintenance":false}].
2023-02-07 19:18:12,457 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-4:null) (logid:40b6260a) Attempting to create storage
pool 73d1d51a-7a3a-3643-833c-a828041be576 (NetworkFilesystem) in libvirt
2023-02-07 19:18:12,457 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-4:null) (logid:40b6260a) Looking for libvirtd connection
at: qemu:///system
2023-02-07 19:18:12,477 WARN [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-4:null) (logid:40b6260a) Storage pool
73d1d51a-7a3a-3643-833c-a828041be576 was not found running in libvirt. Need to
create it.
2023-02-07 19:18:12,477 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-4:null) (logid:40b6260a) Didn't find an existing storage
pool 73d1d51a-7a3a-3643-833c-a828041be576 by UUID, checking for pools with
duplicate paths
2023-02-07 19:18:12,478 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-4:null) (logid:40b6260a) Checking path of existing pool
19986357-feca-4161-830a-2a147fc59d9f against pool we want to create
2023-02-07 19:18:12,481 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-4:null) (logid:40b6260a) Attempting to create storage
pool 73d1d51a-7a3a-3643-833c-a828041be576
2023-02-07 19:18:12,481 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-4:null) (logid:40b6260a) <pool type='netfs'>
<name>73d1d51a-7a3a-3643-833c-a828041be576</name>
<uuid>73d1d51a-7a3a-3643-833c-a828041be576</uuid>
<source>
<host name='3.70.229.74'/>
<dir path='/export/secondary/template/tmpl/1/3'/>
</source>
<target>
<path>/mnt/73d1d51a-7a3a-3643-833c-a828041be576</path>
</target>
</pool>
2023-02-07 19:18:12,481 DEBUG [utils.script.Script]
(agentRequest-Handler-4:null) (logid:40b6260a) Executing: /bin/bash -c
mountpoint -q /mnt/73d1d51a-7a3a-3643-833c-a828041be576
2023-02-07 19:18:12,482 DEBUG [utils.script.Script]
(agentRequest-Handler-4:null) (logid:40b6260a) Executing while with timeout :
3600000
2023-02-07 19:18:12,485 DEBUG [utils.script.Script]
(agentRequest-Handler-4:null) (logid:40b6260a) Execution is successful.
2023-02-07 19:18:12,486 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-4:null) (logid:40b6260a) Attempting to unmount old mount
at /mnt/73d1d51a-7a3a-3643-833c-a828041be576
2023-02-07 19:18:12,486 DEBUG [utils.script.Script]
(agentRequest-Handler-4:null) (logid:40b6260a) Executing: /bin/bash -c umount
-l /mnt/73d1d51a-7a3a-3643-833c-a828041be576
2023-02-07 19:18:12,487 DEBUG [utils.script.Script]
(agentRequest-Handler-4:null) (logid:40b6260a) Executing while with timeout :
3600000
2023-02-07 19:18:12,493 DEBUG [utils.script.Script]
(agentRequest-Handler-4:null) (logid:40b6260a) Execution is successful.
2023-02-07 19:18:12,494 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-4:null) (logid:40b6260a) Succeeded in unmounting
/mnt/73d1d51a-7a3a-3643-833c-a828041be576
2023-02-07 19:18:12,721 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-4:null) (logid:40b6260a) Trying to fetch storage pool
73d1d51a-7a3a-3643-833c-a828041be576 from libvirt
2023-02-07 19:18:12,721 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-4:null) (logid:40b6260a) Looking for libvirtd connection
at: qemu:///system
2023-02-07 19:18:12,750 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-4:null) (logid:40b6260a) Successfully refreshed pool
73d1d51a-7a3a-3643-833c-a828041be576 Capacity: (242.11 GB) 259962437632 Used:
(9.25 GB) 9930014720 Available: (232.86 GB) 250032422912
2023-02-07 19:18:12,799 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-4:null) (logid:40b6260a) Attempting to remove storage
pool 73d1d51a-7a3a-3643-833c-a828041be576 from libvirt
2023-02-07 19:18:12,799 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-4:null) (logid:40b6260a) Looking for libvirtd connection
at: qemu:///system
2023-02-07 19:18:12,817 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-4:null) (logid:40b6260a) Storage pool
73d1d51a-7a3a-3643-833c-a828041be576 has no corresponding secret. Not removing
any secret.
2023-02-07 19:18:12,826 INFO [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-4:null) (logid:40b6260a) Storage pool
73d1d51a-7a3a-3643-833c-a828041be576 was successfully removed from libvirt.
2023-02-07 19:18:12,826 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-4:null) (logid:40b6260a) Seq 7-1562186120744141630: {
Ans: , MgmtId: 2934858251578, via: 7, Ver: v1, Flags: 110,
[{"com.cloud.agent.api.storage.PrimaryStorageDownloadAnswer":{"templateSize":"(0
bytes) 0","result":"false","details":"Failed to get template from pool:
73d1d51a-7a3a-3643-833c-a828041be576","wait":"0","bypassHostMaintenance":"false"}}]
}
2023-02-07 19:18:12,910 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-5:null) (logid:40b6260a) Request:Seq
7-1562186120744141631: { Cmd , MgmtId: 2934858251578, via: 7, Ver: v1, Flags:
100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":"false","controlIp":"169.254.117.245","forceStop":"false","volumesToDisconnect":[],"vmName":"v-174-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}]
}
2023-02-07 19:18:12,911 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-5:null) (logid:40b6260a) Processing command:
com.cloud.agent.api.StopCommand
2023-02-07 19:18:12,911 DEBUG [resource.wrapper.LibvirtStopCommandWrapper]
(agentRequest-Handler-5:null) (logid:40b6260a) backing up the cmdline
2023-02-07 19:18:15,972 DEBUG [resource.wrapper.LibvirtStopCommandWrapper]
(agentRequest-Handler-5:null) (logid:40b6260a) Failed to backup cmdline file
due to There was a problem while connecting to 169.254.117.245:3922
2023-02-07 19:18:15,972 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-5:null) (logid:40b6260a) Looking for libvirtd connection
at: qemu:///system
2023-02-07 19:18:15,993 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-5:null) (logid:40b6260a) Can not find KVM connection for
Instance: v-174-VM, continuing.
2023-02-07 19:18:15,993 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-5:null) (logid:40b6260a) Looking for libvirtd connection
at: lxc:///
2023-02-07 19:18:15,993 INFO [kvm.resource.LibvirtConnection]
(agentRequest-Handler-5:null) (logid:40b6260a) No existing libvirtd connection
found. Opening a new one
2023-02-07 19:18:15,994 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-5:null) (logid:40b6260a) Can not find LXC connection for
Instance: v-174-VM, continuing.
2023-02-07 19:18:15,994 WARN [kvm.resource.LibvirtConnection]
(agentRequest-Handler-5:null) (logid:40b6260a) Can not find a connection for
Instance v-174-VM. Assuming the default connection.
2023-02-07 19:18:15,995 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-5:null) (logid:40b6260a) Looking for libvirtd connection
at: qemu:///system
2023-02-07 19:18:16,013 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-5:null) (logid:40b6260a) Failed to get dom xml:
org.libvirt.LibvirtException: Domain not found: no domain with matching name
'v-174-VM'
2023-02-07 19:18:16,013 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-5:null) (logid:40b6260a) Failed to get dom xml:
org.libvirt.LibvirtException: Domain not found: no domain with matching name
'v-174-VM'
2023-02-07 19:18:16,014 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-5:null) (logid:40b6260a) Failed to get dom xml:
org.libvirt.LibvirtException: Domain not found: no domain with matching name
'v-174-VM'
2023-02-07 19:18:16,014 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-5:null) (logid:40b6260a) Executing:
/usr/share/cloudstack-common/scripts/vm/network/security_group.py
destroy_network_rules_for_vm --vmname v-174-VM
2023-02-07 19:18:16,015 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-5:null) (logid:40b6260a) Executing while with timeout :
1800000
2023-02-07 19:18:16,113 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-5:null) (logid:40b6260a) Execution is successful.
2023-02-07 19:18:16,114 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-5:null) (logid:40b6260a) Failed to get vm :Domain not
found: no domain with matching name 'v-174-VM'
2023-02-07 19:18:16,114 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-5:null) (logid:40b6260a) Try to stop the vm at first
2023-02-07 19:18:16,115 DEBUG [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-5:null) (logid:40b6260a) VM v-174-VM doesn't exist, no
need to stop it
`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
