Hello,
This issue is related to the same upgrade : https://lists.apache.org/thread.html/r94f7600fc509d7968348edd7014160501618ad 14ee0ee012d1ee6ad7%40%3Cusers.cloudstack.apache.org%3E I did a restore, to 4.15.0, everything is working fine. I added again the template and upgraded to 4.15.1, when I try to upgrade the router, fails because SSH fail, but before this, I see this error : Also available on : https://pastebin.com/J9MpBUFc >From cloudstack management server : 2021-07-30 03:17:53,507 DEBUG [o.a.c.s.v.VolumeServiceImpl] (Work-Job-Executor-8:ctx-27139d00 job-33099/job-33101 ctx-1a500aa4) (logid:3543142f) Acquire lock on VMTemplateStoragePool 718 with timeout 3600 seconds 2021-07-30 03:17:53,508 INFO [o.a.c.s.v.VolumeServiceImpl] (Work-Job-Executor-8:ctx-27139d00 job-33099/job-33101 ctx-1a500aa4) (logid:3543142f) lock is acquired for VMTemplateStoragePool 718 2021-07-30 03:17:53,511 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] (Work-Job-Executor-8:ctx-27139d00 job-33099/job-33101 ctx-1a500aa4) (logid:3543142f) copyAsync inspecting src type TEMPLATE copyAsync inspecting dest type TEMPLATE 2021-07-30 03:17:53,516 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru] (Work-Job-Executor-8:ctx-27139d00 job-33099/job-33101 ctx-1a500aa4) (logid:3543142f) getCommandHostDelegation: class org.apache.cloudstack.storage.command.CopyCommand 2021-07-30 03:17:53,522 DEBUG [c.c.a.t.Request] (Work-Job-Executor-8:ctx-27139d00 job-33099/job-33101 ctx-1a500aa4) (logid:3543142f) Seq 27-4204673202103779359: Sending { Cmd , MgmtId: 345049356158, via: 27(s-1709-VM), Ver: v1, Flags: 100111, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache. cloudstack.storage.to.TemplateObjectTO":{"path":"template/tmpl/2/335/78ab08a 4-3aa8-38c4-bb9e-0ff054e48056.ova","origUrl":http://packages.shapeblue.com.s 3-eu-west-1.amazonaws.com/systemvmtemplate/4.15/systemvmtemplate-4.15.1-vmwa re.ova,"uuid":"25878663-bfb5-4a5b-81da-e4240c09eb2a","id":"335","format":"OV A","accountId":"2","checksum":"{SHA-512}c369bc2cec47752ef5727fcf55fcb1f8277e 3f14e8aad3ba87158257fc263658bfd3cb9da6d29b3443e612f2d55c46737bf8bc76487509fd adfcd7953679c16e","hvm":"true","displayText":"systemvm-vmware-4.15.1","image DataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://cloud-emea-nfs.xxx .xx/secondary","_role":"Image"}},"name":"335-2-2046a99f-449e-309d-b360-44cb7 3bc776e","size":"(2.44 GB) 2621440000","hypervisorType":"VMware","bootable":"false","uniqueName":"335-2 -2046a99f-449e-309d-b360-44cb73bc776e","directDownload":"false","deployAsIs" :"true"}},"destTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"or igUrl":http://packages.shapeblue.com.s3-eu-west-1.amazonaws.com/systemvmtemp late/4.15/systemvmtemplate-4.15.1-vmware.ova,"uuid":"25878663-bfb5-4a5b-81da -e4240c09eb2a","id":"335","format":"OVA","accountId":"2","checksum":"{SHA-51 2}c369bc2cec47752ef5727fcf55fcb1f8277e3f14e8aad3ba87158257fc263658bfd3cb9da6 d29b3443e612f2d55c46737bf8bc76487509fdadfcd7953679c16e","hvm":"true","displa yText":"systemvm-vmware-4.15.1","imageDataStore":{"org.apache.cloudstack.sto rage.to.PrimaryDataStoreTO":{"uuid":"e397454f-e8e3-4bd2-8852-9a1969d5d3ea"," id":"5","poolType":"VMFS","host":"VMFS datastore: datastore-539","path":"datastore-539","port":"0","url":"VMFS://VMFS datastore: datastore-539/datastore-539/?ROLE=Primary&STOREUUID=e397454f-e8e3-4bd2-8852- 9a1969d5d3ea","fullCloneFlag":"true","isManaged":"false"}},"name":"335-2-204 6a99f-449e-309d-b360-44cb73bc776e","size":"(2.44 GB) 2621440000","hypervisorType":"VMware","bootable":"false","uniqueName":"335-2 -2046a99f-449e-309d-b360-44cb73bc776e","directDownload":"false","deployAsIs" :"true"}},"executeInSequence":"true","options":{},"options2":{},"wait":"1080 0","bypassHostMaintenance":"false"}}] } 2021-07-30 03:17:53,634 DEBUG [c.c.a.ApiServlet] (qtp769798433-14:ctx-8853f0a2) (logid:bdd86e35) ===START=== 79.114.96.240 -- GET jobid=3543142f-7bde-422b-8c2c-76bcfce30c49&command=queryAsyncJobResult&respo nse=json 2021-07-30 03:17:53,638 DEBUG [c.c.a.ApiServer] (qtp769798433-14:ctx-8853f0a2 ctx-dca36ca1) (logid:bdd86e35) CIDRs from which account 'Acct[9999d31c-ffd2-11e5-aba2-0050568b4668-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-07-30 03:17:53,643 DEBUG [c.c.a.ApiServlet] (qtp769798433-14:ctx-8853f0a2 ctx-dca36ca1) (logid:bdd86e35) ===END=== 79.114.96.240 -- GET jobid=3543142f-7bde-422b-8c2c-76bcfce30c49&command=queryAsyncJobResult&respo nse=json 2021-07-30 03:17:53,942 DEBUG [o.a.c.b.BackupManagerImpl] (BackupPollTask:ctx-8e0b559f) (logid:4ba91ce6) Backup backup.poll is being called at 2021-07-30 07:17:53 GMT 2021-07-30 03:17:53,946 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterMonitor-1:ctx-8f277e36) (logid:c40b534d) Found 0 running routers. 2021-07-30 03:17:53,947 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-1daa7ba4) (logid:6fe12ec4) Found 0 routers to update status. 2021-07-30 03:17:53,947 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-1daa7ba4) (logid:6fe12ec4) Found 0 VPC's to update Redundant State. 2021-07-30 03:17:53,948 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-1daa7ba4) (logid:6fe12ec4) Found 0 networks to update RvR status. 2021-07-30 03:17:53,960 DEBUG [c.c.n.ExternalDeviceUsageManagerImpl] (ExternalNetworkMonitor-1:ctx-04333da2) (logid:62b6d691) External devices are not used. Skipping external device usage collection 2021-07-30 03:17:54,002 DEBUG [c.c.s.s.SnapshotSchedulerImpl] (SnapshotPollTask:ctx-66d2c4a0) (logid:df702db0) Snapshot scheduler.poll is being called at 2021-07-30 07:17:54 GMT 2021-07-30 03:17:54,002 DEBUG [c.c.s.s.SnapshotSchedulerImpl] (SnapshotPollTask:ctx-66d2c4a0) (logid:df702db0) Got 0 snapshots to be executed at 2021-07-30 07:17:54 GMT 2021-07-30 03:17:54,598 DEBUG [c.c.a.ApiServlet] (qtp769798433-16:ctx-96ec74aa) (logid:5501dd37) ===START=== 79.114.96.240 -- GET jobId=3543142f-7bde-422b-8c2c-76bcfce30c49&command=queryAsyncJobResult&respo nse=json 2021-07-30 03:17:54,602 DEBUG [c.c.a.ApiServer] (qtp769798433-16:ctx-96ec74aa ctx-bf68f8e3) (logid:5501dd37) CIDRs from which account 'Acct[9999d31c-ffd2-11e5-aba2-0050568b4668-admin]' is allowed to perform API calls: 0.0.0.0/0,::/0 2021-07-30 03:17:54,608 DEBUG [c.c.a.ApiServlet] (qtp769798433-16:ctx-96ec74aa ctx-bf68f8e3) (logid:5501dd37) ===END=== 79.114.96.240 -- GET jobId=3543142f-7bde-422b-8c2c-76bcfce30c49&command=queryAsyncJobResult&respo nse=json 2021-07-30 03:17:54,734 DEBUG [c.c.a.t.Request] (AgentManager-Handler-15:null) (logid:) Seq 27-4204673202103779359: Processing: { Ans: , MgmtId: 345049356158, via: 27, Ver: v1, Flags: 110, [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":"false","d etails":"Unable to copy template to primary storage due to exception:Exception: com.cloud.utils.exception.CloudRuntimeException Message: Failed to create an import spec from /mnt/SecStorage/fc8b20a7-1eb2-3da7-aec9-e60f546ab40b/template/tmpl/2/335/sys temvmtemplate-vmware.ovf. Check log for details. ","wait":"0","bypassHostMaintenance":"false"}}] } 2021-07-30 03:17:54,735 DEBUG [c.c.a.t.Request] (Work-Job-Executor-8:ctx-27139d00 job-33099/job-33101 ctx-1a500aa4) (logid:3543142f) Seq 27-4204673202103779359: Received: { Ans: , MgmtId: 345049356158, via: 27(s-1709-VM), Ver: v1, Flags: 110, { CopyCmdAnswer } } 2021-07-30 03:17:54,736 DEBUG [c.c.a.m.AgentAttache] (AgentManager-Handler-15:null) (logid:) Seq 27-4204673202103779359: No more commands found 2021-07-30 03:17:54,739 INFO [o.a.c.s.v.VolumeServiceImpl] (Work-Job-Executor-8:ctx-27139d00 job-33099/job-33101 ctx-1a500aa4) (logid:3543142f) releasing lock for VMTemplateStoragePool 718 2021-07-30 03:17:54,739 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-8:ctx-27139d00 job-33099/job-33101 ctx-1a500aa4) (logid:3543142f) Unable to create Vol[2688|vm=1712|ROOT]:Unable to copy template to primary storage due to exception:Exception: com.cloud.utils.exception.CloudRuntimeException Message: Failed to create an import spec from /mnt/SecStorage/fc8b20a7-1eb2-3da7-aec9-e60f546ab40b/template/tmpl/2/335/sys temvmtemplate-vmware.ovf. Check log for details. 2021-07-30 03:17:54,739 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-8:ctx-27139d00 job-33099/job-33101 ctx-1a500aa4) (logid:3543142f) Unable to contact resource. com.cloud.exception.StorageUnavailableException: Resource [StoragePool:5] is unreachable: Unable to create Vol[2688|vm=1712|ROOT]:Unable to copy template to primary storage due to exception:Exception: com.cloud.utils.exception.CloudRuntimeException Message: Failed to create an import spec from /mnt/SecStorage/fc8b20a7-1eb2-3da7-aec9-e60f546ab40b/template/tmpl/2/335/sys temvmtemplate-vmware.ovf. Check log for details. at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.recreateVolume (VolumeOrchestrator.java:1477) at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.prepare(Volume Orchestrator.java:1552) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage rImpl.java:1160) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManage rImpl.java:5502) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodA ccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Delegatin gMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.jav a:107) at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManager Impl.java:5669) at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext (AsyncJobManagerImpl.java:620) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedCo ntextRunnable.java:48) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(Defa ultManagedContext.java:55) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithCon text(DefaultManagedContext.java:102) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithCont ext(DefaultManagedContext.java:52) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedCont extRunnable.java:45) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJob ManagerImpl.java:568) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java :515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecut or.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecu tor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) 2021-07-30 03:17:54,740 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-8:ctx-27139d00 job-33099/job-33101 ctx-1a500aa4) (logid:3543142f) Cleaning up resources for the vm VM[DomainRouter|r-1712-VM] in Starting state 2021-07-30 03:17:54,743 DEBUG [c.c.a.t.Request] (Work-Job-Executor-8:ctx-27139d00 job-33099/job-33101 ctx-1a500aa4) (logid:3543142f) Seq 9-837388055714201655: Sending { Cmd , MgmtId: 345049356158, via: 9(ns3xxx.ip-176-xxx-xxx.eu), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup": "false","forceStop":"false","volumesToDisconnect":[],"vmName":"r-1712-VM","e xecuteInSequence":"true","wait":"0","bypassHostMaintenance":"false"}}] } 2021-07-30 03:17:54,743 DEBUG [c.c.a.t.Request] (Work-Job-Executor-8:ctx-27139d00 job-33099/job-33101 ctx-1a500aa4) (logid:3543142f) Seq 9-837388055714201655: Executing: { Cmd , MgmtId: 345049356158, via: 9(ns3xxx.ip-176-xxx-xxx.eu), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup": "false","forceStop":"false","volumesToDisconnect":[],"vmName":"r-1712-VM","e xecuteInSequence":"true","wait":"0","bypassHostMaintenance":"false"}}] } >From secondary storage ( I can see this only if I do not try to rebuild the SSVM, rebuild fails because of the same error ): 2021-07-30 07:28:14,113 INFO [storage.resource.VmwareSecondaryStorageResourceHandler] (agentRequest-Handler-2:job-33102/job-33104, cmd: CopyCommand) Open new VmwareContext. vCenter: 172.11.11.12, user: administrator@vsphere.local <mailto:administrator@vsphere.local> , password: y*********** 2021-07-30 07:28:14,279 INFO [storage.resource.VmwareSecondaryStorageResourceHandler] (agentRequest-Handler-2:job-33102/job-33104, cmd: CopyCommand) Setup firewall rule for host: 172.11.11.104 2021-07-30 07:28:14,359 INFO [storage.resource.VmwareSecondaryStorageResourceHandler] (agentRequest-Handler-2:job-33102/job-33104, cmd: CopyCommand) Setup firewall rule for host: 172.11.11.105 2021-07-30 07:28:14,659 INFO [storage.resource.VmwareStorageProcessor] (agentRequest-Handler-2:job-33102/job-33104, cmd: CopyCommand) Template 78ab08a4-3aa8-38c4-bb9e-0ff054e48056 is not setup yet. Set up template from secondary storage with uuid name: 39e741a1-3f95-4f2b-a051-877db641e2be 2021-07-30 07:28:14,661 INFO [storage.resource.VmwareStorageProcessor] (agentRequest-Handler-2:job-33102/job-33104, cmd: CopyCommand) Executing copyTemplateFromSecondaryToPrimary. secondaryStorage: nfs://cloud-emea-nfs.xxx.xx/secondary, templatePathAtSecondaryStorage: template/tmpl/2/335/, templateName: 78ab08a4-3aa8-38c4-bb9e-0ff054e48056, configuration: null 2021-07-30 07:28:14,662 INFO [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-2:job-33102/job-33104, cmd: CopyCommand) Determined host cloud-emea-nfs.xxx.xx corresponds to IP 172.11.11.6 2021-07-30 07:28:14,682 INFO [storage.resource.VmwareStorageProcessor] (agentRequest-Handler-2:job-33102/job-33104, cmd: CopyCommand) Secondary storage mount point: /mnt/SecStorage/fc8b20a7-1eb2-3da7-aec9-e60f546ab40b 2021-07-30 07:28:15,269 ERROR [vmware.mo.HypervisorHostHelper] (agentRequest-Handler-2:job-33102/job-33104, cmd: CopyCommand) createImportSpec error: Line 1: Unsupported hardware family 'vmx-11'. 2021-07-30 07:28:15,270 INFO [vmware.util.VmwareHelper] (agentRequest-Handler-2:job-33102/job-33104, cmd: CopyCommand) [ignored]failed to get message for exception: Failed to create an import spec from /mnt/SecStorage/fc8b20a7-1eb2-3da7-aec9-e60f546ab40b/template/tmpl/2/335/sys temvmtemplate-vmware.ovf. Check log for details. 2021-07-30 07:28:15,270 ERROR [storage.resource.VmwareStorageProcessor] (agentRequest-Handler-2:job-33102/job-33104, cmd: CopyCommand) Unable to copy template to primary storage due to exception:Exception: com.cloud.utils.exception.CloudRuntimeException Message: Failed to create an import spec from /mnt/SecStorage/fc8b20a7-1eb2-3da7-aec9-e60f546ab40b/template/tmpl/2/335/sys temvmtemplate-vmware.ovf. Check log for details. com.cloud.utils.exception.CloudRuntimeException: Failed to create an import spec from /mnt/SecStorage/fc8b20a7-1eb2-3da7-aec9-e60f546ab40b/template/tmpl/2/335/sys temvmtemplate-vmware.ovf. Check log for details. at com.cloud.hypervisor.vmware.mo.HypervisorHostHelper.importVmFromOVF(Hypervis orHostHelper.java:1825) at com.cloud.hypervisor.vmware.mo.HostMO.importVmFromOVF(HostMO.java:784) at com.cloud.storage.resource.VmwareStorageProcessor.copyTemplateFromSecondaryT oPrimary(VmwareStorageProcessor.java:529) at com.cloud.storage.resource.VmwareStorageProcessor.copyTemplateToPrimaryStora ge(VmwareStorageProcessor.java:692) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(Storag eSubsystemCommandHandlerBase.java:92) at com.cloud.storage.resource.VmwareStorageSubsystemCommandHandler.execute(Vmwa reStorageSubsystemCommandHandler.java:197) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageC ommands(StorageSubsystemCommandHandlerBase.java:58) at com.cloud.storage.resource.VmwareSecondaryStorageResourceHandler.executeRequ est(VmwareSecondaryStorageResourceHandler.java:116) at com.cloud.storage.resource.PremiumSecondaryStorageResource.executeRequest(Pr emiumSecondaryStorageResource.java:57) at com.cloud.agent.Agent.processRequest(Agent.java:661) at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1079) at com.cloud.utils.nio.Task.call(Task.java:83) at com.cloud.utils.nio.Task.call(Task.java:29) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecut or.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecu tor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) 2021-07-30 07:28:15,518 INFO [storage.resource.VmwareSecondaryStorageResourceHandler] (agentRequest-Handler-4:job-33102/job-33104, cmd: CopyCommand) Open new VmwareContext. vCenter: 172.11.11.12, user: administrator@vsphere.local <mailto:administrator@vsphere.local> , password: y*********** 2021-07-30 07:28:15,527 INFO [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-3:null) Determined host cloud-emea-nfs.xxx.xx corresponds to IP 172.11.11.6 2021-07-30 07:28:15,676 INFO [storage.resource.VmwareSecondaryStorageResourceHandler] (agentRequest-Handler-4:job-33102/job-33104, cmd: CopyCommand) Setup firewall rule for host: 172.11.11.104 2021-07-30 07:28:15,764 INFO [storage.resource.VmwareSecondaryStorageResourceHandler] (agentRequest-Handler-4:job-33102/job-33104, cmd: CopyCommand) Setup firewall rule for host: 172.11.11.105 2021-07-30 07:28:16,090 INFO [storage.resource.VmwareStorageProcessor] (agentRequest-Handler-4:job-33102/job-33104, cmd: CopyCommand) Template 78ab08a4-3aa8-38c4-bb9e-0ff054e48056 is not setup yet. Set up template from secondary storage with uuid name: ed45c9a9-6607-4c13-b86f-cc72fea681c0 2021-07-30 07:28:16,091 INFO [storage.resource.VmwareStorageProcessor] (agentRequest-Handler-4:job-33102/job-33104, cmd: CopyCommand) Executing copyTemplateFromSecondaryToPrimary. secondaryStorage: nfs://cloud-emea-nfs.xxx.xx/secondary, templatePathAtSecondaryStorage: template/tmpl/2/335/, templateName: 78ab08a4-3aa8-38c4-bb9e-0ff054e48056, configuration: null 2021-07-30 07:28:16,091 INFO [storage.resource.NfsSecondaryStorageResource] (agentRequest-Handler-4:job-33102/job-33104, cmd: CopyCommand) Determined host cloud-emea-nfs.xxx.xx corresponds to IP 172.11.11.6 2021-07-30 07:28:16,104 INFO [storage.resource.VmwareStorageProcessor] (agentRequest-Handler-4:job-33102/job-33104, cmd: CopyCommand) Secondary storage mount point: /mnt/SecStorage/fc8b20a7-1eb2-3da7-aec9-e60f546ab40b 2021-07-30 07:28:16,681 ERROR [vmware.mo.HypervisorHostHelper] (agentRequest-Handler-4:job-33102/job-33104, cmd: CopyCommand) createImportSpec error: Line 1: Unsupported hardware family 'vmx-11'. 2021-07-30 07:28:16,682 INFO [vmware.util.VmwareHelper] (agentRequest-Handler-4:job-33102/job-33104, cmd: CopyCommand) [ignored]failed to get message for exception: Failed to create an import spec from /mnt/SecStorage/fc8b20a7-1eb2-3da7-aec9-e60f546ab40b/template/tmpl/2/335/sys temvmtemplate-vmware.ovf. Check log for details. 2021-07-30 07:28:16,683 ERROR [storage.resource.VmwareStorageProcessor] (agentRequest-Handler-4:job-33102/job-33104, cmd: CopyCommand) Unable to copy template to primary storage due to exception:Exception: com.cloud.utils.exception.CloudRuntimeException Message: Failed to create an import spec from /mnt/SecStorage/fc8b20a7-1eb2-3da7-aec9-e60f546ab40b/template/tmpl/2/335/sys temvmtemplate-vmware.ovf. Check log for details. com.cloud.utils.exception.CloudRuntimeException: Failed to create an import spec from /mnt/SecStorage/fc8b20a7-1eb2-3da7-aec9-e60f546ab40b/template/tmpl/2/335/sys temvmtemplate-vmware.ovf. Check log for details. at com.cloud.hypervisor.vmware.mo.HypervisorHostHelper.importVmFromOVF(Hypervis orHostHelper.java:1825) at com.cloud.hypervisor.vmware.mo.HostMO.importVmFromOVF(HostMO.java:784) at com.cloud.storage.resource.VmwareStorageProcessor.copyTemplateFromSecondaryT oPrimary(VmwareStorageProcessor.java:529) at com.cloud.storage.resource.VmwareStorageProcessor.copyTemplateToPrimaryStora ge(VmwareStorageProcessor.java:692) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(Storag eSubsystemCommandHandlerBase.java:92) at com.cloud.storage.resource.VmwareStorageSubsystemCommandHandler.execute(Vmwa reStorageSubsystemCommandHandler.java:197) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageC ommands(StorageSubsystemCommandHandlerBase.java:58) at com.cloud.storage.resource.VmwareSecondaryStorageResourceHandler.executeRequ est(VmwareSecondaryStorageResourceHandler.java:116) at com.cloud.storage.resource.PremiumSecondaryStorageResource.executeRequest(Pr emiumSecondaryStorageResource.java:57) at com.cloud.agent.Agent.processRequest(Agent.java:661) at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1079) at com.cloud.utils.nio.Task.call(Task.java:83) at com.cloud.utils.nio.Task.call(Task.java:29) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecut or.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecu tor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) Any suggestions? Thanks!