Sailaja Mada created CLOUDSTACK-4476:
----------------------------------------
Summary: VM is failing to deploy on a mixed zone after adding zone
wide primary storage (Xen & VMWARE)
Key: CLOUDSTACK-4476
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4476
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: Storage Controller, VMware, XenServer
Affects Versions: 4.2.1
Reporter: Sailaja Mada
Priority: Critical
Steps:
1. Configure Adv zone with VMWARE and Xen Clusters in 307 ( Cluster level
Primary storage)
2. Upgrade to 4.2
3. Add 2nd VMWARE cluster with Cluster level primary storages
4. Add 2 zone wide primary storage's
5. Put cluster1 Primary storage into maintenance
6. Tried to deploy on VMWARE cluster
Observation:
VM is failing to deploy on a mixed zone after adding zone wide primary storage
(Xen & VMWARE)
2013-08-23 17:17:55,764 DEBUG [cloud.api.ApiServlet] (catalina-exec-16:null)
===END=== 10.144.6.50 -- GET
command=queryAsyncJobResult&jobId=0a06283c-5bf6-4971-8464-a654096e0f6a&response=json&sessionkey=jtG93UVpk7qQV5CmR1fb9zJr%2BVU%3D&_=1377258730071
2013-08-23 17:17:55,770 DEBUG [agent.transport.Request]
(AgentManager-Handler-12:null) Seq 10-209518605: Processing: { Ans: , MgmtId:
187767034175903, via: 10, Ver: v1, Flags: 10,
[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"357c7fd7962231e99d6482134a65d435","id":0,"accountId":0,"hvm":false}},"result":true,"wait":0}}]
}
2013-08-23 17:17:55,771 DEBUG [agent.transport.Request]
(Job-Executor-24:job-130 = [ 0a06283c-5bf6-4971-8464-a654096e0f6a ]) Seq
10-209518605: Received: { Ans: , MgmtId: 187767034175903, via: 10, Ver: v1,
Flags: 10, { CopyCmdAnswer } }
2013-08-23 17:17:55,855 DEBUG [storage.motion.AncientDataMotionStrategy]
(Job-Executor-24:job-130 = [ 0a06283c-5bf6-4971-8464-a654096e0f6a ]) copyAsync
inspecting src type TEMPLATE copyAsync inspecting dest type VOLUME
2013-08-23 17:17:55,866 DEBUG [agent.transport.Request]
(Job-Executor-24:job-130 = [ 0a06283c-5bf6-4971-8464-a654096e0f6a ]) Seq
13-1361380407: Sending { Cmd , MgmtId: 187767034175903, via: 13, Ver: v1,
Flags: 100011,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"357c7fd7962231e99d6482134a65d435","origUrl":"http://download.cloud.com/releases/2.2.0/CentOS5.3-x86_64.ova","uuid":"7","id":7,"format":"OVA","accountId":1,"checksum":"f6f881b7f2292948d8494db837fe0f47","hvm":false,"displayText":"CentOS
5.3(64-bit) no GUI
(vSphere)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c214298f-00cd-32c7-95a9-dd81d8b3dbd1","id":207,"poolType":"NetworkFilesystem","host":"10.102.192.100","path":"/cpg_vol/sailaja/legacyzwps1","port":2049}},"name":"centos53-x64","hypervisorType":"VMware"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"c75bf456-7e12-4978-b4e7-e3596d37eb86","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c214298f-00cd-32c7-95a9-dd81d8b3dbd1","id":207,"poolType":"NetworkFilesystem","host":"10.102.192.100","path":"/cpg_vol/sailaja/legacyzwps1","port":2049}},"name":"ROOT-40","size":2147483648,"volumeId":102,"vmName":"i-4-40-VM","accountId":4,"format":"OVA","id":102,"hypervisorType":"None"}},"executeInSequence":false,"wait":0}}]
}
2013-08-23 17:17:55,867 DEBUG [agent.transport.Request]
(Job-Executor-24:job-130 = [ 0a06283c-5bf6-4971-8464-a654096e0f6a ]) Seq
13-1361380407: Executing: { Cmd , MgmtId: 187767034175903, via: 13, Ver: v1,
Flags: 100011,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"357c7fd7962231e99d6482134a65d435","origUrl":"http://download.cloud.com/releases/2.2.0/CentOS5.3-x86_64.ova","uuid":"7","id":7,"format":"OVA","accountId":1,"checksum":"f6f881b7f2292948d8494db837fe0f47","hvm":false,"displayText":"CentOS
5.3(64-bit) no GUI
(vSphere)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c214298f-00cd-32c7-95a9-dd81d8b3dbd1","id":207,"poolType":"NetworkFilesystem","host":"10.102.192.100","path":"/cpg_vol/sailaja/legacyzwps1","port":2049}},"name":"centos53-x64","hypervisorType":"VMware"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"c75bf456-7e12-4978-b4e7-e3596d37eb86","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c214298f-00cd-32c7-95a9-dd81d8b3dbd1","id":207,"poolType":"NetworkFilesystem","host":"10.102.192.100","path":"/cpg_vol/sailaja/legacyzwps1","port":2049}},"name":"ROOT-40","size":2147483648,"volumeId":102,"vmName":"i-4-40-VM","accountId":4,"format":"OVA","id":102,"hypervisorType":"None"}},"executeInSequence":false,"wait":0}}]
}
2013-08-23 17:17:55,868 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-31:null) Seq 13-1361380407: Executing request
2013-08-23 17:17:56,026 DEBUG [xen.resource.XenServerStorageProcessor]
(DirectAgent-31:null) Catch Exception com.xensource.xenapi.Types$UuidInvalid
:VDI getByUuid for uuid: 357c7fd7962231e99d6482134a65d435 failed due to The
uuid you supplied was invalid.
2013-08-23 17:17:56,027 WARN [xen.resource.XenServerStorageProcessor]
(DirectAgent-31:null) Unable to create volume;
Pool=PrimaryDataStoreTO[uuid=c214298f-00cd-32c7-95a9-dd81d8b3dbd1|name=null|id=207|pooltype=NetworkFilesystem];
Disk:
com.cloud.utils.exception.CloudRuntimeException: Catch Exception
com.xensource.xenapi.Types$UuidInvalid :VDI getByUuid for uuid:
357c7fd7962231e99d6482134a65d435 failed due to The uuid you supplied was
invalid.
at
com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.getVDIbyUuid(XenServerStorageProcessor.java:818)
at
com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.cloneVolumeFromBaseTemplate(XenServerStorageProcessor.java:934)
at
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:73)
at
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:49)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:617)
at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
Caused by: The uuid you supplied was invalid.
at com.xensource.xenapi.Types.checkResponse(Types.java:927)
at com.xensource.xenapi.Connection.dispatch(Connection.java:368)
at
com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:909)
at com.xensource.xenapi.VDI.getByUuid(VDI.java:326)
at
com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.getVDIbyUuid(XenServerStorageProcessor.java:814)
... 14 more
2013-08-23 17:17:56,029 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-31:null) Seq 13-1361380407: Response Received:
2013-08-23 17:17:56,029 DEBUG [agent.transport.Request] (DirectAgent-31:null)
Seq 13-1361380407: Processing: { Ans: , MgmtId: 187767034175903, via: 13, Ver:
v1, Flags: 10,
[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"com.cloud.utils.exception.CloudRuntimeException:
Catch Exception com.xensource.xenapi.Types$UuidInvalid :VDI getByUuid for
uuid: 357c7fd7962231e99d6482134a65d435 failed due to The uuid you supplied was
invalid.","wait":0}}] }
2013-08-23 17:17:56,030 DEBUG [agent.transport.Request]
(Job-Executor-24:job-130 = [ 0a06283c-5bf6-4971-8464-a654096e0f6a ]) Seq
13-1361380407: Received: { Ans: , MgmtId: 187767034175903, via: 13, Ver: v1,
Flags: 10, { CopyCmdAnswer } }
2013-08-23 17:17:56,039 WARN [storage.datastore.ObjectInDataStoreManagerImpl]
(Job-Executor-24:job-130 = [ 0a06283c-5bf6-4971-8464-a654096e0f6a ])
Unsupported data object (VOLUME,
org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@26324a0), no need
to delete from object in store ref table
2013-08-23 17:17:56,040 INFO [storage.volume.VolumeServiceImpl]
(Job-Executor-24:job-130 = [ 0a06283c-5bf6-4971-8464-a654096e0f6a ]) releasing
lock for VMTemplateStoragePool 18
2013-08-23 17:17:56,040 WARN [utils.db.Merovingian2] (Job-Executor-24:job-130
= [ 0a06283c-5bf6-4971-8464-a654096e0f6a ]) Was unable to find lock for the key
template_spool_ref18 and thread id 146124796
2013-08-23 17:17:56,041 DEBUG [cloud.storage.VolumeManagerImpl]
(Job-Executor-24:job-130 = [ 0a06283c-5bf6-4971-8464-a654096e0f6a ]) Unable to
create Vol[102|vm=40|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
Catch Exception com.xensource.xenapi.Types$UuidInvalid :VDI getByUuid for uuid:
357c7fd7962231e99d6482134a65d435 failed due to The uuid you supplied was
invalid.
2013-08-23 17:17:56,041 INFO [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-24:job-130 = [ 0a06283c-5bf6-4971-8464-a654096e0f6a ]) Unable to
contact resource.
2013-08-23 17:17:56,040 INFO [storage.volume.VolumeServiceImpl]
(Job-Executor-24:job-130 = [ 0a06283c-5bf6-4971-8464-a654096e0f6a ]) releasing
lock for VMTemplateStoragePool 18
2013-08-23 17:17:56,040 WARN [utils.db.Merovingian2] (Job-Executor-24:job-130
= [ 0a06283c-5bf6-4971-8464-a654096e0f6a ]) Was unable to find lock for the key
template_spool_ref18 and thread id 146124796
2013-08-23 17:17:56,041 DEBUG [cloud.storage.VolumeManagerImpl]
(Job-Executor-24:job-130 = [ 0a06283c-5bf6-4971-8464-a654096e0f6a ]) Unable to
create Vol[102|vm=40|ROOT]:com.cloud.utils.exception.CloudRuntimeException:
Catch Exception com.xensource.xenapi.Types$UuidInvalid :VDI getByUuid for uuid:
357c7fd7962231e99d6482134a65d435 failed due to The uuid you supplied was
invalid.
2013-08-23 17:17:56,041 INFO [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-24:job-130 = [ 0a06283c-5bf6-4971-8464-a654096e0f6a ]) Unable to
contact resource.
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:207] is
unreachable: Unable to create
Vol[102|vm=40|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Catch
Exception com.xensource.xenapi.Types$UuidInvalid :VDI getByUuid for uuid:
357c7fd7962231e99d6482134a65d435 failed due to The uuid you supplied was
invalid.
at
com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2534)
at
com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2582)
at
com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:885)
at
com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:575)
at
org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227)
at
org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
at
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3406)
at
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2966)
at
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2952)
at
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at
org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
at
com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-08-23 17:17:56,048 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-24:job-130 = [ 0a06283c-5bf6-4971-8464-a654096e0f6a ]) Cleaning
up resources for the vm VM[User|newinstance1] in Starting state
2013-08-23 17:17:56,061 DEBUG [agent.transport.Request]
(Job-Executor-24:job-130 = [ 0a06283c-5bf6-4971-8464-a654096e0f6a ]) Seq
7-1010566867: Sending { Cmd , MgmtId: 187767034175903, via: 7, Ver: v1, Flags:
100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"i-4-40-VM","wait":0}}]
}
2013-08-23 17:17:56,061 DEBUG [agent.transport.Request]
(Job-Executor-24:job-130 = [ 0a06283c-5bf6-4971-8464-a654096e0f6a ]) Seq
7-1010566867: Executing: { Cmd , MgmtId: 187767034175903, via: 7, Ver: v1,
Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"i-4-40-VM","wait":0}}]
}
2013-08-23 17:17:56,096 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-236:null) Seq 7-1010566867: Executing request
2013-08-23 17:17:56,096 INFO [vmware.resource.VmwareResource]
(DirectAgent-236:10.102.192.20) Executing resource StopCommand:
{"isProxy":false,"executeInSequence":false,"vmName":"i-4-40-VM","wait":0}
2013-08-23 17:17:56,096 DEBUG [vmware.mo.HostMO]
(DirectAgent-236:10.102.192.20) find VM i-4-40-VM on host
2013-08-23 17:17:56,096 INFO [vmware.mo.HostMO]
(DirectAgent-236:10.102.192.20) VM i-4-40-VM not found in host cache
2013-08-23 17:17:56,096 DEBUG [vmware.mo.HostMO]
(DirectAgent-236:10.102.192.20) load VM cache on host
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
