Hello,

I create VM and first time migrated successfully. Second migration error
Cloudstack 4.4, xenserver 6.2

Error /var/log/cloudstack/management/management-server.log:

2014-09-05 09:29:46,059 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-150:ctx-2f13d1ed) Seq 13-4340625615855026180: Executing request 2014-09-05 09:29:46,080 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-150:ctx-2f13d1ed) Seq 13-4340625615855026180: Response Received: 2014-09-05 09:29:46,080 DEBUG [c.c.a.t.Request] (DirectAgent-150:ctx-2f13d1ed) Seq 13-4340625615855026180: Processing: { Ans: , MgmtId: 119169995887277, via: 13, Ver: v1, Flags: 10, [{"com.cloud.agent.api.ClusterVMMetaDataSyncAnswer":{"_clusterId":1,"_isExecuted":false,"result":true,"wait":0}}] } 2014-09-05 09:29:46,858 DEBUG [c.c.a.ApiServlet] (catalina-exec-3:ctx-83b8f412) ===START=== 192.168.11.36 -- GET command=migrateVirtualMachineWithVolume&hostid=2144c7a0-c0fe-4003-a3a7-31a6e6074f5b&virtualmachineid=37f0ac55-94f3-4c27-a318-83e48d6724e4&response=json&sessionkey=%2FiZXhDJVVkXsA%2BF%2FBpWlQmV%2BbEM%3D&_=1409923786054 2014-09-05 09:29:46,918 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (catalina-exec-3:ctx-83b8f412 ctx-d4b68fa7) submit async job-1505, details: AsyncJobVO {id:1505, userId: 2, accountId: 2, instanceType: None, instanceId: null, cmd: org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd, cmdInfo: {"response":"json","sessionkey":"/iZXhDJVVkXsA+F/BpWlQmV+bEM\u003d","ctxDetails":"{\"com.cloud.vm.VirtualMachine\":\"37f0ac55-94f3-4c27-a318-83e48d6724e4\",\"com.cloud.host.Host\":\"2144c7a0-c0fe-4003-a3a7-31a6e6074f5b\"}","virtualmachineid":"37f0ac55-94f3-4c27-a318-83e48d6724e4","cmdEventType":"VM.MIGRATE","hostid":"2144c7a0-c0fe-4003-a3a7-31a6e6074f5b","ctxUserId":"2","httpmethod":"GET","_":"1409923786054","ctxAccountId":"2","ctxStartEventId":"1636"}, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 119169995887277, completeMsid: null, lastUpdated: null, lastPolled: null, created: null} 2014-09-05 09:29:46,919 INFO [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-117:ctx-b9105a91 job-1505) Add job-1505 into job monitoring 2014-09-05 09:29:46,919 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-117:ctx-b9105a91 job-1505) Executing AsyncJobVO {id:1505, userId: 2, accountId: 2, instanceType: None, instanceId: null, cmd: org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd, cmdInfo: {"response":"json","sessionkey":"/iZXhDJVVkXsA+F/BpWlQmV+bEM\u003d","ctxDetails":"{\"com.cloud.vm.VirtualMachine\":\"37f0ac55-94f3-4c27-a318-83e48d6724e4\",\"com.cloud.host.Host\":\"2144c7a0-c0fe-4003-a3a7-31a6e6074f5b\"}","virtualmachineid":"37f0ac55-94f3-4c27-a318-83e48d6724e4","cmdEventType":"VM.MIGRATE","hostid":"2144c7a0-c0fe-4003-a3a7-31a6e6074f5b","ctxUserId":"2","httpmethod":"GET","_":"1409923786054","ctxAccountId":"2","ctxStartEventId":"1636"}, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 119169995887277, completeMsid: null, lastUpdated: null, lastPolled: null, created: null} 2014-09-05 09:29:46,919 DEBUG [c.c.a.ApiServlet] (catalina-exec-3:ctx-83b8f412 ctx-d4b68fa7) ===END=== 192.168.11.36 -- GET command=migrateVirtualMachineWithVolume&hostid=2144c7a0-c0fe-4003-a3a7-31a6e6074f5b&virtualmachineid=37f0ac55-94f3-4c27-a318-83e48d6724e4&response=json&sessionkey=%2FiZXhDJVVkXsA%2BF%2FBpWlQmV%2BbEM%3D&_=1409923786054 2014-09-05 09:29:47,003 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-117:ctx-b9105a91 job-1505 ctx-61fde895) Sync job-1506 execution on object VmWorkJobQueue.159 2014-09-05 09:29:47,004 WARN [c.c.u.d.Merovingian2] (API-Job-Executor-117:ctx-b9105a91 job-1505 ctx-61fde895) Was unable to find lock for the key vm_instance159 and thread id 1202515378 2014-09-05 09:29:47,290 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-04c93a4f) Execute sync-queue item: SyncQueueItemVO {id:263, queueId: 910, contentType: AsyncJob, contentId: 1506, lastProcessMsid: null, lastprocessNumber: null, lastProcessTime: null, created: Fri Sep 05 09:29:47 EDT 2014} 2014-09-05 09:29:47,291 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-04c93a4f) Schedule queued job-1506 2014-09-05 09:29:47,324 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506) Add job-1506 into job monitoring 2014-09-05 09:29:47,324 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506) Executing AsyncJobVO {id:1506, userId: 2, accountId: 2, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkMigrate, cmdInfo: rO0ABXNyACVjb20uY2xvdWQudm0uVm1Xb3JrTWlncmF0ZVdpdGhTdG9yYWdlsew9z6UxtXMCAANKAApkZXN0SG9zdElkSgAJc3JjSG9zdElkTAAMdm9sdW1lVG9Qb29sdAAPTGphdmEvdXRpbC9NYXA7eHIAE2NvbS5jbG91ZC52bS5WbVdvcmufmbZW8CVnawIABEoACWFjY291bnRJZEoABnVzZXJJZEoABHZtSWRMAAtoYW5kbGVyTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO3hwAAAAAAAAAAIAAAAAAAAAAgAAAAAAAACfdAAZVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbAAAAAAAAAANAAAAAAAAABtzcgARamF2YS51dGlsLkhhc2hNYXAFB9rBwxZg0QMAAkYACmxvYWRGYWN0b3JJAAl0aHJlc2hvbGR4cD9AAAAAAAAQdwgAAAAQAAAAAHg, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 119169995887277, completeMsid: null, lastUpdated: null, lastPolled: null, created: Fri Sep 05 09:29:47 EDT 2014} 2014-09-05 09:29:47,324 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506) Run VM work job: com.cloud.vm.VmWorkMigrate for VM 159, job origin: 1505 2014-09-05 09:29:47,325 DEBUG [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Execute VM work job: com.cloud.vm.VmWorkMigrateWithStorage{"srcHostId":27,"destHostId":13,"volumeToPool":{},"userId":2,"accountId":2,"vmId":159,"handlerName":"VirtualMachineManagerImpl"} 2014-09-05 09:29:47,329 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) LocalStoragePoolAllocator trying to find storage pool to fit the vm 2014-09-05 09:29:47,331 DEBUG [o.a.c.s.a.AbstractStoragePoolAllocator] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Checking if storage pool is suitable, name: null ,poolId: 4 2014-09-05 09:29:47,332 DEBUG [c.c.s.StorageManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Checking pool 4 for storage, totalSize: 976041701376, usedBytes: 35246186496, usedPct: 0.036111353076728976, disable threshold: 0.85 2014-09-05 09:29:47,334 DEBUG [c.c.s.StorageManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Found storage pool xenserver-eembacuy Local Storage of type EXT 2014-09-05 09:29:47,334 DEBUG [c.c.s.StorageManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Total capacity of the pool xenserver-eembacuy Local Storage id: 4 is 976041701376 2014-09-05 09:29:47,334 DEBUG [c.c.s.StorageManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Checking pool: 4 for volume allocation [Vol[167|vm=159|ROOT]], maxSize : 976041701376, totalAllocatedSize : 193273528320, askingSize : 0, allocated disable threshold: 0.85 2014-09-05 09:29:47,334 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Found suitable local storage pool 4, adding to list 2014-09-05 09:29:47,334 DEBUG [o.a.c.s.a.LocalStoragePoolAllocator] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) LocalStoragePoolAllocator returning 1 suitable storage pools 2014-09-05 09:29:47,334 DEBUG [o.a.c.s.a.ClusterScopeStoragePoolAllocator] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) ClusterScopeStoragePoolAllocator looking for storage pool 2014-09-05 09:29:47,334 DEBUG [o.a.c.s.a.ZoneWideStoragePoolAllocator] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) ZoneWideStoragePoolAllocator to find storage pool 2014-09-05 09:29:47,337 DEBUG [c.c.n.NetworkModelImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Service SecurityGroup is not supported in the network id=210 2014-09-05 09:29:47,339 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Preparing 1 volumes for VM[User|i-2-159-VM] 2014-09-05 09:29:47,432 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) VM state transitted from :Running to Migrating with event: MigrationRequestedvm's original host id: 27 new host id: 13 host id before state transition: 27 2014-09-05 09:29:47,435 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Hosts's actual total CPU: 4262 and CPU after applying overprovisioning: 4262 2014-09-05 09:29:47,435 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) We are allocating VM, increasing the used capacity of this host:13 2014-09-05 09:29:47,435 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Current Used CPU: 1500 , Free CPU:2762 ,Requested CPU: 500 2014-09-05 09:29:47,435 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Current Used RAM: 1207959552 , Free RAM:3023634688 ,Requested RAM: 536870912 2014-09-05 09:29:47,435 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) CPU STATS after allocation: for host: 13, old used: 1500, old reserved: 0, actual total: 4262, total with overprovisioning: 4262; new used:2000, reserved:0; requested cpu:500,alloc_from_last:false 2014-09-05 09:29:47,435 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) RAM STATS after allocation: for host: 13, old used: 1207959552, old reserved: 0, total: 4231594240; new used: 1744830464, reserved: 0; requested mem: 536870912,alloc_from_last:false 2014-09-05 09:29:47,485 DEBUG [c.c.a.t.Request] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Seq 13-4340625615855026623: Sending { Cmd , MgmtId: 119169995887277, via: 13(xenserver-eembacuy), Ver: v1, Flags: 100111, [{"com.cloud.agent.api.MigrateWithStorageCommand":{"vm":{"id":159,"name":"i-2-159-VM","bootloader":"PyGrub","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"CentOS 5.6 (64-bit)","platformEmulator":"CentOS 5 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"af08bf6289719410","params":{"hypervisortoolsversion":"xenserver56","platform":"viridian:true;acpi:1;apic:true;pae:true;nx:true"},"uuid":"37f0ac55-94f3-4c27-a318-83e48d6724e4","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"5a75c76f-6ffc-474e-9676-c12b735c6a62","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"ae5ba79f-5aab-16c5-c568-32b562f6d713","id":9,"poolType":"EXT","host":"192.168.1.71","path":"ext","port":0,"url":"EXT://192.168.1.71/ext/?ROLE=Primary&STOREUUID=ae5ba79f-5aab-16c5-c568-32b562f6d713"}},"name":"ROOT-159","size":21474836480,"path":"d7149ccf-eb75-4ae9-ab1a-a4400eb31bff","volumeId":167,"vmName":"i-2-159-VM","accountId":2,"format":"VHD","id":167,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"d7149ccf-eb75-4ae9-ab1a-a4400eb31bff","type":"ROOT","_details":{"managed":"false","storagePort":"0","storageHost":"192.168.1.71","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"296c9771-e43c-442f-a189-1dc9b13fb4d8","ip":"10.1.1.24","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:35:3f:00:50","dns1":"8.8.8.8","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://627","isolationUri":"vlan://627","isSecurityGroupEnabled":false,"name":"xenbr2"}]},"volumeToFiler":{"Vol[167|ROOT|d7149ccf-eb75-4ae9-ab1a-a4400eb31bff|21474836480]":{"id":4,"uuid":"bbef162e-2f01-33ad-245e-730f2b7e50de","host":"192.168.1.4","path":"ext","port":0,"type":"EXT"}},"wait":0}}] } 2014-09-05 09:29:47,485 DEBUG [c.c.a.t.Request] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Seq 13-4340625615855026623: Executing: { Cmd , MgmtId: 119169995887277, via: 13(xenserver-eembacuy), Ver: v1, Flags: 100111, [{"com.cloud.agent.api.MigrateWithStorageCommand":{"vm":{"id":159,"name":"i-2-159-VM","bootloader":"PyGrub","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"CentOS 5.6 (64-bit)","platformEmulator":"CentOS 5 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"af08bf6289719410","params":{"hypervisortoolsversion":"xenserver56","platform":"viridian:true;acpi:1;apic:true;pae:true;nx:true"},"uuid":"37f0ac55-94f3-4c27-a318-83e48d6724e4","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"5a75c76f-6ffc-474e-9676-c12b735c6a62","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"ae5ba79f-5aab-16c5-c568-32b562f6d713","id":9,"poolType":"EXT","host":"192.168.1.71","path":"ext","port":0,"url":"EXT://192.168.1.71/ext/?ROLE=Primary&STOREUUID=ae5ba79f-5aab-16c5-c568-32b562f6d713"}},"name":"ROOT-159","size":21474836480,"path":"d7149ccf-eb75-4ae9-ab1a-a4400eb31bff","volumeId":167,"vmName":"i-2-159-VM","accountId":2,"format":"VHD","id":167,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"d7149ccf-eb75-4ae9-ab1a-a4400eb31bff","type":"ROOT","_details":{"managed":"false","storagePort":"0","storageHost":"192.168.1.71","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"296c9771-e43c-442f-a189-1dc9b13fb4d8","ip":"10.1.1.24","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:35:3f:00:50","dns1":"8.8.8.8","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://627","isolationUri":"vlan://627","isSecurityGroupEnabled":false,"name":"xenbr2"}]},"volumeToFiler":{"Vol[167|ROOT|d7149ccf-eb75-4ae9-ab1a-a4400eb31bff|21474836480]":{"id":4,"uuid":"bbef162e-2f01-33ad-245e-730f2b7e50de","host":"192.168.1.4","path":"ext","port":0,"type":"EXT"}},"wait":0}}] } 2014-09-05 09:29:47,486 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-249:ctx-1467e9ce) Seq 13-4340625615855026623: Executing request 2014-09-05 09:29:47,504 WARN [c.c.h.x.r.XenServer610Resource] (DirectAgent-249:ctx-1467e9ce) Catch Exception com.xensource.xenapi.Types$UuidInvalid. Storage motion failed due to The uuid you supplied was invalid.
The uuid you supplied was invalid.
    at com.xensource.xenapi.Types.checkResponse(Types.java:1482)
    at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
    at com.xensource.xenapi.PIF.getByUuid(PIF.java:320)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNativeNetworkForTraffic(CitrixResourceBase.java:894) at com.cloud.hypervisor.xen.resource.XenServer610Resource.execute(XenServer610Resource.java:157) at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:92) at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65) at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
2014-09-05 09:29:47,505 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-249:ctx-1467e9ce) Seq 13-4340625615855026623: Response Received: 2014-09-05 09:29:47,505 DEBUG [c.c.a.t.Request] (DirectAgent-249:ctx-1467e9ce) Seq 13-4340625615855026623: Processing: { Ans: , MgmtId: 119169995887277, via: 13, Ver: v1, Flags: 110, [{"com.cloud.agent.api.MigrateWithStorageAnswer":{"result":false,"details":"Exception: com.xensource.xenapi.Types$UuidInvalid\nMessage: null\nStack: The uuid you supplied was invalid.\n\tat com.xensource.xenapi.Types.checkResponse(Types.java:1482)\n\tat com.xensource.xenapi.Connection.dispatch(Connection.java:350)\n\tat com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)\n\tat com.xensource.xenapi.PIF.getByUuid(PIF.java:320)\n\tat com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNativeNetworkForTraffic(CitrixResourceBase.java:894)\n\tat com.cloud.hypervisor.xen.resource.XenServer610Resource.execute(XenServer610Resource.java:157)\n\tat com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:92)\n\tat com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)\n\tat com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)\n\tat org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)\n\tat org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)\n\tat org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)\n\tat org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)\n\tat org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:262)\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat java.lang.Thread.run(Thread.java:745)\n","wait":0}}] } 2014-09-05 09:29:47,505 DEBUG [c.c.a.m.AgentAttache] (DirectAgent-249:ctx-1467e9ce) Seq 13-4340625615855026623: No more commands found 2014-09-05 09:29:47,505 DEBUG [c.c.a.t.Request] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Seq 13-4340625615855026623: Received: { Ans: , MgmtId: 119169995887277, via: 13, Ver: v1, Flags: 110, { MigrateWithStorageAnswer } } 2014-09-05 09:29:47,505 ERROR [o.a.c.s.m.XenServerStorageMotionStrategy] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Migration with storage of vm VM[User|i-2-159-VM] failed. Details: Exception: com.xensource.xenapi.Types$UuidInvalid
Message: null
Stack: The uuid you supplied was invalid.
    at com.xensource.xenapi.Types.checkResponse(Types.java:1482)
    at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
    at com.xensource.xenapi.PIF.getByUuid(PIF.java:320)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNativeNetworkForTraffic(CitrixResourceBase.java:894) at com.cloud.hypervisor.xen.resource.XenServer610Resource.execute(XenServer610Resource.java:157) at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:92) at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65) at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

2014-09-05 09:29:47,505 ERROR [o.a.c.s.m.XenServerStorageMotionStrategy] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) copy failed com.cloud.utils.exception.CloudRuntimeException: Error while migrating the vm VM[User|i-2-159-VM] to host Host[-13-Routing]. Exception: com.xensource.xenapi.Types$UuidInvalid
Message: null
Stack: The uuid you supplied was invalid.
    at com.xensource.xenapi.Types.checkResponse(Types.java:1482)
    at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
    at com.xensource.xenapi.PIF.getByUuid(PIF.java:320)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNativeNetworkForTraffic(CitrixResourceBase.java:894) at com.cloud.hypervisor.xen.resource.XenServer610Resource.execute(XenServer610Resource.java:157) at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:92) at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65) at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

at org.apache.cloudstack.storage.motion.XenServerStorageMotionStrategy.migrateVmWithVolumesWithinCluster(XenServerStorageMotionStrategy.java:211) at org.apache.cloudstack.storage.motion.XenServerStorageMotionStrategy.copyAsync(XenServerStorageMotionStrategy.java:115) at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:90) at org.apache.cloudstack.storage.volume.VolumeServiceImpl.migrateVolumes(VolumeServiceImpl.java:1144) at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.migrateVolumes(VolumeOrchestrator.java:939) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:2164) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:5239)
    at sun.reflect.GeneratedMethodAccessor466.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5335) at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
2014-09-05 09:29:47,557 WARN [o.a.c.s.d.ObjectInDataStoreManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Unsupported data object (VOLUME, org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@2bc8d34e), no need to delete from object in store ref table 2014-09-05 09:29:47,557 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Failed to migrated vm VM[User|i-2-159-VM] along with its volumes. com.cloud.utils.exception.CloudRuntimeException: Error while migrating the vm VM[User|i-2-159-VM] to host Host[-13-Routing]. Exception: com.xensource.xenapi.Types$UuidInvalid
Message: null
Stack: The uuid you supplied was invalid.
    at com.xensource.xenapi.Types.checkResponse(Types.java:1482)
    at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
    at com.xensource.xenapi.PIF.getByUuid(PIF.java:320)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNativeNetworkForTraffic(CitrixResourceBase.java:894) at com.cloud.hypervisor.xen.resource.XenServer610Resource.execute(XenServer610Resource.java:157) at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:92) at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65) at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

2014-09-05 09:29:47,557 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Migration was unsuccessful. Cleaning up: VM[User|i-2-159-VM] 2014-09-05 09:29:47,557 WARN [o.a.c.alerts] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) alertType:: 17 // dataCenterId:: 2 // podId:: 2 // clusterId:: null // message:: Unable to migrate vm i-2-159-VM from host xenserver-fbtqrtyj in zone xenzone2 and pod xenzone2 2014-09-05 09:29:47,558 DEBUG [c.c.a.AlertManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Have already sent: 1 emails for alert type '17' -- skipping send email 2014-09-05 09:29:47,558 DEBUG [c.c.a.m.AgentAttache] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Seq 13-4340625615855026624: Routed from 119169995887277 2014-09-05 09:29:47,558 DEBUG [c.c.a.t.Request] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Seq 13-4340625615855026624: Sending { Cmd , MgmtId: 119169995887277, via: 13(xenserver-eembacuy), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-2-159-VM","wait":0}}] } 2014-09-05 09:29:47,558 DEBUG [c.c.a.t.Request] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Seq 13-4340625615855026624: Executing: { Cmd , MgmtId: 119169995887277, via: 13(xenserver-eembacuy), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-2-159-VM","wait":0}}] } 2014-09-05 09:29:47,559 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-101:ctx-3b26d1ad) Seq 13-4340625615855026624: Executing request 2014-09-05 09:29:47,574 INFO [c.c.h.x.r.CitrixResourceBase] (DirectAgent-101:ctx-3b26d1ad) VM does not exist on XenServer174a7a0c-9aaf-4a3a-8896-4bd13eeacb92 2014-09-05 09:29:47,574 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-101:ctx-3b26d1ad) Seq 13-4340625615855026624: Response Received: 2014-09-05 09:29:47,574 DEBUG [c.c.a.t.Request] (DirectAgent-101:ctx-3b26d1ad) Seq 13-4340625615855026624: Processing: { Ans: , MgmtId: 119169995887277, via: 13, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM does not exist","wait":0}}] } 2014-09-05 09:29:47,574 DEBUG [c.c.a.m.AgentAttache] (DirectAgent-101:ctx-3b26d1ad) Seq 13-4340625615855026624: Unable to find listener. 2014-09-05 09:29:47,581 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) VM state transitted from :Migrating to Running with event: OperationFailedvm's original host id: 27 new host id: 27 host id before state transition: 13 2014-09-05 09:29:47,584 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Hosts's actual total CPU: 4262 and CPU after applying overprovisioning: 4262 2014-09-05 09:29:47,584 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Hosts's actual total RAM: 4231594240 and RAM after applying overprovisioning: 4231594240 2014-09-05 09:29:47,584 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) release cpu from host: 13, old used: 2000,reserved: 0, actual total: 4262, total with overprovisioning: 4262; new used: 1500,reserved:0; movedfromreserved: false,moveToReserveredfalse 2014-09-05 09:29:47,584 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) release mem from host: 13, old used: 1744830464,reserved: 0, total: 4231594240; new used: 1207959552,reserved:0; movedfromreserved: false,moveToReserveredfalse 2014-09-05 09:29:47,647 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Invocation exception, caused by: com.cloud.utils.exception.CloudRuntimeException: Failed to migrated vm VM[User|i-2-159-VM] along with its volumes. com.cloud.utils.exception.CloudRuntimeException: Error while migrating the vm VM[User|i-2-159-VM] to host Host[-13-Routing]. Exception: com.xensource.xenapi.Types$UuidInvalid
Message: null
Stack: The uuid you supplied was invalid.
    at com.xensource.xenapi.Types.checkResponse(Types.java:1482)
    at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
    at com.xensource.xenapi.PIF.getByUuid(PIF.java:320)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNativeNetworkForTraffic(CitrixResourceBase.java:894) at com.cloud.hypervisor.xen.resource.XenServer610Resource.execute(XenServer610Resource.java:157) at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:92) at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65) at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

2014-09-05 09:29:47,647 INFO [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506 ctx-aec4e979) Rethrow exception com.cloud.utils.exception.CloudRuntimeException: Failed to migrated vm VM[User|i-2-159-VM] along with its volumes. com.cloud.utils.exception.CloudRuntimeException: Error while migrating the vm VM[User|i-2-159-VM] to host Host[-13-Routing]. Exception: com.xensource.xenapi.Types$UuidInvalid
Message: null
Stack: The uuid you supplied was invalid.
    at com.xensource.xenapi.Types.checkResponse(Types.java:1482)
    at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
    at com.xensource.xenapi.PIF.getByUuid(PIF.java:320)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNativeNetworkForTraffic(CitrixResourceBase.java:894) at com.cloud.hypervisor.xen.resource.XenServer610Resource.execute(XenServer610Resource.java:157) at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:92) at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65) at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

2014-09-05 09:29:47,648 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506) Done with run of VM work job: com.cloud.vm.VmWorkMigrate for VM 159, job origin: 1505 2014-09-05 09:29:47,648 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506) Unable to complete AsyncJobVO {id:1506, userId: 2, accountId: 2, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkMigrate, cmdInfo: rO0ABXNyACVjb20uY2xvdWQudm0uVm1Xb3JrTWlncmF0ZVdpdGhTdG9yYWdlsew9z6UxtXMCAANKAApkZXN0SG9zdElkSgAJc3JjSG9zdElkTAAMdm9sdW1lVG9Qb29sdAAPTGphdmEvdXRpbC9NYXA7eHIAE2NvbS5jbG91ZC52bS5WbVdvcmufmbZW8CVnawIABEoACWFjY291bnRJZEoABnVzZXJJZEoABHZtSWRMAAtoYW5kbGVyTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO3hwAAAAAAAAAAIAAAAAAAAAAgAAAAAAAACfdAAZVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbAAAAAAAAAANAAAAAAAAABtzcgARamF2YS51dGlsLkhhc2hNYXAFB9rBwxZg0QMAAkYACmxvYWRGYWN0b3JJAAl0aHJlc2hvbGR4cD9AAAAAAAAQdwgAAAAQAAAAAHg, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 119169995887277, completeMsid: null, lastUpdated: null, lastPolled: null, created: Fri Sep 05 09:29:47 EDT 2014}, job origin:1505 com.cloud.utils.exception.CloudRuntimeException: Failed to migrated vm VM[User|i-2-159-VM] along with its volumes. com.cloud.utils.exception.CloudRuntimeException: Error while migrating the vm VM[User|i-2-159-VM] to host Host[-13-Routing]. Exception: com.xensource.xenapi.Types$UuidInvalid
Message: null
Stack: The uuid you supplied was invalid.
    at com.xensource.xenapi.Types.checkResponse(Types.java:1482)
    at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
    at com.xensource.xenapi.PIF.getByUuid(PIF.java:320)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNativeNetworkForTraffic(CitrixResourceBase.java:894) at com.cloud.hypervisor.xen.resource.XenServer610Resource.execute(XenServer610Resource.java:157) at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:92) at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65) at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

at org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.migrateVolumes(VolumeOrchestrator.java:944) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:2164) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateWithStorage(VirtualMachineManagerImpl.java:5239)
    at sun.reflect.GeneratedMethodAccessor466.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5335) at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
2014-09-05 09:29:47,649 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506) Complete async job-1506, jobStatus: FAILED, resultCode: 0, result: rO0ABXNyABpqYXZhLmxhbmcuUnVudGltZUV4Y2VwdGlvbp5fBkcKNIPlAgAAeHIAE2phdmEubGFuZy5FeGNlcHRpb27Q_R8-GjscxAIAAHhyABNqYXZhLmxhbmcuVGhyb3dhYmxl1cY1Jzl3uMsDAARMAAVjYXVzZXQAFUxqYXZhL2xhbmcvVGhyb3dhYmxlO0wADWRldGFpbE1lc3NhZ2V0ABJMamF2YS9sYW5nL1N0cmluZztbAApzdGFja1RyYWNldAAeW0xqYXZhL2xhbmcvU3RhY2tUcmFjZUVsZW1lbnQ7TAAUc3VwcHJlc3NlZEV4Y2VwdGlvbnN0ABBMamF2YS91dGlsL0xpc3Q7eHBxAH4AB3QI5UpvYiBmYWlsZWQgZHVlIHRvIGV4Y2VwdGlvbiBGYWlsZWQgdG8gbWlncmF0ZWQgdm0gVk1bVXNlcnxpLTItMTU5LVZNXSBhbG9uZyB3aXRoIGl0cyB2b2x1bWVzLiBjb20uY2xvdWQudXRpbHMuZXhjZXB0aW9uLkNsb3VkUnVudGltZUV4Y2VwdGlvbjogRXJyb3Igd2hpbGUgbWlncmF0aW5nIHRoZSB2bSBWTVtVc2VyfGktMi0xNTktVk1dIHRvIGhvc3QgSG9zdFstMTMtUm91dGluZ10uIEV4Y2VwdGlvbjogY29tLnhlbnNvdXJjZS54ZW5hcGkuVHlwZXMkVXVpZEludmFsaWQKTWVzc2FnZTogbnVsbApTdGFjazogVGhlIHV1aWQgeW91IHN1cHBsaWVkIHdhcyBpbnZhbGlkLgoJYXQgY29tLnhlbnNvdXJjZS54ZW5hcGkuVHlwZXMuY2hlY2tSZXNwb25zZShUeXBlcy5qYXZhOjE0ODIpCglhdCBjb20ueGVuc291cmNlLnhlbmFwaS5Db25uZWN0aW9uLmRpc3BhdGNoKENvbm5lY3Rpb24uamF2YTozNTApCglhdCBjb20uY2xvdWQuaHlwZXJ2aXNvci54ZW4ucmVzb3VyY2UuWGVuU2VydmVyQ29ubmVjdGlvblBvb2wkWGVuU2VydmVyQ29ubmVjdGlvbi5kaXNwYXRjaChYZW5TZXJ2ZXJDb25uZWN0aW9uUG9vbC5qYXZhOjQ1NikKCWF0IGNvbS54ZW5zb3VyY2UueGVuYXBpLlBJRi5nZXRCeVV1aWQoUElGLmphdmE6MzIwKQoJYXQgY29tLmNsb3VkLmh5cGVydmlzb3IueGVuLnJlc291cmNlLkNpdHJpeFJlc291cmNlQmFzZS5nZXROYXRpdmVOZXR3b3JrRm9yVHJhZmZpYyhDaXRyaXhSZXNvdXJjZUJhc2UuamF2YTo4OTQpCglhdCBjb20uY2xvdWQuaHlwZXJ2aXNvci54ZW4ucmVzb3VyY2UuWGVuU2VydmVyNjEwUmVzb3VyY2UuZXhlY3V0ZShYZW5TZXJ2ZXI2MTBSZXNvdXJjZS5qYXZhOjE1NykKCWF0IGNvbS5jbG91ZC5oeXBlcnZpc29yLnhlbi5yZXNvdXJjZS5YZW5TZXJ2ZXI2MTBSZXNvdXJjZS5leGVjdXRlUmVxdWVzdChYZW5TZXJ2ZXI2MTBSZXNvdXJjZS5qYXZhOjkyKQoJYXQgY29tLmNsb3VkLmh5cGVydmlzb3IueGVuLnJlc291cmNlLlhlblNlcnZlcjYyMFNQMVJlc291cmNlLmV4ZWN1dGVSZXF1ZXN0KFhlblNlcnZlcjYyMFNQMVJlc291cmNlLmphdmE6NjUpCglhdCBjb20uY2xvdWQuYWdlbnQubWFuYWdlci5EaXJlY3RBZ2VudEF0dGFjaGUkVGFzay5ydW5JbkNvbnRleHQoRGlyZWN0QWdlbnRBdHRhY2hlLmphdmE6MjE2KQoJYXQgb3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDEucnVuKE1hbmFnZWRDb250ZXh0UnVubmFibGUuamF2YTo0OSkKCWF0IG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMS5jYWxsKERlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhOjU2KQoJYXQgb3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5pbXBsLkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5jYWxsV2l0aENvbnRleHQoRGVmYXVsdE1hbmFnZWRDb250ZXh0LmphdmE6MTAzKQoJYXQgb3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5pbXBsLkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5ydW5XaXRoQ29udGV4dChEZWZhdWx0TWFuYWdlZENvbnRleHQuamF2YTo1MykKCWF0IG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZS5ydW4oTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhOjQ2KQoJYXQgamF2YS51dGlsLmNvbmN1cnJlbnQuRXhlY3V0b3JzJFJ1bm5hYmxlQWRhcHRlci5jYWxsKEV4ZWN1dG9ycy5qYXZhOjQ3MSkKCWF0IGphdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2sucnVuKEZ1dHVyZVRhc2suamF2YToyNjIpCglhdCBqYXZhLnV0aWwuY29uY3VycmVudC5TY2hlZHVsZWRUaHJlYWRQb29sRXhlY3V0b3IkU2NoZWR1bGVkRnV0dXJlVGFzay5hY2Nlc3MkMjAxKFNjaGVkdWxlZFRocmVhZFBvb2xFeGVjdXRvci5qYXZhOjE3OCkKCWF0IGphdmEudXRpbC5jb25jdXJyZW50LlNjaGVkdWxlZFRocmVhZFBvb2xFeGVjdXRvciRTY2hlZHVsZWRGdXR1cmVUYXNrLnJ1bihTY2hlZHVsZWRUaHJlYWRQb29sRXhlY3V0b3IuamF2YToyOTIpCglhdCBqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IucnVuV29ya2VyKFRocmVhZFBvb2xFeGVjdXRvci5qYXZhOjExNDUpCglhdCBqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VyLnJ1bihUaHJlYWRQb29sRXhlY3V0b3IuamF2YTo2MTUpCglhdCBqYXZhLmxhbmcuVGhyZWFkLnJ1bihUaHJlYWQuamF2YTo3NDUpCnVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAADXNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAETAAIZmlsZU5hbWVxAH4ABEwACm1ldGhvZE5hbWVxAH4ABHhwAAAAcnQAIGNvbS5jbG91ZC52bS5WbVdvcmtKb2JEaXNwYXRjaGVydAAYVm1Xb3JrSm9iRGlzcGF0Y2hlci5qYXZhdAAGcnVuSm9ic3EAfgALAAAB93QAP29yZy5hcGFjaGUuY2xvdWRzdGFjay5mcmFtZXdvcmsuam9icy5pbXBsLkFzeW5jSm9iTWFuYWdlckltcGwkNXQAGEFzeW5jSm9iTWFuYWdlckltcGwuamF2YXQADHJ1bkluQ29udGV4dHNxAH4ACwAAADF0AD5vcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGUkMXQAG01hbmFnZWRDb250ZXh0UnVubmFibGUuamF2YXQAA3J1bnNxAH4ACwAAADh0AEJvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0JDF0ABpEZWZhdWx0TWFuYWdlZENvbnRleHQuamF2YXQABGNhbGxzcQB-AAsAAABndABAb3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5pbXBsLkRlZmF1bHRNYW5hZ2VkQ29udGV4dHEAfgAadAAPY2FsbFdpdGhDb250ZXh0c3EAfgALAAAANXEAfgAdcQB-ABp0AA5ydW5XaXRoQ29udGV4dHNxAH4ACwAAAC50ADxvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGVxAH4AFnEAfgAXc3EAfgALAAABzHEAfgARcQB-ABJxAH4AF3NxAH4ACwAAAdd0AC5qYXZhLnV0aWwuY29uY3VycmVudC5FeGVjdXRvcnMkUnVubmFibGVBZGFwdGVydAAORXhlY3V0b3JzLmphdmFxAH4AG3NxAH4ACwAAAQZ0AB9qYXZhLnV0aWwuY29uY3VycmVudC5GdXR1cmVUYXNrdAAPRnV0dXJlVGFzay5qYXZhcQB-ABdzcQB-AAsAAAR5dAAnamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5Xb3JrZXJzcQB-AAsAAAJndAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZWN1dG9yJFdvcmtlcnEAfgAscQB-ABdzcQB-AAsAAALpdAAQamF2YS5sYW5nLlRocmVhZHQAC1RocmVhZC5qYXZhcQB-ABdzcgAmamF2YS51dGlsLkNvbGxlY3Rpb25zJFVubW9kaWZpYWJsZUxpc3T8DyUxteyOEAIAAUwABGxpc3RxAH4ABnhyACxqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlQ29sbGVjdGlvbhlCAIDLXvceAgABTAABY3QAFkxqYXZhL3V0aWwvQ29sbGVjdGlvbjt4cHNyABNqYXZhLnV0aWwuQXJyYXlMaXN0eIHSHZnHYZ0DAAFJAARzaXpleHAAAAAAdwQAAAAAeHEAfgA4eA 2014-09-05 09:29:47,681 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506) Done executing com.cloud.vm.VmWorkMigrate for job-1506 2014-09-05 09:29:47,781 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506) Sync queue (910) is currently empty 2014-09-05 09:29:47,782 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-50:ctx-4b8faae5 job-1505/job-1506) Remove job-1506 from job monitoring 2014-09-05 09:29:47,814 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-117:ctx-b9105a91 job-1505) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd java.lang.RuntimeException: Job failed due to exception Failed to migrated vm VM[User|i-2-159-VM] along with its volumes. com.cloud.utils.exception.CloudRuntimeException: Error while migrating the vm VM[User|i-2-159-VM] to host Host[-13-Routing]. Exception: com.xensource.xenapi.Types$UuidInvalid
Message: null
Stack: The uuid you supplied was invalid.
    at com.xensource.xenapi.Types.checkResponse(Types.java:1482)
    at com.xensource.xenapi.Connection.dispatch(Connection.java:350)
at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)
    at com.xensource.xenapi.PIF.getByUuid(PIF.java:320)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNativeNetworkForTraffic(CitrixResourceBase.java:894) at com.cloud.hypervisor.xen.resource.XenServer610Resource.execute(XenServer610Resource.java:157) at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:92) at com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65) at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:114) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
2014-09-05 09:29:47,815 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-117:ctx-b9105a91 job-1505) Complete async job-1505, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Job failed due to exception Failed to migrated vm VM[User|i-2-159-VM] along with its volumes. com.cloud.utils.exception.CloudRuntimeException: Error while migrating the vm VM[User|i-2-159-VM] to host Host[-13-Routing]. Exception: com.xensource.xenapi.Types$UuidInvalid\nMessage: null\nStack: The uuid you supplied was invalid.\n\tat com.xensource.xenapi.Types.checkResponse(Types.java:1482)\n\tat com.xensource.xenapi.Connection.dispatch(Connection.java:350)\n\tat com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:456)\n\tat com.xensource.xenapi.PIF.getByUuid(PIF.java:320)\n\tat com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNativeNetworkForTraffic(CitrixResourceBase.java:894)\n\tat com.cloud.hypervisor.xen.resource.XenServer610Resource.execute(XenServer610Resource.java:157)\n\tat com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:92)\n\tat com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)\n\tat com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)\n\tat org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)\n\tat org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)\n\tat org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)\n\tat org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)\n\tat org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:262)\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat java.lang.Thread.run(Thread.java:745)\n"} 2014-09-05 09:29:47,847 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-117:ctx-b9105a91 job-1505) Done executing org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd for job-1505 2014-09-05 09:29:47,880 INFO [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-117:ctx-b9105a91 job-1505) Remove job-1505 from job monitoring 2014-09-05 09:29:49,998 DEBUG [c.c.a.ApiServlet] (catalina-exec-21:ctx-1e5e47fe) ===START=== 192.168.11.36 -- GET command=queryAsyncJobResult&jobId=ac21b2b7-1179-473d-9508-1db8b21d1c6a&response=json&sessionkey=%2FiZXhDJVVkXsA%2BF%2FBpWlQmV%2BbEM%3D&_=1409923789231 2014-09-05 09:29:50,035 DEBUG [c.c.a.ApiServlet] (catalina-exec-21:ctx-1e5e47fe ctx-637ae957) ===END=== 192.168.11.36 -- GET command=queryAsyncJobResult&jobId=ac21b2b7-1179-473d-9508-1db8b21d1c6a&response=json&sessionkey=%2FiZXhDJVVkXsA%2BF%2FBpWlQmV%2BbEM%3D&_=1409923789231 2014-09-05 09:29:50,717 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-d1acbf10) AutoScaling Monitor is running...


Reply via email to