Abhinav Roy created CLOUDSTACK-7372:
---------------------------------------
Summary: [vGPU] When a host is put in maintenance mode, vGPU
enabled VMs failed to migrate to the other host in the cluster
Key: CLOUDSTACK-7372
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7372
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: Management Server
Affects Versions: 4.5.0
Environment: Xenserver 6.2.5 + PV drivers
CS - 4.5.0
Reporter: Abhinav Roy
Assignee: Sanjay Tripathi
Priority: Critical
Fix For: 4.5.0
Steps :
==========================
1. Deploy an advanced zone CS setup with 2 XEN hosts in a cluster having same
configuration and vGPU cards.
2. Create 2 VMs with vGPU offering on Host1
3. Put Host1 in maintenance mode
Expected behavior:
=========================
VMs should migrate to Host 2 and Host1 should go to "maintenance" state
Observed behavior :
=========================
VMs fail to migrate to Host2 with the following error and Host1 is stuck in
"prepareformaintenance" state.
2014-08-18 19:16:00,608 DEBUG [c.c.a.ApiServlet] (catalina-exec-1:ctx-a55c13e9)
===START=== 10.144.7.6 -- GET
command=prepareHostForMaintenance&id=f68de59b-3d28-452f-b82b-86fca83f6f16&response=json&sessionkey=XUBDEpbw7%2BbvhRUGB0j2dKXFW08%3D&_=1408369336934
2014-08-18 19:16:00,651 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(catalina-exec-1:ctx-a55c13e9 ctx-5208dce9) submit async job-690, details:
AsyncJobVO {id:690, userId: 2, accountId: 2, instanceType: Host, instanceId: 8,
cmd: org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd,
cmdInfo:
{"id":"f68de59b-3d28-452f-b82b-86fca83f6f16","response":"json","sessionkey":"XUBDEpbw7+bvhRUGB0j2dKXFW08\u003d","ctxDetails":"{\"com.cloud.host.Host\":\"f68de59b-3d28-452f-b82b-86fca83f6f16\"}","cmdEventType":"MAINT.PREPARE","ctxUserId":"2","httpmethod":"GET","_":"1408369336934","uuid":"f68de59b-3d28-452f-b82b-86fca83f6f16","ctxAccountId":"2","ctxStartEventId":"983"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 213737702773493, completeMsid: null, lastUpdated: null,
lastPolled: null, created: null}
2014-08-18 19:16:00,653 DEBUG [c.c.a.ApiServlet] (catalina-exec-1:ctx-a55c13e9
ctx-5208dce9) ===END=== 10.144.7.6 -- GET
command=prepareHostForMaintenance&id=f68de59b-3d28-452f-b82b-86fca83f6f16&response=json&sessionkey=XUBDEpbw7%2BbvhRUGB0j2dKXFW08%3D&_=1408369336934
2014-08-18 19:16:00,654 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-3:ctx-5d223414 job-690) Add job-690 into job monitoring
2014-08-18 19:16:00,655 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-3:ctx-5d223414 job-690) Executing AsyncJobVO {id:690, userId:
2, accountId: 2, instanceType: Host, instanceId: 8, cmd:
org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd, cmdInfo:
{"id":"f68de59b-3d28-452f-b82b-86fca83f6f16","response":"json","sessionkey":"XUBDEpbw7+bvhRUGB0j2dKXFW08\u003d","ctxDetails":"{\"com.cloud.host.Host\":\"f68de59b-3d28-452f-b82b-86fca83f6f16\"}","cmdEventType":"MAINT.PREPARE","ctxUserId":"2","httpmethod":"GET","_":"1408369336934","uuid":"f68de59b-3d28-452f-b82b-86fca83f6f16","ctxAccountId":"2","ctxStartEventId":"983"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 213737702773493, completeMsid: null, lastUpdated: null,
lastPolled: null, created: null}
2014-08-18 19:16:00,683 DEBUG [c.c.a.t.Request]
(API-Job-Executor-3:ctx-5d223414 job-690 ctx-4be20e99) Seq
8-5623307084725485586: Sending { Cmd , MgmtId: 213737702773493, via:
8(cldstk-R720-66), Ver: v1, Flags: 100111,
[{"com.cloud.agent.api.MaintainCommand":{"wait":0}}] }
2014-08-18 19:16:00,684 DEBUG [c.c.a.t.Request]
(API-Job-Executor-3:ctx-5d223414 job-690 ctx-4be20e99) Seq
8-5623307084725485586: Executing: { Cmd , MgmtId: 213737702773493, via:
8(cldstk-R720-66), Ver: v1, Flags: 100111,
[{"com.cloud.agent.api.MaintainCommand":{"wait":0}}] }
2014-08-18 19:16:00,684 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-30:ctx-d6521262) Seq 8-5623307084725485586: Executing request
2014-08-18 19:16:00,713 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-30:ctx-d6521262) Seq 8-5623307084725485586: Response Received:
2014-08-18 19:16:00,713 DEBUG [c.c.a.t.Request] (DirectAgent-30:ctx-d6521262)
Seq 8-5623307084725485586: Processing: { Ans: , MgmtId: 213737702773493, via:
8, Ver: v1, Flags: 110,
[{"com.cloud.agent.api.MaintainAnswer":{"willMigrate":true,"result":true,"wait":0}}]
}
2014-08-18 19:16:00,713 DEBUG [c.c.a.t.Request]
(API-Job-Executor-3:ctx-5d223414 job-690 ctx-4be20e99) Seq
8-5623307084725485586: Received: { Ans: , MgmtId: 213737702773493, via: 8,
Ver: v1, Flags: 110, { MaintainAnswer } }
2014-08-18 19:16:00,717 DEBUG [c.c.a.m.AgentAttache]
(DirectAgent-30:ctx-d6521262) Seq 8-5623307084725485586: No more commands found
2014-08-18 19:16:00,725 DEBUG [c.c.r.ResourceState]
(API-Job-Executor-3:ctx-5d223414 job-690 ctx-4be20e99) Resource state update:
[id = 8; name = cldstk-R720-66; old state = Enabled; event =
AdminAskMaintenace; new state = PrepareForMaintenance]
2014-08-18 19:16:00,725 DEBUG [c.c.a.m.AgentAttache]
(API-Job-Executor-3:ctx-5d223414 job-690 ctx-4be20e99) Seq
8-5623307084725485570: Sending disconnect to class
com.cloud.network.security.SecurityGroupListener
2014-08-18 19:16:00,750 INFO [c.c.h.HighAvailabilityManagerImpl]
(HA-Worker-0:ctx-258603ac work-6) Processing
HAWork[6-Migration-44-Running-Scheduled]
2014-08-18 19:16:00,758 WARN [o.a.c.f.j.AsyncJobExecutionContext]
(HA-Worker-0:ctx-258603ac work-6) Job is executed without a context, setup
psudo job for the executing thread
2014-08-18 19:16:00,769 INFO [c.c.h.HighAvailabilityManagerImpl]
(HA-Worker-1:ctx-aab890b8 work-7) Processing
HAWork[7-Migration-45-Running-Scheduled]
2014-08-18 19:16:00,780 WARN [o.a.c.f.j.AsyncJobExecutionContext]
(HA-Worker-1:ctx-aab890b8 work-7) Job is executed without a context, setup
psudo job for the executing thread
2014-08-18 19:16:00,785 DEBUG [c.c.r.ResourceManagerImpl]
(API-Job-Executor-3:ctx-5d223414 job-690 ctx-4be20e99) Sent resource event
EVENT_PREPARE_MAINTENANCE_AFTER to listener CapacityManagerImpl
2014-08-18 19:16:00,834 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(HA-Worker-0:ctx-258603ac work-6) Sync job-693 execution on object
VmWorkJobQueue.44
2014-08-18 19:16:00,837 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(HA-Worker-1:ctx-aab890b8 work-7) Sync job-694 execution on object
VmWorkJobQueue.45
2014-08-18 19:16:00,839 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-3:ctx-5d223414 job-690 ctx-4be20e99) Complete async job-690,
jobStatus: SUCCEEDED, resultCode: 0, result:
org.apache.cloudstack.api.response.HostResponse/host/{"id":"f68de59b-3d28-452f-b82b-86fca83f6f16","name":"cldstk-R720-66","state":"Up","disconnected":"2014-08-18T19:11:52+0530","type":"Routing","ipaddress":"10.102.192.66","zoneid":"5c4147e0-8545-454e-8761-191d90fded70","zonename":"vgpu","podid":"25c2e27b-b43b-4fba-a775-304e6dc5ac02","podname":"vgpu","version":"4.5.0-SNAPSHOT","hypervisor":"XenServer","cpusockets":2,"cpunumber":24,"cpuspeed":2300,"cpuallocated":"0%","cpuused":"0%","cpuwithoverprovisioning":"55200.0","networkkbsread":0,"networkkbswrite":0,"memorytotal":15719168512,"memoryallocated":0,"memoryused":0,"gpugroup":[{"gpugroupname":"Group
of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpu":[{"vgputype":"GRID
K220Q","videoram":503316480,"maxheads":2,"maxresolutionx":2560,"maxresolutiony":1600,"maxvgpuperpgpu":8,"remainingcapacity":0,"maxcapacity":16},{"vgputype":"GRID
K200","videoram":268435456,"maxheads":2,"maxresolutionx":1920,"maxresolutiony":1200,"maxvgpuperpgpu":8,"remainingcapacity":7,"maxcapacity":16},{"vgputype":"passthrough","videoram":0,"maxheads":0,"maxresolutionx":0,"maxresolutiony":0,"maxvgpuperpgpu":1,"remainingcapacity":0,"maxcapacity":2},{"vgputype":"GRID
K260Q","videoram":2013265920,"maxheads":4,"maxresolutionx":2560,"maxresolutiony":1600,"maxvgpuperpgpu":2,"remainingcapacity":0,"maxcapacity":4},{"vgputype":"GRID
K240Q","videoram":1006632960,"maxheads":2,"maxresolutionx":2560,"maxresolutiony":1600,"maxvgpuperpgpu":4,"remainingcapacity":3,"maxcapacity":8}]}],"capabilities":"xen-3.0-x86_64
, xen-3.0-x86_32p , hvm-3.0-x86_32 , hvm-3.0-x86_32p ,
hvm-3.0-x86_64","lastpinged":"1970-01-17T03:32:40+0530","managementserverid":213737702773493,"clusterid":"d9404bb3-55f7-4e33-acb8-ba6a447e84d3","clustername":"vgpu1","clustertype":"CloudManaged","islocalstorageactive":false,"created":"2014-08-18T18:33:02+0530","events":"AgentConnected;
HostDown; PingTimeout; ManagementServerDown; ShutdownRequested; Remove;
StartAgentRebalance; Ping;
AgentDisconnected","resourcestate":"PrepareForMaintenance","hypervisorversion":"6.2.0","hahost":false,"jobid":"94713beb-05bc-4dee-becc-c4ea303e86d0","jobstatus":0}
2014-08-18 19:16:00,858 WARN [c.c.u.d.Merovingian2] (HA-Worker-1:ctx-aab890b8
work-7) Was unable to find lock for the key vm_instance45 and thread id
1183613611
2014-08-18 19:16:00,858 WARN [c.c.u.d.Merovingian2] (HA-Worker-0:ctx-258603ac
work-6) Was unable to find lock for the key vm_instance44 and thread id
741663394
2014-08-18 19:16:00,873 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-3:ctx-5d223414 job-690) Done executing
org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd for
job-690
2014-08-18 19:16:00,885 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-3:ctx-5d223414 job-690) Remove job-690 from job monitoring
2014-08-18 19:16:01,259 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-d7897592) Execute sync-queue item: SyncQueueItemVO
{id:285, queueId: 261, contentType: AsyncJob, contentId: 693, lastProcessMsid:
null, lastprocessNumber: null, lastProcessTime: null, created: Mon Aug 18
19:16:00 IST 2014}
2014-08-18 19:16:01,261 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-d7897592) Schedule queued job-693
2014-08-18 19:16:01,264 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-d7897592) Execute sync-queue item: SyncQueueItemVO
{id:286, queueId: 262, contentType: AsyncJob, contentId: 694, lastProcessMsid:
null, lastprocessNumber: null, lastProcessTime: null, created: Mon Aug 18
19:16:00 IST 2014}
2014-08-18 19:16:01,266 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-d7897592) Schedule queued job-694
2014-08-18 19:16:01,268 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693) Add job-693 into job
monitoring
2014-08-18 19:16:01,269 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693) Executing AsyncJobVO
{id:693, userId: 1, accountId: 1, instanceType: null, instanceId: null, cmd:
com.cloud.vm.VmWorkMigrateAway, cmdInfo:
rO0ABXNyAB5jb20uY2xvdWQudm0uVm1Xb3JrTWlncmF0ZUF3YXmt4MX4jtcEmwIAAUoACXNyY0hvc3RJZHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAALHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAACA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 213737702773493, completeMsid: null, lastUpdated: null,
lastPolled: null, created: Mon Aug 18 19:16:00 IST 2014}
2014-08-18 19:16:01,270 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693) Run VM work job:
com.cloud.vm.VmWorkMigrateAway for VM 44, job origin: 691
2014-08-18 19:16:01,272 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Execute VM work
job:
com.cloud.vm.VmWorkMigrateAway{"srcHostId":8,"userId":1,"accountId":1,"vmId":44,"handlerName":"VirtualMachineManagerImpl"}
2014-08-18 19:16:01,276 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694) Add job-694 into job
monitoring
2014-08-18 19:16:01,278 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694) Executing AsyncJobVO
{id:694, userId: 1, accountId: 1, instanceType: null, instanceId: null, cmd:
com.cloud.vm.VmWorkMigrateAway, cmdInfo:
rO0ABXNyAB5jb20uY2xvdWQudm0uVm1Xb3JrTWlncmF0ZUF3YXmt4MX4jtcEmwIAAUoACXNyY0hvc3RJZHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAALXQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAACA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 213737702773493, completeMsid: null, lastUpdated: null,
lastPolled: null, created: Mon Aug 18 19:16:00 IST 2014}
2014-08-18 19:16:01,279 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694) Run VM work job:
com.cloud.vm.VmWorkMigrateAway for VM 45, job origin: 692
2014-08-18 19:16:01,282 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Execute VM work
job:
com.cloud.vm.VmWorkMigrateAway{"srcHostId":8,"userId":1,"accountId":1,"vmId":45,"handlerName":"VirtualMachineManagerImpl"}
2014-08-18 19:16:01,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Deploy avoids
pods: [], clusters: [], hosts: [8]
2014-08-18 19:16:01,323 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Deploy avoids
pods: [], clusters: [], hosts: [8]
2014-08-18 19:16:01,326 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d)
DeploymentPlanner allocation algorithm:
com.cloud.deploy.FirstFitPlanner@37eb3935
2014-08-18 19:16:01,326 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Trying to
allocate a host and storage pools from dc:1, pod:1,cluster:1, requested cpu:
3200, requested ram: 3221225472
2014-08-18 19:16:01,327 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Is ROOT volume
READY (pool already allocated)?: Yes
2014-08-18 19:16:01,327 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) This VM has
last host_id specified, trying to choose the same host: 8
2014-08-18 19:16:01,326 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a)
DeploymentPlanner allocation algorithm:
com.cloud.deploy.FirstFitPlanner@37eb3935
2014-08-18 19:16:01,328 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Trying to
allocate a host and storage pools from dc:1, pod:1,cluster:1, requested cpu:
3200, requested ram: 3221225472
2014-08-18 19:16:01,328 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Is ROOT volume
READY (pool already allocated)?: Yes
2014-08-18 19:16:01,328 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) This VM has
last host_id specified, trying to choose the same host: 8
2014-08-18 19:16:01,330 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) The last host
of this VM is in avoid set
2014-08-18 19:16:01,332 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Cannot choose
the last host to deploy this VM
2014-08-18 19:16:01,332 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Searching
resources only under specified Cluster: 1
2014-08-18 19:16:01,332 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) The last host
of this VM is in avoid set
2014-08-18 19:16:01,333 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Cannot choose
the last host to deploy this VM
2014-08-18 19:16:01,333 DEBUG [c.c.d.FirstFitPlanner]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Searching
resources only under specified Cluster: 1
2014-08-18 19:16:01,351 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Checking
resources in Cluster: 1 under Pod: 1
2014-08-18 19:16:01,351 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Checking
resources in Cluster: 1 under Pod: 1
2014-08-18 19:16:01,357 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d
FirstFitRoutingAllocator) Looking for hosts in dc: 1 pod:1 cluster:1
2014-08-18 19:16:01,357 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a
FirstFitRoutingAllocator) Looking for hosts in dc: 1 pod:1 cluster:1
2014-08-18 19:16:01,366 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d
FirstFitRoutingAllocator) FirstFitAllocator has 2 hosts to check for
allocation: [Host[-5-Routing], Host[-1-Routing]]
2014-08-18 19:16:01,368 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a
FirstFitRoutingAllocator) FirstFitAllocator has 2 hosts to check for
allocation: [Host[-5-Routing], Host[-1-Routing]]
2014-08-18 19:16:01,383 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d
FirstFitRoutingAllocator) Found 2 hosts for allocation after prioritization:
[Host[-5-Routing], Host[-1-Routing]]
2014-08-18 19:16:01,383 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d
FirstFitRoutingAllocator) Looking for speed=3200Mhz, Ram=3072
2014-08-18 19:16:01,383 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a
FirstFitRoutingAllocator) Found 2 hosts for allocation after prioritization:
[Host[-5-Routing], Host[-1-Routing]]
2014-08-18 19:16:01,384 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a
FirstFitRoutingAllocator) Looking for speed=3200Mhz, Ram=3072
2014-08-18 19:16:01,399 DEBUG [c.c.r.ResourceManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a
FirstFitRoutingAllocator) Host ID: 5 does not have GPU device available
2014-08-18 19:16:01,399 INFO [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a
FirstFitRoutingAllocator) Host name: cldstk-R720-55, hostId: 5 does not have
required GPU devices available
2014-08-18 19:16:01,399 DEBUG [c.c.r.ResourceManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d
FirstFitRoutingAllocator) Host ID: 5 does not have GPU device available
2014-08-18 19:16:01,400 INFO [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d
FirstFitRoutingAllocator) Host name: cldstk-R720-55, hostId: 5 does not have
required GPU devices available
2014-08-18 19:16:01,423 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d
FirstFitRoutingAllocator) Host: 1 has cpu capability (cpu:24, speed:2300) to
support requested CPU: 2 and requested speed: 1600
2014-08-18 19:16:01,423 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d
FirstFitRoutingAllocator) Checking if host: 1 has enough capacity for requested
CPU: 3200 and requested RAM: 3221225472 , cpuOverprovisioningFactor: 1.0
2014-08-18 19:16:01,423 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a
FirstFitRoutingAllocator) Host: 1 has cpu capability (cpu:24, speed:2300) to
support requested CPU: 2 and requested speed: 1600
2014-08-18 19:16:01,424 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a
FirstFitRoutingAllocator) Checking if host: 1 has enough capacity for requested
CPU: 3200 and requested RAM: 3221225472 , cpuOverprovisioningFactor: 1.0
2014-08-18 19:16:01,430 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a
FirstFitRoutingAllocator) Hosts's actual total CPU: 55200 and CPU after
applying overprovisioning: 55200
2014-08-18 19:16:01,430 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a
FirstFitRoutingAllocator) Free CPU: 53200 , Requested CPU: 3200
2014-08-18 19:16:01,430 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a
FirstFitRoutingAllocator) Free RAM: 13840119808 , Requested RAM: 3221225472
2014-08-18 19:16:01,431 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a
FirstFitRoutingAllocator) Host has enough CPU and RAM available
2014-08-18 19:16:01,431 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a
FirstFitRoutingAllocator) STATS: Can alloc CPU from host: 1, used: 2000,
reserved: 0, actual total: 55200, total with overprovisioning: 55200; requested
cpu:3200,alloc_from_last_host?:false ,considerReservedCapacity?: true
2014-08-18 19:16:01,431 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a
FirstFitRoutingAllocator) STATS: Can alloc MEM from host: 1, used: 1879048192,
reserved: 0, total: 15719168000; requested mem:
3221225472,alloc_from_last_host?:false ,considerReservedCapacity?: true
2014-08-18 19:16:01,431 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a
FirstFitRoutingAllocator) Found a suitable host, adding to list: 1
2014-08-18 19:16:01,431 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a
FirstFitRoutingAllocator) Host Allocator returning 1 suitable hosts
2014-08-18 19:16:01,430 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d
FirstFitRoutingAllocator) Hosts's actual total CPU: 55200 and CPU after
applying overprovisioning: 55200
2014-08-18 19:16:01,432 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d
FirstFitRoutingAllocator) Free CPU: 53200 , Requested CPU: 3200
2014-08-18 19:16:01,432 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d
FirstFitRoutingAllocator) Free RAM: 13840119808 , Requested RAM: 3221225472
2014-08-18 19:16:01,432 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d
FirstFitRoutingAllocator) Host has enough CPU and RAM available
2014-08-18 19:16:01,432 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d
FirstFitRoutingAllocator) STATS: Can alloc CPU from host: 1, used: 2000,
reserved: 0, actual total: 55200, total with overprovisioning: 55200; requested
cpu:3200,alloc_from_last_host?:false ,considerReservedCapacity?: true
2014-08-18 19:16:01,432 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d
FirstFitRoutingAllocator) STATS: Can alloc MEM from host: 1, used: 1879048192,
reserved: 0, total: 15719168000; requested mem:
3221225472,alloc_from_last_host?:false ,considerReservedCapacity?: true
2014-08-18 19:16:01,432 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d
FirstFitRoutingAllocator) Found a suitable host, adding to list: 1
2014-08-18 19:16:01,432 DEBUG [c.c.a.m.a.i.FirstFitAllocator]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d
FirstFitRoutingAllocator) Host Allocator returning 1 suitable hosts
2014-08-18 19:16:01,439 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Checking
suitable pools for volume (Id, Type): (60,ROOT)
2014-08-18 19:16:01,441 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Volume has pool
already allocated, checking if pool can be reused, poolId: 1
2014-08-18 19:16:01,441 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Checking
suitable pools for volume (Id, Type): (59,ROOT)
2014-08-18 19:16:01,442 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Volume has pool
already allocated, checking if pool can be reused, poolId: 1
2014-08-18 19:16:01,443 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Planner need
not allocate a pool for this volume since its READY
2014-08-18 19:16:01,445 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Trying to find
a potenial host and associated storage pools from the suitable host/pool lists
for this VM
2014-08-18 19:16:01,444 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Planner need
not allocate a pool for this volume since its READY
2014-08-18 19:16:01,446 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Trying to find
a potenial host and associated storage pools from the suitable host/pool lists
for this VM
2014-08-18 19:16:01,446 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Checking if
host: 1 can access any suitable storage pool for volume: ROOT
2014-08-18 19:16:01,446 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Checking if
host: 1 can access any suitable storage pool for volume: ROOT
2014-08-18 19:16:01,452 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Host: 1 can
access pool: 1
2014-08-18 19:16:01,455 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Host: 1 can
access pool: 1
2014-08-18 19:16:01,455 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Found a
potential host id: 1 name: cldstk-R720-10 and associated storage pools for this
VM
2014-08-18 19:16:01,460 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Found a
potential host id: 1 name: cldstk-R720-10 and associated storage pools for this
VM
2014-08-18 19:16:01,461 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Returning
Deployment Destination:
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
2014-08-18 19:16:01,463 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Found
destination
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()] for migrating to.
2014-08-18 19:16:01,463 INFO [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Migrating
VM[User|i-3-44-VM] to
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
2014-08-18 19:16:01,463 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Returning
Deployment Destination:
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
2014-08-18 19:16:01,464 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Found
destination
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()] for migrating to.
2014-08-18 19:16:01,464 INFO [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Migrating
VM[User|i-3-45-VM] to
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
2014-08-18 19:16:01,503 DEBUG [c.c.n.NetworkModelImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Service
SecurityGroup is not supported in the network id=204
2014-08-18 19:16:01,504 DEBUG [c.c.n.NetworkModelImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Service
SecurityGroup is not supported in the network id=204
2014-08-18 19:16:01,592 DEBUG [c.c.n.NetworkModelImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Service
SecurityGroup is not supported in the network id=204
2014-08-18 19:16:01,593 DEBUG [c.c.n.NetworkModelImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Service
SecurityGroup is not supported in the network id=204
2014-08-18 19:16:01,614 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Preparing 1
volumes for VM[User|i-3-45-VM]
2014-08-18 19:16:01,617 DEBUG [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Preparing 1
volumes for VM[User|i-3-44-VM]
2014-08-18 19:16:01,792 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq
1-7894810146780479507: Sending { Cmd , MgmtId: 213737702773493, via:
1(cldstk-R720-10), Ver: v1, Flags: 100111,
[{"com.cloud.agent.api.PrepareForMigrationCommand":{"vm":{"id":45,"name":"i-3-45-VM","bootloader":"PyGrub","type":"User","cpus":2,"minSpeed":1600,"maxSpeed":1600,"minRam":3221225472,"maxRam":3221225472,"arch":"x86_64","os":"Windows
8 (64-bit)","platformEmulator":"Windows 8
(64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"fJUipR1PM6PLvPSTXWLuKg==","params":{"platform":"vgpu_config:/usr/share/nvidia/vgx/grid_k240q.conf;viridian:true;acpi:1;apic:true;pae:true;vgpu_pci_id:0000:07:00.0;videoram:8;device_id:0002;nx:true;timeoffset:-1;vga:std","hypervisortoolsversion":"xenserver61","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"3aa5d887-38e0-40e4-9c11-d3f27dd80f62","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"ac0b242e-d2cc-40ca-9e1a-4e88c0976b39","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3feb450-3c8e-380f-bbf5-4e8e2424b5f9","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Abhinav/vgpu-ps1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Abhinav/vgpu-ps1/?ROLE=Primary&STOREUUID=e3feb450-3c8e-380f-bbf5-4e8e2424b5f9"}},"name":"ROOT-45","size":21474836480,"path":"8343dcb8-fa69-479a-9c6d-5d7d28270b8e","volumeId":60,"vmName":"i-3-45-VM","accountId":3,"format":"VHD","provisioningType":"THIN","id":60,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"8343dcb8-fa69-479a-9c6d-5d7d28270b8e","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"871a3594-2404-4501-bf40-b043240bc737","uuid":"47535f05-4cde-4db0-b899-fcfebd1857c1","ip":"10.1.1.84","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:0d:e0:00:15","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://700","isolationUri":"vlan://700","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group
of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID
K240Q"}},"wait":0}}] }
2014-08-18 19:16:01,800 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq
1-7894810146780479507: Executing: { Cmd , MgmtId: 213737702773493, via:
1(cldstk-R720-10), Ver: v1, Flags: 100111,
[{"com.cloud.agent.api.PrepareForMigrationCommand":{"vm":{"id":45,"name":"i-3-45-VM","bootloader":"PyGrub","type":"User","cpus":2,"minSpeed":1600,"maxSpeed":1600,"minRam":3221225472,"maxRam":3221225472,"arch":"x86_64","os":"Windows
8 (64-bit)","platformEmulator":"Windows 8
(64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"fJUipR1PM6PLvPSTXWLuKg==","params":{"platform":"vgpu_config:/usr/share/nvidia/vgx/grid_k240q.conf;viridian:true;acpi:1;apic:true;pae:true;vgpu_pci_id:0000:07:00.0;videoram:8;device_id:0002;nx:true;timeoffset:-1;vga:std","hypervisortoolsversion":"xenserver61","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"3aa5d887-38e0-40e4-9c11-d3f27dd80f62","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"ac0b242e-d2cc-40ca-9e1a-4e88c0976b39","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3feb450-3c8e-380f-bbf5-4e8e2424b5f9","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Abhinav/vgpu-ps1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Abhinav/vgpu-ps1/?ROLE=Primary&STOREUUID=e3feb450-3c8e-380f-bbf5-4e8e2424b5f9"}},"name":"ROOT-45","size":21474836480,"path":"8343dcb8-fa69-479a-9c6d-5d7d28270b8e","volumeId":60,"vmName":"i-3-45-VM","accountId":3,"format":"VHD","provisioningType":"THIN","id":60,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"8343dcb8-fa69-479a-9c6d-5d7d28270b8e","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"871a3594-2404-4501-bf40-b043240bc737","uuid":"47535f05-4cde-4db0-b899-fcfebd1857c1","ip":"10.1.1.84","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:0d:e0:00:15","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://700","isolationUri":"vlan://700","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group
of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID
K240Q"}},"wait":0}}] }
2014-08-18 19:16:01,805 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-31:ctx-f3087b9e) Seq 1-7894810146780479507: Executing request
2014-08-18 19:16:01,813 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Seq
1-7894810146780479508: Waiting for Seq 7894810146780479507 Scheduling: { Cmd ,
MgmtId: 213737702773493, via: 1(cldstk-R720-10), Ver: v1, Flags: 100111,
[{"com.cloud.agent.api.PrepareForMigrationCommand":{"vm":{"id":44,"name":"i-3-44-VM","bootloader":"PyGrub","type":"User","cpus":2,"minSpeed":1600,"maxSpeed":1600,"minRam":3221225472,"maxRam":3221225472,"arch":"x86_64","os":"Windows
8 (64-bit)","platformEmulator":"Windows 8
(64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"dZkRpXoT4v5TCEO0OP5G9A==","params":{"platform":"vgpu_config:/usr/share/nvidia/vgx/grid_k200.conf;viridian:true;acpi:1;apic:true;pae:true;vgpu_pci_id:0000:08:00.0;videoram:8;device_id:0002;nx:true;timeoffset:-1;vga:std","hypervisortoolsversion":"xenserver61","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"13d4c38e-74e7-43b2-861f-bbb4ad5677b0","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"2bffd799-c085-4aa6-b151-fd8c3a8c36be","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3feb450-3c8e-380f-bbf5-4e8e2424b5f9","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Abhinav/vgpu-ps1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Abhinav/vgpu-ps1/?ROLE=Primary&STOREUUID=e3feb450-3c8e-380f-bbf5-4e8e2424b5f9"}},"name":"ROOT-44","size":21474836480,"path":"fde95825-32fb-43ec-a0f9-2eed21dc9623","volumeId":59,"vmName":"i-3-44-VM","accountId":3,"format":"VHD","provisioningType":"THIN","id":59,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"fde95825-32fb-43ec-a0f9-2eed21dc9623","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"6f8b2a4a-45e1-49e5-9772-5ca7c93c5e71","uuid":"47535f05-4cde-4db0-b899-fcfebd1857c1","ip":"10.1.1.177","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:2d:50:00:14","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://700","isolationUri":"vlan://700","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group
of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID
K200"}},"wait":0}}] }
2014-08-18 19:16:01,815 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-31:ctx-f3087b9e) Preparing host for migrating
com.cloud.agent.api.to.VirtualMachineTO@7d4d9b13
2014-08-18 19:16:01,890 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-31:ctx-f3087b9e) Found a network called
VLAN-7d3eae71-f914-1cf0-fd49-0ba9560014a2-700 on host=10.102.192.10;
Network=5ffbcf9d-e864-fc86-551c-1c46cc8eb7f0;
pif=f66d3a35-c6d1-c1cb-5082-ca798220ed10
2014-08-18 19:16:01,890 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-31:ctx-f3087b9e) 4. The VM i-3-45-VM is in Migrating state
2014-08-18 19:16:01,891 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-31:ctx-f3087b9e) Seq 1-7894810146780479507: Response Received:
2014-08-18 19:16:01,891 DEBUG [c.c.a.t.Request] (DirectAgent-31:ctx-f3087b9e)
Seq 1-7894810146780479507: Processing: { Ans: , MgmtId: 213737702773493, via:
1, Ver: v1, Flags: 110,
[{"com.cloud.agent.api.PrepareForMigrationAnswer":{"result":true,"wait":0}}] }
2014-08-18 19:16:01,894 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq
1-7894810146780479507: Received: { Ans: , MgmtId: 213737702773493, via: 1,
Ver: v1, Flags: 110, { PrepareForMigrationAnswer } }
2014-08-18 19:16:01,899 DEBUG [c.c.a.m.AgentAttache]
(DirectAgent-31:ctx-f3087b9e) Seq 1-7894810146780479508: Sending now. is
current sequence.
2014-08-18 19:16:01,910 DEBUG [c.c.a.t.Request] (DirectAgent-31:ctx-f3087b9e)
Seq 1-7894810146780479508: Executing: { Cmd , MgmtId: 213737702773493, via:
1(cldstk-R720-10), Ver: v1, Flags: 100111,
[{"com.cloud.agent.api.PrepareForMigrationCommand":{"vm":{"id":44,"name":"i-3-44-VM","bootloader":"PyGrub","type":"User","cpus":2,"minSpeed":1600,"maxSpeed":1600,"minRam":3221225472,"maxRam":3221225472,"arch":"x86_64","os":"Windows
8 (64-bit)","platformEmulator":"Windows 8
(64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"dZkRpXoT4v5TCEO0OP5G9A==","params":{"platform":"vgpu_config:/usr/share/nvidia/vgx/grid_k200.conf;viridian:true;acpi:1;apic:true;pae:true;vgpu_pci_id:0000:08:00.0;videoram:8;device_id:0002;nx:true;timeoffset:-1;vga:std","hypervisortoolsversion":"xenserver61","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"13d4c38e-74e7-43b2-861f-bbb4ad5677b0","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"2bffd799-c085-4aa6-b151-fd8c3a8c36be","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3feb450-3c8e-380f-bbf5-4e8e2424b5f9","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Abhinav/vgpu-ps1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Abhinav/vgpu-ps1/?ROLE=Primary&STOREUUID=e3feb450-3c8e-380f-bbf5-4e8e2424b5f9"}},"name":"ROOT-44","size":21474836480,"path":"fde95825-32fb-43ec-a0f9-2eed21dc9623","volumeId":59,"vmName":"i-3-44-VM","accountId":3,"format":"VHD","provisioningType":"THIN","id":59,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"fde95825-32fb-43ec-a0f9-2eed21dc9623","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"6f8b2a4a-45e1-49e5-9772-5ca7c93c5e71","uuid":"47535f05-4cde-4db0-b899-fcfebd1857c1","ip":"10.1.1.177","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:2d:50:00:14","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://700","isolationUri":"vlan://700","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group
of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID
K200"}},"wait":0}}] }
2014-08-18 19:16:01,910 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-10:ctx-afc70703) Seq 1-7894810146780479508: Executing request
2014-08-18 19:16:01,916 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-10:ctx-afc70703) Preparing host for migrating
com.cloud.agent.api.to.VirtualMachineTO@f5798a8
2014-08-18 19:16:01,951 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) VM state
transitted from :Running to Migrating with event: MigrationRequestedvm's
original host id: 8 new host id: 1 host id before state transition: 8
2014-08-18 19:16:01,983 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Hosts's actual
total CPU: 55200 and CPU after applying overprovisioning: 55200
2014-08-18 19:16:01,983 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) We are
allocating VM, increasing the used capacity of this host:1
2014-08-18 19:16:01,983 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Current Used
CPU: 2000 , Free CPU:53200 ,Requested CPU: 3200
2014-08-18 19:16:01,983 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Current Used
RAM: 1879048192 , Free RAM:13840119808 ,Requested RAM: 3221225472
2014-08-18 19:16:01,986 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) CPU STATS after
allocation: for host: 1, old used: 2000, old reserved: 0, actual total: 55200,
total with overprovisioning: 55200; new used:5200, reserved:0; requested
cpu:3200,alloc_from_last:false
2014-08-18 19:16:01,986 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) RAM STATS after
allocation: for host: 1, old used: 1879048192, old reserved: 0, total:
15719168000; new used: 5100273664, reserved: 0; requested mem:
3221225472,alloc_from_last:false
2014-08-18 19:16:02,014 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-10:ctx-afc70703) Found a network called
VLAN-7d3eae71-f914-1cf0-fd49-0ba9560014a2-700 on host=10.102.192.10;
Network=5ffbcf9d-e864-fc86-551c-1c46cc8eb7f0;
pif=f66d3a35-c6d1-c1cb-5082-ca798220ed10
2014-08-18 19:16:02,016 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-10:ctx-afc70703) 4. The VM i-3-44-VM is in Migrating state
2014-08-18 19:16:02,017 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-10:ctx-afc70703) Seq 1-7894810146780479508: Response Received:
2014-08-18 19:16:02,017 DEBUG [c.c.a.t.Request] (DirectAgent-10:ctx-afc70703)
Seq 1-7894810146780479508: Processing: { Ans: , MgmtId: 213737702773493, via:
1, Ver: v1, Flags: 110,
[{"com.cloud.agent.api.PrepareForMigrationAnswer":{"result":true,"wait":0}}] }
2014-08-18 19:16:02,017 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Seq
1-7894810146780479508: Received: { Ans: , MgmtId: 213737702773493, via: 1,
Ver: v1, Flags: 110, { PrepareForMigrationAnswer } }
2014-08-18 19:16:02,019 DEBUG [c.c.a.m.AgentAttache]
(DirectAgent-10:ctx-afc70703) Seq 1-7894810146780479508: No more commands found
2014-08-18 19:16:02,028 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq
8-5623307084725485587: Sending { Cmd , MgmtId: 213737702773493, via:
8(cldstk-R720-66), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.MigrateCommand":{"vmName":"i-3-45-VM","destIp":"10.102.192.10","hostGuid":"964fb4ce-dcaa-4eac-a957-29289cb6d784","isWindows":true,"vmTO":{"id":45,"name":"i-3-45-VM","bootloader":"PyGrub","type":"User","cpus":2,"minSpeed":1600,"maxSpeed":1600,"minRam":3221225472,"maxRam":3221225472,"arch":"x86_64","os":"Windows
8 (64-bit)","platformEmulator":"Windows 8
(64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"fJUipR1PM6PLvPSTXWLuKg==","params":{"platform":"vgpu_config:/usr/share/nvidia/vgx/grid_k240q.conf;viridian:true;acpi:1;apic:true;pae:true;vgpu_pci_id:0000:07:00.0;videoram:8;device_id:0002;nx:true;timeoffset:-1;vga:std","hypervisortoolsversion":"xenserver61","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"3aa5d887-38e0-40e4-9c11-d3f27dd80f62","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"ac0b242e-d2cc-40ca-9e1a-4e88c0976b39","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3feb450-3c8e-380f-bbf5-4e8e2424b5f9","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Abhinav/vgpu-ps1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Abhinav/vgpu-ps1/?ROLE=Primary&STOREUUID=e3feb450-3c8e-380f-bbf5-4e8e2424b5f9"}},"name":"ROOT-45","size":21474836480,"path":"8343dcb8-fa69-479a-9c6d-5d7d28270b8e","volumeId":60,"vmName":"i-3-45-VM","accountId":3,"format":"VHD","provisioningType":"THIN","id":60,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"8343dcb8-fa69-479a-9c6d-5d7d28270b8e","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"871a3594-2404-4501-bf40-b043240bc737","uuid":"47535f05-4cde-4db0-b899-fcfebd1857c1","ip":"10.1.1.84","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:0d:e0:00:15","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://700","isolationUri":"vlan://700","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group
of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID
K240Q"}},"executeInSequence":false,"wait":0}}] }
2014-08-18 19:16:02,030 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq
8-5623307084725485587: Executing: { Cmd , MgmtId: 213737702773493, via:
8(cldstk-R720-66), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.MigrateCommand":{"vmName":"i-3-45-VM","destIp":"10.102.192.10","hostGuid":"964fb4ce-dcaa-4eac-a957-29289cb6d784","isWindows":true,"vmTO":{"id":45,"name":"i-3-45-VM","bootloader":"PyGrub","type":"User","cpus":2,"minSpeed":1600,"maxSpeed":1600,"minRam":3221225472,"maxRam":3221225472,"arch":"x86_64","os":"Windows
8 (64-bit)","platformEmulator":"Windows 8
(64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"fJUipR1PM6PLvPSTXWLuKg==","params":{"platform":"vgpu_config:/usr/share/nvidia/vgx/grid_k240q.conf;viridian:true;acpi:1;apic:true;pae:true;vgpu_pci_id:0000:07:00.0;videoram:8;device_id:0002;nx:true;timeoffset:-1;vga:std","hypervisortoolsversion":"xenserver61","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"3aa5d887-38e0-40e4-9c11-d3f27dd80f62","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"ac0b242e-d2cc-40ca-9e1a-4e88c0976b39","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3feb450-3c8e-380f-bbf5-4e8e2424b5f9","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Abhinav/vgpu-ps1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Abhinav/vgpu-ps1/?ROLE=Primary&STOREUUID=e3feb450-3c8e-380f-bbf5-4e8e2424b5f9"}},"name":"ROOT-45","size":21474836480,"path":"8343dcb8-fa69-479a-9c6d-5d7d28270b8e","volumeId":60,"vmName":"i-3-45-VM","accountId":3,"format":"VHD","provisioningType":"THIN","id":60,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"8343dcb8-fa69-479a-9c6d-5d7d28270b8e","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"871a3594-2404-4501-bf40-b043240bc737","uuid":"47535f05-4cde-4db0-b899-fcfebd1857c1","ip":"10.1.1.84","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:0d:e0:00:15","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://700","isolationUri":"vlan://700","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group
of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID
K240Q"}},"executeInSequence":false,"wait":0}}] }
2014-08-18 19:16:02,036 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-32:ctx-02f4f02c) Seq 8-5623307084725485587: Executing request
2014-08-18 19:16:02,040 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) VM state
transitted from :Running to Migrating with event: MigrationRequestedvm's
original host id: 8 new host id: 1 host id before state transition: 8
2014-08-18 19:16:02,057 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Hosts's actual
total CPU: 55200 and CPU after applying overprovisioning: 55200
2014-08-18 19:16:02,057 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) We are
allocating VM, increasing the used capacity of this host:1
2014-08-18 19:16:02,057 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Current Used
CPU: 5200 , Free CPU:50000 ,Requested CPU: 3200
2014-08-18 19:16:02,057 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Current Used
RAM: 5100273664 , Free RAM:10618894336 ,Requested RAM: 3221225472
2014-08-18 19:16:02,057 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) CPU STATS after
allocation: for host: 1, old used: 5200, old reserved: 0, actual total: 55200,
total with overprovisioning: 55200; new used:8400, reserved:0; requested
cpu:3200,alloc_from_last:false
2014-08-18 19:16:02,057 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) RAM STATS after
allocation: for host: 1, old used: 5100273664, old reserved: 0, total:
15719168000; new used: 8321499136, reserved: 0; requested mem:
3221225472,alloc_from_last:false
2014-08-18 19:16:02,081 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Seq
8-5623307084725485588: Sending { Cmd , MgmtId: 213737702773493, via:
8(cldstk-R720-66), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.MigrateCommand":{"vmName":"i-3-44-VM","destIp":"10.102.192.10","hostGuid":"964fb4ce-dcaa-4eac-a957-29289cb6d784","isWindows":true,"vmTO":{"id":44,"name":"i-3-44-VM","bootloader":"PyGrub","type":"User","cpus":2,"minSpeed":1600,"maxSpeed":1600,"minRam":3221225472,"maxRam":3221225472,"arch":"x86_64","os":"Windows
8 (64-bit)","platformEmulator":"Windows 8
(64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"dZkRpXoT4v5TCEO0OP5G9A==","params":{"platform":"vgpu_config:/usr/share/nvidia/vgx/grid_k200.conf;viridian:true;acpi:1;apic:true;pae:true;vgpu_pci_id:0000:08:00.0;videoram:8;device_id:0002;nx:true;timeoffset:-1;vga:std","hypervisortoolsversion":"xenserver61","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"13d4c38e-74e7-43b2-861f-bbb4ad5677b0","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"2bffd799-c085-4aa6-b151-fd8c3a8c36be","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3feb450-3c8e-380f-bbf5-4e8e2424b5f9","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Abhinav/vgpu-ps1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Abhinav/vgpu-ps1/?ROLE=Primary&STOREUUID=e3feb450-3c8e-380f-bbf5-4e8e2424b5f9"}},"name":"ROOT-44","size":21474836480,"path":"fde95825-32fb-43ec-a0f9-2eed21dc9623","volumeId":59,"vmName":"i-3-44-VM","accountId":3,"format":"VHD","provisioningType":"THIN","id":59,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"fde95825-32fb-43ec-a0f9-2eed21dc9623","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"6f8b2a4a-45e1-49e5-9772-5ca7c93c5e71","uuid":"47535f05-4cde-4db0-b899-fcfebd1857c1","ip":"10.1.1.177","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:2d:50:00:14","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://700","isolationUri":"vlan://700","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group
of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID
K200"}},"executeInSequence":false,"wait":0}}] }
2014-08-18 19:16:02,083 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Seq
8-5623307084725485588: Executing: { Cmd , MgmtId: 213737702773493, via:
8(cldstk-R720-66), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.MigrateCommand":{"vmName":"i-3-44-VM","destIp":"10.102.192.10","hostGuid":"964fb4ce-dcaa-4eac-a957-29289cb6d784","isWindows":true,"vmTO":{"id":44,"name":"i-3-44-VM","bootloader":"PyGrub","type":"User","cpus":2,"minSpeed":1600,"maxSpeed":1600,"minRam":3221225472,"maxRam":3221225472,"arch":"x86_64","os":"Windows
8 (64-bit)","platformEmulator":"Windows 8
(64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"dZkRpXoT4v5TCEO0OP5G9A==","params":{"platform":"vgpu_config:/usr/share/nvidia/vgx/grid_k200.conf;viridian:true;acpi:1;apic:true;pae:true;vgpu_pci_id:0000:08:00.0;videoram:8;device_id:0002;nx:true;timeoffset:-1;vga:std","hypervisortoolsversion":"xenserver61","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"13d4c38e-74e7-43b2-861f-bbb4ad5677b0","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"2bffd799-c085-4aa6-b151-fd8c3a8c36be","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e3feb450-3c8e-380f-bbf5-4e8e2424b5f9","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.101","path":"/Abhinav/vgpu-ps1","port":2049,"url":"NetworkFilesystem://10.102.192.101/Abhinav/vgpu-ps1/?ROLE=Primary&STOREUUID=e3feb450-3c8e-380f-bbf5-4e8e2424b5f9"}},"name":"ROOT-44","size":21474836480,"path":"fde95825-32fb-43ec-a0f9-2eed21dc9623","volumeId":59,"vmName":"i-3-44-VM","accountId":3,"format":"VHD","provisioningType":"THIN","id":59,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"fde95825-32fb-43ec-a0f9-2eed21dc9623","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.102.192.101","volumeSize":"21474836480"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"6f8b2a4a-45e1-49e5-9772-5ca7c93c5e71","uuid":"47535f05-4cde-4db0-b899-fcfebd1857c1","ip":"10.1.1.177","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:2d:50:00:14","dns1":"10.140.50.5","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://700","isolationUri":"vlan://700","isSecurityGroupEnabled":false}],"gpuDevice":{"gpuGroup":"Group
of NVIDIA Corporation GK104GL [GRID K2] GPUs","vgpuType":"GRID
K200"}},"executeInSequence":false,"wait":0}}] }
2014-08-18 19:16:02,086 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-11:ctx-84f8fe28) Seq 8-5623307084725485588: Executing request
2014-08-18 19:16:03,089 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-32:ctx-02f4f02c) Task failed! Task record: uuid:
e4b5c630-e6e1-1abe-88de-0eec313abaf8
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Tue Aug 19 00:28:48 IST 2014
finished: Tue Aug 19 00:28:48 IST 2014
status: failure
residentOn: com.xensource.xenapi.Host@534d97dd
progress: 1.0
type: <none/>
result:
errorInfo: [VM_HAS_VGPU,
OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
2014-08-18 19:16:03,097 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-32:ctx-02f4f02c) Unable to migrate VM(i-3-45-VM) from
host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed! Task record:
uuid: e4b5c630-e6e1-1abe-88de-0eec313abaf8
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Tue Aug 19 00:28:48 IST 2014
finished: Tue Aug 19 00:28:48 IST 2014
status: failure
residentOn: com.xensource.xenapi.Host@534d97dd
progress: 1.0
type: <none/>
result:
errorInfo: [VM_HAS_VGPU,
OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
Task failed! Task record: uuid:
e4b5c630-e6e1-1abe-88de-0eec313abaf8
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Tue Aug 19 00:28:48 IST 2014
finished: Tue Aug 19 00:28:48 IST 2014
status: failure
residentOn: com.xensource.xenapi.Host@534d97dd
progress: 1.0
type: <none/>
result:
errorInfo: [VM_HAS_VGPU,
OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:3166)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.migrateVM(CitrixResourceBase.java:3319)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2916)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:455)
at
com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
at
com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:101)
at
com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:294)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2014-08-18 19:16:03,104 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-32:ctx-02f4f02c) unable to destroy
task(com.xensource.xenapi.Task@ec446b4c) on
host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to You gave an invalid object
reference. The object may have recently been deleted. The class parameter
gives the type of reference given, and the handle parameter echoes the bad
value given.
2014-08-18 19:16:03,105 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-32:ctx-02f4f02c) Catch Exception
com.cloud.utils.exception.CloudRuntimeException: Migration failed due to
com.cloud.utils.exception.CloudRuntimeException: Unable to migrate
VM(i-3-45-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task
failed! Task record: uuid: e4b5c630-e6e1-1abe-88de-0eec313abaf8
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Tue Aug 19 00:28:48 IST 2014
finished: Tue Aug 19 00:28:48 IST 2014
status: failure
residentOn: com.xensource.xenapi.Host@534d97dd
progress: 1.0
type: <none/>
result:
errorInfo: [VM_HAS_VGPU,
OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
com.cloud.utils.exception.CloudRuntimeException: Unable to migrate
VM(i-3-45-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task
failed! Task record: uuid: e4b5c630-e6e1-1abe-88de-0eec313abaf8
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Tue Aug 19 00:28:48 IST 2014
finished: Tue Aug 19 00:28:48 IST 2014
status: failure
residentOn: com.xensource.xenapi.Host@534d97dd
progress: 1.0
type: <none/>
result:
errorInfo: [VM_HAS_VGPU,
OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.migrateVM(CitrixResourceBase.java:3330)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2916)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:455)
at
com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
at
com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:101)
at
com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:294)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2014-08-18 19:16:03,106 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-32:ctx-02f4f02c) Seq 8-5623307084725485587: Response Received:
2014-08-18 19:16:03,107 DEBUG [c.c.a.t.Request] (DirectAgent-32:ctx-02f4f02c)
Seq 8-5623307084725485587: Processing: { Ans: , MgmtId: 213737702773493, via:
8, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.MigrateAnswer":{"result":false,"details":"Catch
Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed due
to com.cloud.utils.exception.CloudRuntimeException: Unable to migrate
VM(i-3-45-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task
failed! Task record: uuid:
e4b5c630-e6e1-1abe-88de-0eec313abaf8\n nameLabel:
Async.VM.pool_migrate\n nameDescription: \n allowedOperations: []\n
currentOperations: {}\n created: Tue Aug 19 00:28:48 IST 2014\n
finished: Tue Aug 19 00:28:48 IST 2014\n status: failure\n
residentOn: com.xensource.xenapi.Host@534d97dd\n progress:
1.0\n type: <none/>\n result: \n
errorInfo: [VM_HAS_VGPU, OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]\n
otherConfig: {}\n subtaskOf: com.xensource.xenapi.Task@aaf13f6f\n
subtasks: []\n","wait":0}}] }
2014-08-18 19:16:03,107 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq
8-5623307084725485587: Received: { Ans: , MgmtId: 213737702773493, via: 8,
Ver: v1, Flags: 10, { MigrateAnswer } }
2014-08-18 19:16:03,107 INFO [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Migration was
unsuccessful. Cleaning up: VM[User|i-3-45-VM]
2014-08-18 19:16:03,115 WARN [o.a.c.alerts] (Work-Job-Executor-5:ctx-6ba6232b
job-692/job-694 ctx-a880303d) alertType:: 17 // dataCenterId:: 1 // podId:: 1
// clusterId:: null // message:: Unable to migrate vm i-3-45-VM from host
cldstk-R720-66 in zone vgpu and pod vgpu
2014-08-18 19:16:03,119 DEBUG [c.c.a.AlertManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Have already
sent: 1 emails for alert type '17' -- skipping send email
2014-08-18 19:16:03,121 DEBUG [c.c.a.m.AgentAttache]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq
1-7894810146780479509: Routed from 213737702773493
2014-08-18 19:16:03,121 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq
1-7894810146780479509: Sending { Cmd , MgmtId: 213737702773493, via:
1(cldstk-R720-10), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-3-45-VM","wait":0}}]
}
2014-08-18 19:16:03,122 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Seq
1-7894810146780479509: Executing: { Cmd , MgmtId: 213737702773493, via:
1(cldstk-R720-10), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-3-45-VM","wait":0}}]
}
2014-08-18 19:16:03,122 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-17:ctx-d4a3374c) Seq 1-7894810146780479509: Executing request
2014-08-18 19:16:03,143 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) VM state
transitted from :Migrating to Running with event: OperationFailedvm's original
host id: 8 new host id: 8 host id before state transition: 1
2014-08-18 19:16:03,146 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-11:ctx-84f8fe28) Task failed! Task record: uuid:
f40dee3d-b4fc-d49d-cb2e-4fbdad143171
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Tue Aug 19 00:28:48 IST 2014
finished: Tue Aug 19 00:28:48 IST 2014
status: failure
residentOn: com.xensource.xenapi.Host@534d97dd
progress: 1.0
type: <none/>
result:
errorInfo: [VM_HAS_VGPU,
OpaqueRef:87823176-562f-9b52-680c-5a7ef70d8241]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
2014-08-18 19:16:03,149 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-17:ctx-d4a3374c) Seq 1-7894810146780479509: Response Received:
2014-08-18 19:16:03,152 DEBUG [c.c.a.t.Request] (DirectAgent-17:ctx-d4a3374c)
Seq 1-7894810146780479509: Processing: { Ans: , MgmtId: 213737702773493, via:
1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM does not
exist","wait":0}}] }
2014-08-18 19:16:03,152 DEBUG [c.c.a.m.AgentAttache]
(DirectAgent-17:ctx-d4a3374c) Seq 1-7894810146780479509: Unable to find
listener.
2014-08-18 19:16:03,159 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Hosts's actual
total CPU: 55200 and CPU after applying overprovisioning: 55200
2014-08-18 19:16:03,159 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Hosts's actual
total RAM: 15719168512 and RAM after applying overprovisioning: 15719168000
2014-08-18 19:16:03,160 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-11:ctx-84f8fe28) Unable to migrate VM(i-3-44-VM) from
host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task failed! Task record:
uuid: f40dee3d-b4fc-d49d-cb2e-4fbdad143171
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Tue Aug 19 00:28:48 IST 2014
finished: Tue Aug 19 00:28:48 IST 2014
status: failure
residentOn: com.xensource.xenapi.Host@534d97dd
progress: 1.0
type: <none/>
result:
errorInfo: [VM_HAS_VGPU,
OpaqueRef:87823176-562f-9b52-680c-5a7ef70d8241]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
Task failed! Task record: uuid:
f40dee3d-b4fc-d49d-cb2e-4fbdad143171
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Tue Aug 19 00:28:48 IST 2014
finished: Tue Aug 19 00:28:48 IST 2014
status: failure
residentOn: com.xensource.xenapi.Host@534d97dd
progress: 1.0
type: <none/>
result:
errorInfo: [VM_HAS_VGPU,
OpaqueRef:87823176-562f-9b52-680c-5a7ef70d8241]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:3166)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.migrateVM(CitrixResourceBase.java:3319)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2916)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:455)
at
com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
at
com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:101)
at
com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:294)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2014-08-18 19:16:03,161 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) release cpu
from host: 1, old used: 8400,reserved: 0, actual total: 55200, total with
overprovisioning: 55200; new used: 5200,reserved:0; movedfromreserved:
false,moveToReserveredfalse
2014-08-18 19:16:03,162 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) release mem
from host: 1, old used: 8321499136,reserved: 0, total: 15719168000; new used:
5100273664,reserved:0; movedfromreserved: false,moveToReserveredfalse
2014-08-18 19:16:03,168 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-11:ctx-84f8fe28) unable to destroy
task(com.xensource.xenapi.Task@1e06f89f) on
host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to You gave an invalid object
reference. The object may have recently been deleted. The class parameter
gives the type of reference given, and the handle parameter echoes the bad
value given.
2014-08-18 19:16:03,171 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-11:ctx-84f8fe28) Catch Exception
com.cloud.utils.exception.CloudRuntimeException: Migration failed due to
com.cloud.utils.exception.CloudRuntimeException: Unable to migrate
VM(i-3-44-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task
failed! Task record: uuid: f40dee3d-b4fc-d49d-cb2e-4fbdad143171
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Tue Aug 19 00:28:48 IST 2014
finished: Tue Aug 19 00:28:48 IST 2014
status: failure
residentOn: com.xensource.xenapi.Host@534d97dd
progress: 1.0
type: <none/>
result:
errorInfo: [VM_HAS_VGPU,
OpaqueRef:87823176-562f-9b52-680c-5a7ef70d8241]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
com.cloud.utils.exception.CloudRuntimeException: Unable to migrate
VM(i-3-44-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task
failed! Task record: uuid: f40dee3d-b4fc-d49d-cb2e-4fbdad143171
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Tue Aug 19 00:28:48 IST 2014
finished: Tue Aug 19 00:28:48 IST 2014
status: failure
residentOn: com.xensource.xenapi.Host@534d97dd
progress: 1.0
type: <none/>
result:
errorInfo: [VM_HAS_VGPU,
OpaqueRef:87823176-562f-9b52-680c-5a7ef70d8241]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.migrateVM(CitrixResourceBase.java:3330)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2916)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:455)
at
com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
at
com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:101)
at
com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:294)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2014-08-18 19:16:03,173 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-11:ctx-84f8fe28) Seq 8-5623307084725485588: Response Received:
2014-08-18 19:16:03,174 DEBUG [c.c.a.t.Request] (DirectAgent-11:ctx-84f8fe28)
Seq 8-5623307084725485588: Processing: { Ans: , MgmtId: 213737702773493, via:
8, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.MigrateAnswer":{"result":false,"details":"Catch
Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed due
to com.cloud.utils.exception.CloudRuntimeException: Unable to migrate
VM(i-3-44-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task
failed! Task record: uuid:
f40dee3d-b4fc-d49d-cb2e-4fbdad143171\n nameLabel:
Async.VM.pool_migrate\n nameDescription: \n allowedOperations: []\n
currentOperations: {}\n created: Tue Aug 19 00:28:48 IST 2014\n
finished: Tue Aug 19 00:28:48 IST 2014\n status: failure\n
residentOn: com.xensource.xenapi.Host@534d97dd\n progress:
1.0\n type: <none/>\n result: \n
errorInfo: [VM_HAS_VGPU, OpaqueRef:87823176-562f-9b52-680c-5a7ef70d8241]\n
otherConfig: {}\n subtaskOf: com.xensource.xenapi.Task@aaf13f6f\n
subtasks: []\n","wait":0}}] }
2014-08-18 19:16:03,175 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Seq
8-5623307084725485588: Received: { Ans: , MgmtId: 213737702773493, via: 8,
Ver: v1, Flags: 10, { MigrateAnswer } }
2014-08-18 19:16:03,175 INFO [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Migration was
unsuccessful. Cleaning up: VM[User|i-3-44-VM]
2014-08-18 19:16:03,179 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Invocation
exception, caused by: com.cloud.utils.exception.CloudRuntimeException: Unable
to migrate due to Catch Exception
com.cloud.utils.exception.CloudRuntimeException: Migration failed due to
com.cloud.utils.exception.CloudRuntimeException: Unable to migrate
VM(i-3-45-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task
failed! Task record: uuid: e4b5c630-e6e1-1abe-88de-0eec313abaf8
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Tue Aug 19 00:28:48 IST 2014
finished: Tue Aug 19 00:28:48 IST 2014
status: failure
residentOn: com.xensource.xenapi.Host@534d97dd
progress: 1.0
type: <none/>
result:
errorInfo: [VM_HAS_VGPU,
OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
2014-08-18 19:16:03,180 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694 ctx-a880303d) Rethrow
exception com.cloud.utils.exception.CloudRuntimeException: Unable to migrate
due to Catch Exception com.cloud.utils.exception.CloudRuntimeException:
Migration failed due to com.cloud.utils.exception.CloudRuntimeException: Unable
to migrate VM(i-3-45-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to
Task failed! Task record: uuid:
e4b5c630-e6e1-1abe-88de-0eec313abaf8
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Tue Aug 19 00:28:48 IST 2014
finished: Tue Aug 19 00:28:48 IST 2014
status: failure
residentOn: com.xensource.xenapi.Host@534d97dd
progress: 1.0
type: <none/>
result:
errorInfo: [VM_HAS_VGPU,
OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
2014-08-18 19:16:03,180 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694) Done with run of VM work
job: com.cloud.vm.VmWorkMigrateAway for VM 45, job origin: 692
2014-08-18 19:16:03,181 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694) Unable to complete
AsyncJobVO {id:694, userId: 1, accountId: 1, instanceType: null, instanceId:
null, cmd: com.cloud.vm.VmWorkMigrateAway, cmdInfo:
rO0ABXNyAB5jb20uY2xvdWQudm0uVm1Xb3JrTWlncmF0ZUF3YXmt4MX4jtcEmwIAAUoACXNyY0hvc3RJZHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAALXQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAACA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 213737702773493, completeMsid: null, lastUpdated: null,
lastPolled: null, created: Mon Aug 18 19:16:00 IST 2014}, job origin:692
com.cloud.utils.exception.CloudRuntimeException: Unable to migrate due to Catch
Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed due
to com.cloud.utils.exception.CloudRuntimeException: Unable to migrate
VM(i-3-45-VM) from host(76520b97-5ce8-4c71-9572-b0834b62dc35) due to Task
failed! Task record: uuid: e4b5c630-e6e1-1abe-88de-0eec313abaf8
nameLabel: Async.VM.pool_migrate
nameDescription:
allowedOperations: []
currentOperations: {}
created: Tue Aug 19 00:28:48 IST 2014
finished: Tue Aug 19 00:28:48 IST 2014
status: failure
residentOn: com.xensource.xenapi.Host@534d97dd
progress: 1.0
type: <none/>
result:
errorInfo: [VM_HAS_VGPU,
OpaqueRef:714675bb-47c1-d791-d537-bed377ca3426]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.vm.VirtualMachineManagerImpl.migrate(VirtualMachineManagerImpl.java:1899)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateAway(VirtualMachineManagerImpl.java:2332)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateMigrateAway(VirtualMachineManagerImpl.java:4619)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4738)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2014-08-18 19:16:03,191 WARN [o.a.c.alerts] (Work-Job-Executor-4:ctx-55733e40
job-691/job-693 ctx-d9d7d51a) alertType:: 17 // dataCenterId:: 1 // podId:: 1
// clusterId:: null // message:: Unable to migrate vm i-3-44-VM from host
cldstk-R720-66 in zone vgpu and pod vgpu
2014-08-18 19:16:03,198 DEBUG [c.c.a.AlertManagerImpl]
(Work-Job-Executor-4:ctx-55733e40 job-691/job-693 ctx-d9d7d51a) Have already
sent: 1 emails for alert type '17' -- skipping send email
2014-08-18 19:16:03,200 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-5:ctx-6ba6232b job-692/job-694) Complete async job-694,
jobStatus: FAILED, resultCode: 0, result:
rO0ABXNyABpqYXZhLmxhbmcuUnVudGltZUV4Y2VwdGlvbp5fBkcKNIPlAgAAeHIAE2phdmEubGFuZy5FeGNlcHRpb27Q_R8-GjscxAIAAHhyABNqYXZhLmxhbmcuVGhyb3dhYmxl1cY1Jzl3uMsDAARMAAVjYXVzZXQAFUxqYXZhL2xhbmcvVGhyb3dhYmxlO0wADWRldGFpbE1lc3NhZ2V0ABJMamF2YS9sYW5nL1N0cmluZztbAApzdGFja1RyYWNldAAeW0xqYXZhL2xhbmcvU3RhY2tUcmFjZUVsZW1lbnQ7TAAUc3VwcHJlc3NlZEV4Y2VwdGlvbnN0ABBMamF2YS91dGlsL0xpc3Q7eHBxAH4AB3QDq0pvYiBmYWlsZWQgZHVlIHRvIGV4Y2VwdGlvbiBVbmFibGUgdG8gbWlncmF0ZSBkdWUgdG8gQ2F0Y2ggRXhjZXB0aW9uI
--
This message was sent by Atlassian JIRA
(v6.2#6252)
