Hi everybody, I use CloudStack 4.5.2 with ESXi 5.5 and Nexus 1000v - I created "Cluster01" in vCenter. I added this cluster to "Nexus01" with "uplink01" port-profile. I connected this cluster to CloudStack without problem. - I created "Cluster02" in vCenter. I added this cluster to "Nexus01" with "uplink01" port-profile. But I couldn't connect this cluster to CloudStack because CloudStack doesn't let to use one VSM for multiple cluster. - So I created a new VSM named "Nexus02" and added hosts in "Cluster02" with "uplink02" port-profile to it. - In CloudStack I created a new cluster with "Override Public-Traffic" and "Override Guest-Traffic" selected and I entered "uplink02" as "Public/Guest Traffic vSwitch Name". I entered new VSM credential and created cluster successfully. - But when I want to create a new VM in this cluster, I get error. - It seems that CloudStack doesn't pay attention to traffic overriding. Is it normal behaviour or it is a bug?
2015-09-01 14:29:44,810 DEBUG [c.c.h.v.r.VmwareResource] (DirectAgent-124:ctx-f38a0ab9 172.16.78.7, job-190/job-191, cmd: StartCommand) VM i-2-14-VM will be started with NIC device type: Vmxnet3 2015-09-01 14:29:44,811 INFO [c.c.h.v.r.VmwareResource] (DirectAgent-124:ctx-f38a0ab9 172.16.78.7, job-190/job-191, cmd: StartCommand) Prepare NIC device based on NicTO: {"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"da6fac33-2b9c-4d6f-b9bb-fdb2610de7e0","uuid":"ab58bf0f-44b4-4a1c-a1ae-4a3a975c6ba1","ip":"10.1.1.239","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:2d:9c:00:01","dns1":"8.8.8.8","dns2":"4.2.2.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://1205","isolationUri":"vlan://1205","isSecurityGroupEnabled":false,"name":"uplink01,1206,nexusdvs"} 2015-09-01 14:29:44,818 INFO [c.c.h.v.r.VmwareResource] (DirectAgent-124:ctx-f38a0ab9 172.16.78.7, job-190/job-191, cmd: StartCommand) Prepare network on nexusdvs uplink01 with name prefix: cloud.guest 2015-09-01 14:29:44,833 INFO [c.c.h.v.m.HypervisorHostHelper] (DirectAgent-124:ctx-f38a0ab9 172.16.78.7, job-190/job-191, cmd: StartCommand) Found Ethernet port profile uplink01 2015-09-01 14:29:44,833 INFO [c.c.h.v.m.HypervisorHostHelper] (DirectAgent-124:ctx-f38a0ab9 172.16.78.7, job-190/job-191, cmd: StartCommand) Stocking credentials of Nexus VSM 2015-09-01 14:29:44,840 INFO [c.c.h.v.m.HypervisorHostHelper] (DirectAgent-124:ctx-f38a0ab9 172.16.78.7, job-190/job-191, cmd: StartCommand) Port profile cloud.guest.1205.200.1-uplink01 found. 2015-09-01 14:29:45,453 WARN [c.c.h.v.r.VmwareResource] (DirectAgent-124:ctx-f38a0ab9 172.16.78.7, job-190/job-191, cmd: StartCommand) StartCommand failed due to Exception: com.cloud.utils.exception.CloudRuntimeException Message: Severity: error, Error code: InvalidValue, Error type: application, Error Message: Syntax error while parsing 'show port-profile name cloud.guest.1205.200.1-uplink01 ' , Error info: show com.cloud.utils.exception.CloudRuntimeException: Severity: error, Error code: InvalidValue, Error type: application, Error Message: Syntax error while parsing 'show port-profile name cloud.guest.1205.200.1-uplink01 ' , Error info: show at com.cloud.utils.cisco.n1kv.vsm.NetconfHelper.getPortProfileByName(NetconfHelper.java:185) at com.cloud.hypervisor.vmware.mo.HypervisorHostHelper.updatePortProfile(HypervisorHostHelper.java:324) at com.cloud.hypervisor.vmware.mo.HypervisorHostHelper.prepareNetwork(HypervisorHostHelper.java:581) at com.cloud.hypervisor.vmware.resource.VmwareResource.prepareNetworkFromNicInfo(VmwareResource.java:2502) at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:1685) at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:450) at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:302) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 2015-09-01 14:29:45,460 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-124:ctx-f38a0ab9) Seq 6-8176003648514424871: Response Received: 2015-09-01 14:29:45,462 DEBUG [c.c.a.t.Request] (DirectAgent-124:ctx-f38a0ab9) Seq 6-8176003648514424871: Processing: { Ans: , MgmtId: 7155482690335, via: 6, Ver: v1, Flags: 110, [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":14,"name":"i-2-14-VM","bootloader":"HVM","type":"User","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"hostName":"test03","arch":"x86_64","os":"Other CentOS (64-bit)","platformEmulator":"centos64Guest","bootArgs":"","enableHA":false,"limitCpuUse":true,"enableDynamicallyScaleVm":true,"vncPassword":"CvGf7J9IGAVShilJsLkb1Q","params":{"cpuNumber":"1","cpuSpeed":"500","keyboard":"us","nicAdapter":"Vmxnet3","vmware.reserve.cpu":"false","nestedVirtualizationFlag":"false","rootDiskController":"scsi","memory":"512","vmware.reserve.mem":"false"},"uuid":"1bfa1b74-7ac4-4d4d-bc4e-492810684d03","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"289bce8e-1818-4abb-9483-7f0e758a956a","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"5d6156f4-b727-43d5-8260-668b826f11d2","id":4,"poolType":"VMFS","host":"VMFS datastore: datastore-2164","path":"datastore-2164","port":0,"url":"VMFS://VMFS datastore: datastore-2164/datastore-2164/?ROLE=Primary&STOREUUID=5d6156f4-b727-43d5-8260-668b826f11d2"}},"name":"ROOT-14","size":42949672960,"path":"ROOT-14-000001","volumeId":17,"vmName":"i-2-14-VM","accountId":2,"format":"OVA","provisioningType":"THIN","id":17,"deviceId":0,"hypervisorType":"VMware"}},"diskSeq":0,"path":"ROOT-14-000001","type":"ROOT","_details":{"managed":"false","storagePort":"0","storageHost":"VMFS datastore: datastore-2164","volumeSize":"42949672960"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":false,"nicUuid":"da6fac33-2b9c-4d6f-b9bb-fdb2610de7e0","uuid":"ab58bf0f-44b4-4a1c-a1ae-4a3a975c6ba1","ip":"10.1.1.239","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:2d:9c:00:01","dns1":"8.8.8.8","dns2":"4.2.2.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://1205","isolationUri":"vlan://1205","isSecurityGroupEnabled":false,"name":"uplink01,1206,nexusdvs"}]},"result":false,"details":"StartCommand failed due to Exception: com.cloud.utils.exception.CloudRuntimeException\nMessage: Severity: error, Error code: InvalidValue, Error type: application, Error Message: Syntax error while parsing 'show port-profile name cloud.guest.1205.200.1-uplink01 '\n\n, Error info: \n show\n \n","wait":0}}] } 2015-09-01 14:29:45,462 DEBUG [c.c.a.t.Request] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Seq 6-8176003648514424871: Received: { Ans: , MgmtId: 7155482690335, via: 6, Ver: v1, Flags: 110, { StartAnswer } } 2015-09-01 14:29:45,463 DEBUG [c.c.a.m.AgentAttache] (DirectAgent-124:ctx-f38a0ab9) Seq 6-8176003648514424871: No more commands found 2015-09-01 14:29:45,465 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Unable to start VM on Host[-6-Routing] due to StartCommand failed due to Exception: com.cloud.utils.exception.CloudRuntimeException Message: Severity: error, Error code: InvalidValue, Error type: application, Error Message: Syntax error while parsing 'show port-profile name cloud.guest.1205.200.1-uplink01 ' , Error info: show 2015-09-01 14:29:45,474 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Cleaning up resources for the vm VM[User|i-2-14-VM] in Starting state 2015-09-01 14:29:45,477 DEBUG [c.c.a.t.Request] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Seq 6-8176003648514424872: Sending { Cmd , MgmtId: 7155482690335, via: 6(172.16.78.7), Ver: v1, Flags: 100111, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"checkBeforeCleanup":false,"vmName":"i-2-14-VM","wait":0}}] } 2015-09-01 14:29:45,477 DEBUG [c.c.a.t.Request] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Seq 6-8176003648514424872: Executing: { Cmd , MgmtId: 7155482690335, via: 6(172.16.78.7), Ver: v1, Flags: 100111, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"checkBeforeCleanup":false,"vmName":"i-2-14-VM","wait":0}}] } 2015-09-01 14:29:45,484 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-496:ctx-a6c715a6) Seq 6-8176003648514424872: Executing request 2015-09-01 14:29:45,485 INFO [c.c.h.v.r.VmwareResource] (DirectAgent-496:ctx-a6c715a6 172.16.78.7, job-190/job-191, cmd: StopCommand) Executing resource StopCommand: {"isProxy":false,"executeInSequence":true,"checkBeforeCleanup":false,"vmName":"i-2-14-VM","wait":0} 2015-09-01 14:29:45,501 DEBUG [c.c.h.v.m.HostMO] (DirectAgent-496:ctx-a6c715a6 172.16.78.7, job-190/job-191, cmd: StopCommand) find VM i-2-14-VM on host 2015-09-01 14:29:45,501 INFO [c.c.h.v.m.HostMO] (DirectAgent-496:ctx-a6c715a6 172.16.78.7, job-190/job-191, cmd: StopCommand) VM i-2-14-VM not found in host cache 2015-09-01 14:29:45,501 DEBUG [c.c.h.v.m.HostMO] (DirectAgent-496:ctx-a6c715a6 172.16.78.7, job-190/job-191, cmd: StopCommand) load VM cache on host 2015-09-01 14:29:45,583 DEBUG [c.c.a.ApiServlet] (catalina-exec-12:ctx-881f1374) ===START=== 79.175.168.183 -- GET command=queryAsyncJobResult&jobId=ac30ca74-9e84-4afc-a031-b2bc3339ca2f&response=json&_=1441101609377 2015-09-01 14:29:45,588 INFO [c.c.h.v.r.VmwareResource] (DirectAgent-496:ctx-a6c715a6 172.16.78.7, job-190/job-191, cmd: StopCommand) VM i-2-14-VM is already in stopped state 2015-09-01 14:29:45,589 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-496:ctx-a6c715a6) Seq 6-8176003648514424872: Response Received: 2015-09-01 14:29:45,589 DEBUG [c.c.a.t.Request] (DirectAgent-496:ctx-a6c715a6) Seq 6-8176003648514424872: Processing: { Ans: , MgmtId: 7155482690335, via: 6, Ver: v1, Flags: 110, [{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM i-2-14-VM is already in stopped state","wait":0}}] } 2015-09-01 14:29:45,589 DEBUG [c.c.a.t.Request] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Seq 6-8176003648514424872: Received: { Ans: , MgmtId: 7155482690335, via: 6, Ver: v1, Flags: 110, { StopAnswer } } 2015-09-01 14:29:45,594 DEBUG [c.c.a.m.AgentAttache] (DirectAgent-496:ctx-a6c715a6) Seq 6-8176003648514424872: No more commands found 2015-09-01 14:29:45,598 DEBUG [c.c.n.NetworkModelImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Service SecurityGroup is not supported in the network id=210 2015-09-01 14:29:45,601 DEBUG [c.c.a.ApiServlet] (catalina-exec-12:ctx-881f1374 ctx-f042c0a7) ===END=== 79.175.168.183 -- GET command=queryAsyncJobResult&jobId=ac30ca74-9e84-4afc-a031-b2bc3339ca2f&response=json&_=1441101609377 2015-09-01 14:29:45,602 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Changing active number of nics for network id=210 on -1 2015-09-01 14:29:45,635 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Asking VirtualRouter to release NicProfile[24-14-ad2dfa71-a5e0-49ae-851b-db19ce732393-10.1.1.239-null 2015-09-01 14:29:45,636 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Successfully released network resources for the vm VM[User|i-2-14-VM] 2015-09-01 14:29:45,636 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Successfully cleanued up resources for the vm VM[User|i-2-14-VM] in Starting state 2015-09-01 14:29:45,638 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Root volume is ready, need to place VM in volume's cluster 2015-09-01 14:29:45,656 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Deploy avoids pods: [], clusters: [], hosts: [6] 2015-09-01 14:29:45,657 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) DeploymentPlanner allocation algorithm: com.cloud.deploy.FirstFitPlanner@2ccd896c 2015-09-01 14:29:45,657 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Trying to allocate a host and storage pools from dc:2, pod:2,cluster:6, requested cpu: 500, requested ram: 536870912 2015-09-01 14:29:45,657 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Is ROOT volume READY (pool already allocated)?: Yes 2015-09-01 14:29:45,657 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) DeploymentPlan has host_id specified, choosing this host and making no checks on this host: 6 2015-09-01 14:29:45,658 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) The specified host is in avoid set 2015-09-01 14:29:45,658 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Cannnot deploy to specified host, returning. 2015-09-01 14:29:45,670 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) VM state transitted from :Starting to Stopped with event: OperationFailedvm's original host id: null new host id: null host id before state transition: 6 2015-09-01 14:29:45,677 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Hosts's actual total CPU: 18128 and CPU after applying overprovisioning: 18128 2015-09-01 14:29:45,677 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Hosts's actual total RAM: 17169104896 and RAM after applying overprovisioning: 17169104896 2015-09-01 14:29:45,677 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) release cpu from host: 6, old used: 500,reserved: 0, actual total: 18128, total with overprovisioning: 18128; new used: 0,reserved:0; movedfromreserved: false,moveToReserveredfalse 2015-09-01 14:29:45,677 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) release mem from host: 6, old used: 536870912,reserved: 0, total: 17169104896; new used: 0,reserved:0; movedfromreserved: false,moveToReserveredfalse 2015-09-01 14:29:45,712 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Invocation exception, caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-14-VM]Scope=interface com.cloud.dc.DataCenter; id=2 2015-09-01 14:29:45,712 INFO [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191 ctx-5e425c12) Rethrow exception com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-14-VM]Scope=interface com.cloud.dc.DataCenter; id=2 2015-09-01 14:29:45,713 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191) Done with run of VM work job: com.cloud.vm.VmWorkStart for VM 14, job origin: 190 2015-09-01 14:29:45,713 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-48:ctx-da3b0826 job-190/job-191) Unable to complete AsyncJobVO {id:191, userId: 2, accountId: 2, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo: rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAADnQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAnBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAAZxAH4ACnBwcHNxAH4ACAAAAAAAAAACcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAEnJPMEFCWFFBQm1SWE1uRnliZ3hw, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 7155482690335, completeMsid: null, lastUpdated: null, lastPolled: null, created: Tue Sep 01 14:19:14 IRDT 2015}, job origin:190 com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-14-VM]Scope=interface com.cloud.dc.DataCenter; id=2 at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:947) at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4481) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107) at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4637) at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:103) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:537) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:494) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)