[
https://issues.apache.org/jira/browse/CLOUDSTACK-3715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815687#comment-13815687
]
Likitha Shetty commented on CLOUDSTACK-3715:
--------------------------------------------
VMware vCenter Session timeout value has now been increased to 20 minutes. This
should resolve the timeout issues seen in setups with slow storage.
It is straight forward to make this value configurable for tasks that are
invoked by the Management server. But it is quite tricky for the ones invoked
by SSVM. Hence created issue
https://issues.apache.org/jira/browse/CLOUDSTACK-5069 to track the effort to
make this timeout value configurable (4.3).
> Live Migration of Virtual instances operation is getting timedout
> ------------------------------------------------------------------
>
> Key: CLOUDSTACK-3715
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3715
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Management Server, VMware
> Affects Versions: 4.2.0
> Reporter: Sailaja Mada
> Assignee: Likitha Shetty
> Priority: Critical
> Fix For: 4.2.1
>
> Attachments: 195113management-server.log.gz,
> 195117management-server.log.gz, apilog.log, cloud-backup.dmp.gz,
> cloud-backup.sql.gz, management-server.log, migrationlogs.rar
>
>
> Setup: Multinode Management setup.
> Steps:
> 1. Configure Adv Zone with 2 VMWARE clusters each with one hosts with Zone
> wide primary storage ( Standard vSwitch cluster)
> 2. Deploy VM using User account
> 3. Tried to Live migrate VM from cluster1 (host 1) to Cluster 2 (host2 )
> Observation:
> 1. Migration took very log time and finally failed saying operation timed out
> :
> 2013-07-22 17:46:06,288 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-40:job-133 = [ 4a2f6d84-236d-4bdd-a2e5-72fa336e0274 ]) VM state
> transitted from :Migrating to Running with event: OperationFailedvm's
> original host id: 4 new host id: 4 host id before state transition: 1
> 2013-07-22 17:46:06,292 INFO [vmware.resource.VmwareResource]
> (DirectAgent-421:10.102.192.18) VM i-4-9-VM is no longer in vSphere
> 2013-07-22 17:46:06,293 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-421:null) Seq 1-1311245319: Response Received:
> 2013-07-22 17:46:06,294 DEBUG [agent.transport.Request]
> (DirectAgent-421:null) Seq 1-1311245319: Processing: { Ans: , MgmtId:
> 94838926819810, via: 1, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.StopAnswer":{"vncPort":0,"result":true,"details":"VM
> i-4-9-VM is no longer in vSphere","wait":0}}] }
> 2013-07-22 17:46:06,294 DEBUG [agent.manager.AgentAttache]
> (DirectAgent-421:null) Seq 1-1311245319: Unable to find listener.
> 2013-07-22 17:46:06,307 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-40:job-133 = [ 4a2f6d84-236d-4bdd-a2e5-72fa336e0274 ]) Hosts's
> actual total CPU: 9572 and CPU after applying overprovisioning: 9572
> 2013-07-22 17:46:06,307 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-40:job-133 = [ 4a2f6d84-236d-4bdd-a2e5-72fa336e0274 ]) Hosts's
> actual total RAM: 17166258176 and RAM after applying overprovisioning:
> 17166258176
> 2013-07-22 17:46:06,307 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-40:job-133 = [ 4a2f6d84-236d-4bdd-a2e5-72fa336e0274 ]) release
> cpu from host: 1, old used: 2000,reserved: 0, actual total: 9572, total with
> overprovisioning: 9572; new used: 200,reserved:0; movedfromreserved:
> false,moveToReserveredfalse
> 2013-07-22 17:46:06,307 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-40:job-133 = [ 4a2f6d84-236d-4bdd-a2e5-72fa336e0274 ]) release
> mem from host: 1, old used: 2013265920,reserved: 0, total: 17166258176; new
> used: 2013265920,reserved:0; movedfromreserved: false,moveToReserveredfalse
> 2013-07-22 17:46:06,345 ERROR [cloud.async.AsyncJobManagerImpl]
> (Job-Executor-40:job-133 = [ 4a2f6d84-236d-4bdd-a2e5-72fa336e0274 ])
> Unexpected exception while executing
> org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd
> com.cloud.utils.exception.CloudRuntimeException: Failed to migrated vm
> VM[User|newuser1i1] along with its volumes.
> com.cloud.exception.AgentUnavailableException: Resource [Host:1] is
> unreachable: Host 1: Operation timed out on storage motion for
> VM[User|newuser1i1]
> at
> com.cloud.storage.VolumeManagerImpl.migrateVolumes(VolumeManagerImpl.java:2263)
> at
> com.cloud.vm.VirtualMachineManagerImpl.migrateWithStorage(VirtualMachineManagerImpl.java:1780)
> at
> com.cloud.vm.UserVmManagerImpl.migrateVirtualMachineWithVolume(UserVmManagerImpl.java:4046)
> at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at
> org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd.execute(MigrateVirtualMachineWithVolumeCmd.java:137)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
> at
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-07-22 17:46:06,350 DEBUG [cloud.async.AsyncJobManagerImpl]
> (Job-Executor-40:job-133 = [ 4a2f6d84-236d-4bdd-a2e5-72fa336e0274 ]) Complete
> async job-133 = [ 4a2f6d84-236d-4bdd-a2e5-72fa336e0274 ], jobStatus: 2,
> resultCode: 530, result: Error Code: 530 Error text: Failed to migrated vm
> VM[User|newuser1i1] along with its volumes.
> com.cloud.exception.AgentUnavailableException: Resource [Host:1] is
> unreachable: Host 1: Operation timed out on storage motion for
> VM[User|newuser1i1]
--
This message was sent by Atlassian JIRA
(v6.1#6144)
