[
https://issues.apache.org/jira/browse/CLOUDSTACK-5712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13860132#comment-13860132
]
Koushik Das commented on CLOUDSTACK-5712:
-----------------------------------------
What is the value for config parameter execute.in.sequence.hypervisor.commands?
Based on the error logs and some recent issues (with similar logs) looks like
the config is set to false. Run the automation with this flag set to true for
now.
One side effect of enabling parallel vm operations is that operations on a
single VM also gets executed in parallel and these issues are seen. There needs
to be a mechanism to serialize operations for a single VM.
> [Automation] Failed to stop vm during vm destroy command, observed the error
> "Unable to update VM"
> ---------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-5712
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5712
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Management Server
> Affects Versions: 4.3.0
> Environment: KVM (RHEL 6.3)
> Reporter: Rayees Namathponnan
> Priority: Blocker
> Fix For: 4.3.0
>
> Attachments: management-server.rar
>
>
> This issue is observed during regression automation; there are many vm stop
> operation failed with below error "Unable to update VM"
> 2014-01-01 07:59:40,877 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Job-Executor-107:ctx-740b1e62 ctx-e6d04e1c) Successfully released network
> resources for the vm VM[User|QA-2ce28269-1eea-4514-8d75-59272f71f5ff]
> 2014-01-01 07:59:40,877 DEBUG [c.c.v.VirtualMachineManagerImpl]
> (Job-Executor-107:ctx-740b1e62 ctx-e6d04e1c) Successfully released storage
> resources for the vm VM[User|QA-2ce28269-1eea-4514-8d75-59272f71f5ff]
> 2014-01-01 07:59:40,881 DEBUG [c.c.v.d.VMInstanceDaoImpl]
> (Job-Executor-107:ctx-740b1e62 ctx-e6d04e1c) Unable to update
> VM[User|QA-2ce28269-1eea-4514-8d75-59272f71f5ff]: DB Data={Host=1;
> State=Stopped; updated=10; time=Wed Jan 01 07:57:53 PST 2014} New Data:
> {Host=null; State=Stopped; updated=10; time=Wed Jan 01 07:57:53 PST 2014}
> Stale Data: {Host=1; State=Stopping; updated=9; time=Wed Jan 01 07:57:37 PST
> 2014}
> 2014-01-01 07:59:40,887 ERROR [c.c.a.ApiAsyncJobDispatcher]
> (Job-Executor-107:ctx-740b1e62) Unexpected exception while executing
> org.apache.cloudstack.api.command.user.vm.DestroyVMCmd
> com.cloud.utils.exception.CloudRuntimeException: unable to stop
> VM[User|QA-2ce28269-1eea-4514-8d75-59272f71f5ff]
> at
> com.cloud.vm.VirtualMachineManagerImpl.advanceStop(VirtualMachineManagerImpl.java:1475)
> at
> com.cloud.vm.VirtualMachineManagerImpl.destroy(VirtualMachineManagerImpl.java:1528)
> at
> org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.destroyVirtualMachine(VMEntityManagerImpl.java:256)
> at
> org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.destroy(VirtualMachineEntityImpl.java:223)
> at
> com.cloud.vm.UserVmManagerImpl.destroyVm(UserVmManagerImpl.java:3616)
> at
> com.cloud.vm.UserVmManagerImpl.destroyVm(UserVmManagerImpl.java:2085)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at
> com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:50)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at $Proxy169.destroyVm(Unknown Source)
> at
> org.apache.cloudstack.api.command.user.vm.DestroyVMCmd.execute(DestroyVMCmd.java:112)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
> at
> com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
> at
> com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
> at
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:522)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2014-01-01 07:59:40,888 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Job-Executor-107:ctx-740b1e62) Complete async job-387, jobStatus: FAILED,
> resultCode: 530, result:
> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"unable
> to stop VM[User|QA-2ce28269-1eea-4514-8d75-59272f71f5ff]"}
> 2014-01-01 07:59:40,905 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (Job-Executor-107:ctx-740b1e62) Done executing
> org.apache.cloudstack.api.command.user.vm.DestroyVMCmd for job-387
> 2014-01-01 07:59:40,909 INFO [o.a.c.f.j.i.AsyncJobMonitor]
> (Job-Executor-107:ctx-740b1e62) Remove job-387 from job monitoring
> 2014-01-01 07:59:41,412 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-11:null) SeqA 6-372: Processing Seq 6-372: { Cmd ,
> MgmtId: -1, via: 6, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":3,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2014-01-01 07:59:41,416 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-11:null) SeqA 6-372: Sending Seq
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
