[
https://issues.apache.org/jira/browse/CLOUDSTACK-3478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13717573#comment-13717573
]
Alena Prokharchyk commented on CLOUDSTACK-3478:
-----------------------------------------------
Happens due to the bug in Volume State Transition. There might be the case when
we have to transit the state from Expunging to Expunging, and it doesn't work
with the event DestroyRequested.
> primary:iscsi:cleanupvolumes fail with runtime exception
> --------------------------------------------------------
>
> Key: CLOUDSTACK-3478
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3478
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Management Server
> Affects Versions: 4.2.0
> Reporter: sadhu suresh
> Assignee: Alena Prokharchyk
> Priority: Critical
> Fix For: 4.2.0
>
> Attachments: management-server.rar
>
>
> steps:
> 1.configured the basic zone with network offering"defaultsharednetscalar
> eip&elb n/w offering and with xen6.02 hyper-visor ,primary storage type iscsi
> 2.deploy few vms
> 3.added one more iscsi based primary storage
> 4deployed few more deploys
> 5.destroy the few vms
> Actual result:
> during expunging vms ,cleannup of volumes failed with runtime exception.
> 2013-07-11 22:37:24,974 DEBUG [db.Transaction.Transaction]
> (UserVm-Scavenger-1:null) Rolling back the transaction: Time = 7 Name =
> cleanupVolumes; called by
> -Transaction.rollback:890-Transaction.removeUpTo:833-Transaction.close:657-TransactionContextBuilder.interceptException:63-ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept:133-VirtualMachineManagerImpl.advanceExpunge:464-UserVmManagerImpl.expunge:1514-UserVmManagerImpl$ExpungeTask.run:1683-Executors$RunnableAdapter.call:471-FutureTask$Sync.innerRunAndReset:351-FutureTask.runAndReset:178-ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201:165
> 2013-07-11 22:37:24,975 WARN [cloud.vm.UserVmManagerImpl]
> (UserVm-Scavenger-1:null) Unable to expunge VM[User|sg1]
> com.cloud.utils.exception.CloudRuntimeException: Failed to update
> state:com.cloud.utils.exception.CloudRuntimeException: Failed to transit
> volume: 7, due to: com.cloud.utils.fsm.NoTransitionException: Unable to
> transition to a new state from Expunging via DestroyRequested
> at
> org.apache.cloudstack.storage.volume.VolumeObject.processEvent(VolumeObject.java:292)
> at
> org.apache.cloudstack.storage.volume.VolumeServiceImpl.destroyVolume(VolumeServiceImpl.java:503)
> at
> com.cloud.storage.VolumeManagerImpl.cleanupVolumes(VolumeManagerImpl.java:2095)
> at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at
> com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:464)
> at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1514)
> at
> com.cloud.vm.UserVmManagerImpl$ExpungeTask.run(UserVmManagerImpl.java:1683)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-07-11 22:37:24,976 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (UserVm-Scavenger-1:null) Stopped called on VM[User|sg2] but the state is
> Expunging
> 2013-07-11 22:37:24,985 DEBUG [cloud.capacity.CapacityManagerImpl]
> (UserVm-Scavenger-1:null) VM state transitted from :Expunging to Expunging
> with event: ExpungeOperationvm's original host id: 2 new host id: null host
> id before state transition: null
> 2013-07-11 22:37:24,985 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (UserVm-Scavenger-1:null) Destroying vm VM[User|sg2]
> 2013-07-11 22:37:24,985 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (UserVm-Scavenger-1:null) Cleaning up NICS
> 2013-07-11 22:37:24,985 DEBUG [cloud.network.NetworkManagerImpl]
> (UserVm-Scavenger-1:null) Cleaning network for vm: 8
> 2013-07-11 22:37:24,990 DEBUG [cloud.storage.VolumeManagerImpl]
> (UserVm-Scavenger-1:null) Cleaning storage for vm: 8
> 2013-07-11 22:37:24,999 DEBUG [storage.volume.VolumeObject]
> (UserVm-Scavenger-1:null) Failed to transit volume: 8, due to:
> com.cloud.utils.fsm.NoTransitionException: Unable to transition to a new
> state from Expunging via DestroyRequested
> 2013-07-11 22:37:24,999 DEBUG [storage.volume.VolumeObject]
> (UserVm-Scavenger-1:null) Failed to update state
> com.cloud.utils.exception.CloudRuntimeException: Failed to transit volume: 8,
> due to: com.cloud.utils.fsm.NoTransitionException: Unable to transition to a
> new state from Expunging via DestroyRequested
> at
> org.apache.cloudstack.storage.volume.VolumeObject.stateTransit(VolumeObject.java:158)
> at
> org.apache.cloudstack.storage.volume.VolumeObject.processEvent(VolumeObject.java:289)
> at
> org.apache.cloudstack.storage.volume.VolumeServiceImpl.destroyVolume(VolumeServiceImpl.java:503)
> at
> com.cloud.storage.VolumeManagerImpl.cleanupVolumes(VolumeManagerImpl.java:2095)
> at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at
> com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:464)
> at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1514)
> at
> com.cloud.vm.UserVmManagerImpl$ExpungeTask.run(UserVmManagerImpl.java:1683)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-07-11 22:37:25,000 DEBUG [db.Transaction.Transaction]
> (UserVm-Scavenger-1:null) Rolling back the transaction: Time = 7 Name =
> cleanupVolumes; called by
> -Transaction.rollback:890-Transaction.removeUpTo:833-Transaction.close:657-TransactionContextBuilder.interceptException:63-ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept:133-VirtualMachineManagerImpl.advanceExpunge:464-UserVmManagerImpl.expunge:1514-UserVmManagerImpl$ExpungeTask.run:1683-Executors$RunnableAdapter.call:471-FutureTask$Sync.innerRunAndReset:351-FutureTask.runAndReset:178-ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201:165
> 2013-07-11 22:37:25,001 WARN [cloud.vm.UserVmManagerImpl]
> (UserVm-Scavenger-1:null) Unable to expunge VM[User|sg2]
> com.cloud.utils.exception.CloudRuntimeException: Failed to update
> state:com.cloud.utils.exception.CloudRuntimeException: Failed to transit
> volume: 8, due to: com.cloud.utils.fsm.NoTransitionException: Unable to
> transition to a new state from Expunging via DestroyRequested
> at
> org.apache.cloudstack.storage.volume.VolumeObject.processEvent(VolumeObject.java:292)
> at
> org.apache.cloudstack.storage.volume.VolumeServiceImpl.destroyVolume(VolumeServiceImpl.java:503)
> at
> com.cloud.storage.VolumeManagerImpl.cleanupVolumes(VolumeManagerImpl.java:2095)
> at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at
> com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:464)
> at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1514)
> at
> com.cloud.vm.UserVmManagerImpl$ExpungeTask.run(UserVmManagerImpl.java:1683)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-07-11 22:37:25,882 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-1:null) SeqA 3-285: Processing Seq 3-285: { Cmd ,
> MgmtId: -1, via: 3, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2013-07-11 22:37:25,990 DEBUG [agent.manager.AgentManagerImpl] (Agent
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
