Marcus Sorensen created CLOUDSTACK-2003:
-------------------------------------------
Summary: Deleting domain while deleted account is cleaning up
leaves VMs expunging forever due to 'Failed to update resource count'
Key: CLOUDSTACK-2003
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-2003
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: Management Server
Affects Versions: 4.1.0
Reporter: Marcus Sorensen
Priority: Blocker
Fix For: 4.1.0
Marking this as a blocker because it requires admin to manually edit database
to fix, otherwise VMs can stack up forever in expunging under a certain
scenario.
Create a cloudstack zone
Create a domain
Create an account within the domain
Launch VMs with the account
Delete the account, this should begin stopping/destroying VMs associated with
the account
Before the account job is finished, delete the domain. Now Domain is gone and
account is gone, but associated VMs never clean up.
Expunging will error out due to the domain missing:
2013-04-10 16:34:52,925 ERROR [cloud.resourcelimit.ResourceLimitManagerImpl]
(Job-Executor-7:job-39) Failed to update resource count for account
2013-04-10 16:34:53,019 WARN [cloud.vm.UserVmManagerImpl]
(Job-Executor-7:job-39) Concurrent operations on expunging
VM[User|7646b383-7cd7-48a1-a056-a48fb54111c8]
com.cloud.exception.ConcurrentOperationException: Failed to transit state
at
com.cloud.storage.StorageManagerImpl.destroyVolume(StorageManagerImpl.java:2250)
at
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at
com.cloud.storage.StorageManagerImpl.cleanupVolumes(StorageManagerImpl.java:3725)
at
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at
com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:409)
at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1776)
at
com.cloud.user.AccountManagerImpl.cleanupAccount(AccountManagerImpl.java:578)
at
com.cloud.user.AccountManagerImpl.deleteAccount(AccountManagerImpl.java:515)
at
com.cloud.user.AccountManagerImpl.deleteUserAccount(AccountManagerImpl.java:1180)
If I manually re-enable the domain in the database, expunging completes. Then I
can safely delete the account. This is a regression from the 4.0 behavior,
according to our test suites, where account and domain could be safely removed
and associated resources would clean up.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
