Hi, Dave. What does the VM and it's volumes look like in the database? select * from vm_instance where id = 620; select * from volumes where instance_id = 620;
Best regards, Kirk On 12/16/2012 03:56 PM, David Comerford wrote: > Hey, > > Due to some clocks being a few months forward for a while I'm stuck back in > the present with a number of VMs that just won't expunge. > Logs suggest there's some concurrent operations on said VM which I think > basically means theres some job that lives in the future which is blocking > the current ones. > > My questions: > 1) From the logs below, does my guess sound plausible? > 2) Where in the database is the queue for jobs I need to take a hammer to? > > Cheers guys, > Dave > > 2012-12-16 23:44:01,132 WARN [cloud.vm.UserVmManagerImpl] > (AccountChecker-1:null) Concurrent operations on expunging > VM[User|a1a470ef-368b-45fe-99c7-1b92ea333d78] > com.cloud.exception.ConcurrentOperationException: Failed to transit state > at > com.cloud.storage.StorageManagerImpl.destroyVolume(StorageManagerImpl.java:2060) > at > com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:30) > at > com.cloud.storage.StorageManagerImpl.cleanupVolumes(StorageManagerImpl.java:3486) > at > com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:30) > at > com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:391) > at > com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1160) > at > com.cloud.user.AccountManagerImpl.cleanupAccount(AccountManagerImpl.java:518) > at > com.cloud.user.AccountManagerImpl$AccountCleanupTask.run(AccountManagerImpl.java:1291) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2012-12-16 23:44:01,132 ERROR [cloud.user.AccountManagerImpl] > (AccountChecker-1:null) Unable to expunge vm: 620 >
