[
https://issues.apache.org/jira/browse/CLOUDSTACK-4506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
edison su resolved CLOUDSTACK-4506.
-----------------------------------
Resolution: Fixed
> In a mixed hypervisor setup, destroying a VM whose host has been removed,
> throws a NPE and the ROOT volume of that VM also is not deleted from the
> primary.
> -----------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-4506
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4506
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Storage Controller
> Affects Versions: 4.2.0
> Environment: Advanced zone setup having clusters of different
> hypervisor types. Ex KVM and VMWARE
> Reporter: Abhinav Roy
> Assignee: edison su
> Priority: Critical
> Fix For: 4.3.0
>
> Attachments: CS-4506.zip
>
>
> Steps :
> =========================
> 1. Deploy a CS 4.2 setup with KVM and VMWARE clusters having one host each.
> 2. Create some VMs on KVM (ex- kvm1 and kvm2)
> 3. Put the KVM host in maintenance mode and then remove the Host. Now kvm1
> and kvm2 are in stopped state.
> 4. Destroy kvm1
> Observations :
> ========================
> 1. when kvm1 is destroyed it fails with the following exception
> 2013-08-26 13:10:09,205 DEBUG [cloud.api.ApiServlet] (catalina-exec-19:null)
> ===START=== 10.144.6.17 -- GET
> command=destroyVirtualMachine&id=d613f6e5-c53a-4f6b-be31-32f101eb6c99&response=json&sessionkey=bU49tWdfVUJUrq65PbMmzGbe0PE%3D&_=1377502661180
> 2013-08-26 13:10:09,242 DEBUG [cloud.async.AsyncJobManagerImpl]
> (catalina-exec-19:null) submit async job-78 = [
> 8018e79c-a47e-4a54-b3a4-68963f3c11a9 ], details: AsyncJobVO {id:78, userId:
> 2, accountId: 2, sessionKey: null, instanceType: VirtualMachine, instanceId:
> 3, cmd: org.apache.cloudstack.api.command.user.vm.DestroyVMCmd,
> cmdOriginator: null, cmdInfo:
> {"response":"json","id":"d613f6e5-c53a-4f6b-be31-32f101eb6c99","sessionkey":"bU49tWdfVUJUrq65PbMmzGbe0PE\u003d","cmdEventType":"VM.DESTROY","ctxUserId":"2","httpmethod":"GET","_":"1377502661180","ctxAccountId":"2","ctxStartEventId":"245"},
> cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0,
> processStatus: 0, resultCode: 0, result: null, initMsid: 226870599129537,
> completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
> 2013-08-26 13:10:09,245 DEBUG [cloud.api.ApiServlet] (catalina-exec-19:null)
> ===END=== 10.144.6.17 -- GET
> command=destroyVirtualMachine&id=d613f6e5-c53a-4f6b-be31-32f101eb6c99&response=json&sessionkey=bU49tWdfVUJUrq65PbMmzGbe0PE%3D&_=1377502661180
> 2013-08-26 13:10:09,249 DEBUG [cloud.async.AsyncJobManagerImpl]
> (Job-Executor-50:job-78 = [ 8018e79c-a47e-4a54-b3a4-68963f3c11a9 ]) Executing
> org.apache.cloudstack.api.command.user.vm.DestroyVMCmd for job-78 = [
> 8018e79c-a47e-4a54-b3a4-68963f3c11a9 ]
> 2013-08-26 13:10:09,282 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-50:job-78 = [ 8018e79c-a47e-4a54-b3a4-68963f3c11a9 ])
> Destroying vm VM[User|v1]
> 2013-08-26 13:10:09,282 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-50:job-78 = [ 8018e79c-a47e-4a54-b3a4-68963f3c11a9 ]) VM is
> already stopped: VM[User|v1]
> 2013-08-26 13:10:09,308 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-50:job-78 = [ 8018e79c-a47e-4a54-b3a4-68963f3c11a9 ]) VM state
> transitted from :Stopped to Destroyed with event: DestroyRequestedvm's
> original host id: 1 new host id: null host id before state transition: null
> 2013-08-26 13:10:09,324 ERROR [cloud.async.AsyncJobManagerImpl]
> (Job-Executor-50:job-78 = [ 8018e79c-a47e-4a54-b3a4-68963f3c11a9 ])
> Unexpected exception while executing
> org.apache.cloudstack.api.command.user.vm.DestroyVMCmd
> java.lang.NullPointerException
> at
> com.cloud.capacity.CapacityManagerImpl.releaseVmCapacity(CapacityManagerImpl.java:187)
> at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at
> com.cloud.capacity.CapacityManagerImpl.postStateTransitionEvent(CapacityManagerImpl.java:718)
> at
> com.cloud.capacity.CapacityManagerImpl.postStateTransitionEvent(CapacityManagerImpl.java:101)
> at com.cloud.utils.fsm.StateMachine2.transitTo(StateMachine2.java:117)
> at
> com.cloud.vm.VirtualMachineManagerImpl.stateTransitTo(VirtualMachineManagerImpl.java:1324)
> at
> com.cloud.vm.VirtualMachineManagerImpl.destroy(VirtualMachineManagerImpl.java:1355)
> at
> org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.destroyVirtualMachine(VMEntityManagerImpl.java:259)
> at
> org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.destroy(VirtualMachineEntityImpl.java:225)
> at
> com.cloud.vm.UserVmManagerImpl.destroyVm(UserVmManagerImpl.java:3450)
> at
> com.cloud.vm.UserVmManagerImpl.destroyVm(UserVmManagerImpl.java:1986)
> at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at
> org.apache.cloudstack.api.command.user.vm.DestroyVMCmd.execute(DestroyVMCmd.java:100)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
> at
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:636)
> 2013-08-26 13:10:09,328 DEBUG [cloud.async.AsyncJobManagerImpl]
> (Job-Executor-50:job-78 = [ 8018e79c-a47e-4a54-b3a4-68963f3c11a9 ]) Complete
> async job-78 = [ 8018e79c-a47e-4a54-b3a4-68963f3c11a9 ], jobStatus: 2,
> resultCode: 530, result: Error Code: 530 Error text: null
> 2. The volume of that vm is not removed from the primary
> mysql> SELECT * FROM cloud.volumes where id=16;
> +----+------------+-----------+---------+--------------+-------------+-----------+--------+--------------------------------------+------------+---------------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+
> | id | account_id | domain_id | pool_id | last_pool_id | instance_id |
> device_id | name | uuid | size |
> folder | path | pod_id |
> data_center_id | iscsi_name | host_ip | volume_type | pool_type |
> disk_offering_id | template_id | first_snapshot_backup_uuid | recreatable |
> created | attached | updated | removed | state |
> chain_info | update_count | disk_type | vm_snapshot_chain_size | iso_id |
> display_volume | format | min_iops | max_iops |
> +----+------------+-----------+---------+--------------+-------------+-----------+--------+--------------------------------------+------------+---------------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+
> | 16 | 2 | 1 | 7 | 1 | 6 |
> 0 | ROOT-6 | 9ba1da18-bc84-4c6c-9fdd-107b89e09a03 | 8589934592 |
> /cpg_vol/abhinav/kvm-pri2 | ab000d45-4185-4347-9c7a-68e8a5ebbb49 | 1 |
> 1 | NULL | NULL | ROOT | NULL |
> 1 | 4 | NULL | 0 | 2013-08-23
> 12:01:55 | NULL | 2013-08-26 07:40:46 | NULL | Expunging | NULL
> | 6 | NULL | NULL | 0 | 0
> | NULL | NULL | NULL |
> +----+------------+-----------+---------+--------------+-------------+-----------+--------+--------------------------------------+------------+---------------------------+--------------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+----------------------------+-------------+---------------------+----------+---------------------+---------+-----------+------------+--------------+-----------+------------------------+--------+----------------+--------+----------+----------+
> 1 row in set (0.00 sec)
> [root@MS-CentOS60 mnt]# ls primary/
> ab000d45-4185-4347-9c7a-68e8a5ebbb49 KVMFILELOCK KVMHA
> [root@MS-CentOS60 mnt]# ll primary/
> total 1605080
> -rwxr--r--. 1 root root 1640366080 Aug 26 12:29
> ab000d45-4185-4347-9c7a-68e8a5ebbb49
> drwxr-xr-x. 2 root root 4096 Aug 26 12:28 KVMFILELOCK
> drwxr-xr-x. 2 root root 4096 Aug 23 17:29 KVMHA
> 3. This is applicable for all vms, user vms and system vms
> Attaching management server logs and db dump.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
