Chandan Purushothama created CLOUDSTACK-4222:
------------------------------------------------
Summary: [VMWare] NPE: VM Failed to start after Volume Migration
Key: CLOUDSTACK-4222
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4222
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: Management Server
Affects Versions: 4.2.0
Reporter: Chandan Purushothama
Priority: Critical
Fix For: 4.2.0
Attachments: management-server.zip
Test scenario:
Set up have 3 clusters with 1 host each:
1.As regular user - U1 , Create multiple anti-affinity group.
2. Deploy 2 Vms , say Vm11(host1) and VM12(host2) using affinity group A1.
3. Deploy 1 Vm , say Vm21(host1) using affinity group A2.
4. Stop Vm11.
5. Update the list of affinityGroups for this VM to "A1" and "A2"
6. Start the VM.
We expect the VM to now start in host3 . But this is not happening.
===========
Observations:
===========
2013-08-09 15:04:02,279 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-324:null) Seq 6-485097707: Response Received:
2013-08-09 15:04:02,280 DEBUG [agent.transport.Request] (DirectAgent-324:null)
Seq 6-485097707: Processing: { Ans: , MgmtId: 7471666038533, via: 6, Ver: v1,
Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":true,"details":"Success","wait":0}}] }
2013-08-09 15:04:02,280 DEBUG [agent.transport.Request] (Job-Executor-12:job-42
= [ cd54a414-b0c7-476c-ae33-4157f04bbf6e ]) Seq 6-485097707: Received: { Ans:
, MgmtId: 7471666038533, via: 6, Ver: v1, Flags: 10, { Answer } }
2013-08-09 15:04:02,287 INFO [storage.volume.VolumeServiceImpl]
(Job-Executor-12:job-42 = [ cd54a414-b0c7-476c-ae33-4157f04bbf6e ]) Volume 22
is not referred anywhere, remove it from volumes table
2013-08-09 15:04:02,295 ERROR [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-12:job-42 = [ cd54a414-b0c7-476c-ae33-4157f04bbf6e ]) Failed to
start instance VM[User|Admin-VM-6]
java.lang.NullPointerException
at
com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2555)
at
com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:884)
at
com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:574)
at
org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:242)
at
org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
at
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3402)
at
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:1947)
at
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at
org.apache.cloudstack.api.command.user.vm.StartVMCmd.execute(StartVMCmd.java:120)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
at
com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-08-09 15:04:02,302 DEBUG [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-12:job-42 = [ cd54a414-b0c7-476c-ae33-4157f04bbf6e ]) Cleaning up
resources for the vm VM[User|Admin-VM-6] in Starting state
2013-08-09 15:04:02,303 DEBUG [agent.transport.Request] (Job-Executor-12:job-42
= [ cd54a414-b0c7-476c-ae33-4157f04bbf6e ]) Seq 1-2000555966: Sending { Cmd ,
MgmtId: 7471666038533, via: 1, Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"i-2-14-VMWARE","wait":0}}]
}
2013-08-09 15:04:02,303 DEBUG [agent.transport.Request] (Job-Executor-12:job-42
= [ cd54a414-b0c7-476c-ae33-4157f04bbf6e ]) Seq 1-2000555966: Executing: { Cmd
, MgmtId: 7471666038533, via: 1, Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"i-2-14-VMWARE","wait":0}}]
}
2013-08-09 15:04:02,304 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-435:null) Seq 1-2000555966: Executing request
2013-08-09 15:04:02,304 INFO [vmware.resource.VmwareResource]
(DirectAgent-435:10.223.59.66) Executing resource StopCommand:
{"isProxy":false,"executeInSequence":false,"vmName":"i-2-14-VMWARE","wait":0}
mysql> select * from vm_instance where id=14 \G
*************************** 1. row ***************************
id: 14
name: Admin-VM-6
uuid: 09d79220-ecb1-445d-a77a-436787907e15
instance_name: i-2-14-VMWARE
state: Stopped
vm_template_id: 7
guest_os_id: 12
private_mac_address: 02:00:35:ca:00:0c
private_ip_address: 10.1.1.96
pod_id: 1
data_center_id: 1
host_id: NULL
last_host_id: 6
proxy_id: NULL
proxy_assign_time: NULL
vnc_password: gcUA7Y94VVuay7XhyVKeHjjWnha+IuL4/xAvR4v9sT0=
ha_enabled: 0
limit_cpu_use: 0
update_count: 8
update_time: 2013-08-09 22:04:02
created: 2013-08-09 22:00:16
removed: NULL
type: User
vm_type: User
account_id: 2
domain_id: 1
service_offering_id: 1
reservation_id: 6fe0f6f9-79c9-4fcb-91d5-8e4d26d921bf
hypervisor_type: VMware
disk_offering_id: 3
cpu: NULL
ram: NULL
owner: 2
speed: 500
host_name: Admin-VM-6
display_name: Admin-VM-6
desired_state: NULL
dynamically_scalable: 0
display_vm: 1
1 row in set (0.00 sec)
mysql> select * from volumes where instance_id=14;
+----+------------+-----------+---------+--------------+-------------+-----------+---------+--------------------------------------+------------+---------------------------------------------+----------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+--------+----------------------------+-------------+---------------------+----------+---------------------+---------------------+----------+------------+--------------+-----------+------------------------+----------------+--------+----------+----------+
| id | account_id | domain_id | pool_id | last_pool_id | instance_id |
device_id | name | uuid | size |
folder | path
| pod_id | data_center_id | iscsi_name | host_ip | volume_type | pool_type |
disk_offering_id | template_id | iso_id | first_snapshot_backup_uuid |
recreatable | created | attached | updated | removed
| state | chain_info | update_count | disk_type |
vm_snapshot_chain_size | display_volume | format | min_iops | max_iops |
+----+------------+-----------+---------+--------------+-------------+-----------+---------+--------------------------------------+------------+---------------------------------------------+----------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+--------+----------------------------+-------------+---------------------+----------+---------------------+---------------------+----------+------------+--------------+-----------+------------------------+----------------+--------+----------+----------+
| 22 | 2 | 1 | 202 | NULL | 14 |
0 | ROOT-14 | c470c3cb-0d7e-430d-bc4e-2d168c32166d | 2147483648 | NULL
| ROOT-14 | NULL |
1 | NULL | NULL | ROOT | NULL | 1
| 7 | 0 | NULL | 0 | 2013-08-09
22:00:16 | NULL | 2013-08-09 22:04:02 | 2013-08-09 22:04:02 | Expunged |
NULL | 8 | NULL | NULL | 0
| OVA | NULL | NULL |
| 23 | 2 | 1 | 202 | NULL | 14 |
1 | DATA-14 | dd55a55c-559a-4a98-b3fb-4736b9de31f8 | 5368709120 | NULL
| b9e14eb92e684577a135a6134395ebfb | NULL |
1 | NULL | NULL | DATADISK | NULL | 3
| NULL | 0 | NULL | 0 | 2013-08-09
22:00:16 | NULL | 2013-08-09 22:00:36 | NULL | Ready |
NULL | 2 | NULL | NULL | 0
| OVA | NULL | NULL |
| 24 | 2 | 1 | 200 | 202 | 14 |
0 | ROOT-14 | 04ca33ba-bb57-4553-8fcf-63d850a8e474 | 2147483648 |
/export/home/chandan/307PB-195-103/primary1 | eb321282361c4fd692cb48cdc11137bb
| 1 | 1 | NULL | NULL | ROOT | NULL |
1 | 7 | 0 | NULL | 0
| 2013-08-09 22:01:29 | NULL | 2013-08-09 22:03:57 | NULL |
Ready | NULL | 2 | NULL | NULL |
0 | NULL | NULL | NULL |
+----+------------+-----------+---------+--------------+-------------+-----------+---------+--------------------------------------+------------+---------------------------------------------+----------------------------------+--------+----------------+------------+---------+-------------+-----------+------------------+-------------+--------+----------------------------+-------------+---------------------+----------+---------------------+---------------------+----------+------------+--------------+-----------+------------------------+----------------+--------+----------+----------+
3 rows in set (0.00 sec)
mysql>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
