Sangeetha Hariharan created CLOUDSTACK-4579: -----------------------------------------------
Summary: [Upgrade 2.2.14 - 4.2][KVM] - After upgrade , Not able to deploy Vms with existing templates. Key: CLOUDSTACK-4579 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4579 Project: CloudStack Issue Type: Bug Security Level: Public (Anyone can view this level - this is the default.) Components: Management Server Affects Versions: 4.2.0 Environment: Upgrade from 2.2.14 - > 4.2 with rhel 6.0 hosts. Reporter: Sangeetha Hariharan Assignee: edison su Priority: Blocker Fix For: 4.2.0 Deploy a 2.2.14 setup with 2 rhel 6.0 hosts in cluster. Upgrade to 4.2. Deploy a new VM using an existing template. Vm deployment fails with following exception in the management server logs: 2013-08-31 00:02:35,408 DEBUG [agent.transport.Request] (AgentManager-Handler-4:null) Seq 2-1176895727: Processing: { Ans: , MgmtId: 6860035851996, via: 2, Ver: v1, Flags: 110, [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"com.cloud.utils.exception.CloudRuntimeException: com.cloud.utils.exception.CloudRuntimeException: Can't find volume:/mnt/f691b1f4-e2a1-3777-8747-eda71b158aba/db0c5f4f-86d0-4aa6-9e06-ff91ad3a62ff","wait":0}}] } 2013-08-31 00:02:35,408 DEBUG [agent.manager.AgentAttache] (AgentManager-Handler-4:null) Seq 2-1176895727: No more commands found 2013-08-31 00:02:35,408 DEBUG [agent.transport.Request] (Job-Executor-11:job-35 = [ 4b0d3b97-1cc9-4edc-85d5-80cd7add0035 ]) Seq 2-1176895727: Received: { Ans: , MgmtId: 6860035851996, via: 2, Ver: v1, Flags: 110, { CopyCmdAnswer } } 2013-08-31 00:02:35,413 WARN [storage.datastore.ObjectInDataStoreManagerImpl] (Job-Executor-11:job-35 = [ 4b0d3b97-1cc9-4edc-85d5-80cd7add0035 ]) Unsupported data object (VOLUME, org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@523f72ca), no need to delete from object in store ref table 2013-08-31 00:02:35,413 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-11:job-35 = [ 4b0d3b97-1cc9-4edc-85d5-80cd7add0035 ]) Unable to create Vol[17|vm=14|ROOT]:com.cloud.utils.exception.CloudRuntimeException: com.cloud.utils.exception.CloudRuntimeException: Can't find volume:/mnt/f691b1f4-e2a1-3777-8747-eda71b158aba/db0c5f4f-86d0-4aa6-9e06-ff91ad3a62ff 2013-08-31 00:02:35,413 INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-11:job-35 = [ 4b0d3b97-1cc9-4edc-85d5-80cd7add0035 ]) Unable to contact resource. com.cloud.exception.StorageUnavailableException: Resource [StoragePool:200] is unreachable: Unable to create Vol[17|vm=14|ROOT]:com.cloud.utils.exception.CloudRuntimeException: com.cloud.utils.exception.CloudRuntimeException: Can't find volume:/mnt/f691b1f4-e2a1-3777-8747-eda71b158aba/db0c5f4f-86d0-4aa6-9e06-ff91ad3a62ff at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2544) at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2592) at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:888) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:578) at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227) at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3406) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2966) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2952) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) 2013-08-31 00:02:35,418 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-11:job-35 = [ 4b0d3b97-1cc9-4edc-85d5-80cd7add0035 ]) Cleaning up resources for the vm VM[User|test123] in Starting state 2013-08-31 00:02:35,420 DEBUG [agent.transport.Request] (Job-Executor-11:job-35 = [ 4b0d3b97-1cc9-4edc-85d5-80cd7add0035 ]) Seq 2-1176895743: Sending { Cmd , MgmtId: 6860035851996, via: 2, Ver: v1, Flags: 100111, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vmName":"i-2-14-VM","wait":0}}] } 2013-08-31 00:02:35,495 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-6:null) Ping from 11 2013-08-31 00:02:35,647 DEBUG [agent.transport.Request] (AgentManager-Handler-5:null) Seq 2-1176895743: Processing: { Ans: , MgmtId: 6860035851996, via: 2, Ver: v1, Flags: 110, [{"com.cloud.agent.api.StopAnswer":{"vncPort":0,"result":true,"wait":0}}] } mysql> select * from template_spool_ref; +----+---------+-------------+---------------------+--------------+--------+--------------+----------------+-----------+--------------------------------------+--------------------------------------------------------------------------------+---------------+---------------+-------+--------------+---------------------+ | id | pool_id | template_id | created | last_updated | job_id | download_pct | download_state | error_str | local_path | install_path | template_size | marked_for_gc | state | update_count | updated | +----+---------+-------------+---------------------+--------------+--------+--------------+----------------+-----------+--------------------------------------+--------------------------------------------------------------------------------+---------------+---------------+-------+--------------+---------------------+ | 1 | 200 | 3 | 2013-08-30 19:36:39 | NULL | NULL | 100 | DOWNLOADED | NULL | 9800946f-ebd8-40c2-a4a0-11279a3006aa | /mnt/f691b1f4-e2a1-3777-8747-eda71b158aba/9800946f-ebd8-40c2-a4a0-11279a3006aa | 719642624 | 0 | Ready | 0 | NULL | | 2 | 200 | 202 | 2013-08-30 19:55:03 | NULL | NULL | 100 | DOWNLOADED | NULL | db0c5f4f-86d0-4aa6-9e06-ff91ad3a62ff | /mnt/f691b1f4-e2a1-3777-8747-eda71b158aba/db0c5f4f-86d0-4aa6-9e06-ff91ad3a62ff | 1771274240 | 0 | Ready | 0 | NULL | | 3 | 200 | 4 | 2013-08-30 20:20:20 | NULL | NULL | 100 | DOWNLOADED | NULL | d46f9ae5-d491-41f1-916b-8344695fc27c | /mnt/f691b1f4-e2a1-3777-8747-eda71b158aba/d46f9ae5-d491-41f1-916b-8344695fc27c | 1771274240 | 0 | Ready | 0 | NULL | | 4 | 200 | 206 | 2013-08-30 23:14:34 | NULL | NULL | 100 | DOWNLOADED | NULL | 6fe82ba6-a03b-48d9-8292-812c9885259c | 6fe82ba6-a03b-48d9-8292-812c9885259c | 0 | 0 | Ready | 2 | 2013-08-30 23:14:56 | | 5 | 200 | 207 | 2013-08-31 00:39:43 | NULL | NULL | 100 | DOWNLOADED | NULL | 3d4346a2-539c-4833-8afd-3add8b36f0cd | 3d4346a2-539c-4833-8afd-3add8b36f0cd | 0 | 0 | Ready | 2 | 2013-08-31 00:40:59 | +----+---------+-------------+---------------------+--------------+--------+--------------+----------------+-----------+--------------------------------------+--------------------------------------------------------------------------------+---------------+---------------+-------+--------------+---------------------+ 5 rows in set (0.00 sec) Note - Deploying a Vm from a template that gets registered after the upgrade works fine. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira