[
https://issues.apache.org/jira/browse/CLOUDSTACK-3946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13733541#comment-13733541
]
Rajesh Battala commented on CLOUDSTACK-3946:
--------------------------------------------
Looking at the Log, the exception is thrown while creating a clone of datadisk
VO object. Once the VO object is created then only copy will start and later
the source volume will be deleted.
The above statement tells that, copy is done and source volume is deleted, but
the copy itself has not even processed.
Am working on the fix, Copy Volume/Migrate Volume when volume is in ZWPS in KVM
> [KVM] [Zone Wide Primary Storages] migrateVolume is deleting the volume if
> the source and destination pool are same.
> --------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-3946
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3946
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Storage Controller
> Affects Versions: 4.2.0
> Environment: commit # 6275d697e340be5b520f37b15d72343fa47c00a9
> Reporter: venkata swamybabu budumuru
> Assignee: Rajesh Battala
> Priority: Critical
> Fix For: 4.2.0
>
> Attachments: logs.tgz
>
>
> Steps to reproduce :
> 1. Have at least one advanced zone with KVM cluster (2 kvm hosts)
> 2. Make sure that the setup has at least 2 zone wide primary storages. (from
> ex: zwps1, zwps2)
> 3. Deploy a VM as admin
> 4. Create a datadisk of size 5 GB as admin on zwps1
> 5. attach the above datadisk to the VM in step 3
> 6. detach the data disk
> 7. initiate migrateVolume command for the above datadisk to again zwps1
> Observations:
> (i) volume migrate command involved 2 things
> - copyCommand
> - delete the original
> (ii) In my case, I tried to migrate the volume which is on zwps1 to again
> zwps1. This initiated copyCommand followed by storage.command.DeleteCommand.
> (iii) Due to the issue mentioned in CLOUDSTACK-3943, copy command failed.
> The subsequent delete command deleted the orignal volume because the "path
> uuid" for original volume and new volume is maintained with same uuid
> temporarily.
> *** Either we shouldn't let user fire migrateVolume API with storageId if the
> volume's original storageId and destStorageId are same.
> (OR)
> *** we should use a unique path UUID so that in case of copy command failure,
> it will only delete the new volume.
> MGMT server log snippet :
> 2013-07-30 15:40:35,945 DEBUG [cloud.storage.StorageManagerImpl]
> (StorageManager-Scavenger-1:null) Secondary storage garbage collector found 0
> volumes to cleanup on volume_store_ref for store: secondary2
> 2013-07-30 15:40:35,967 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-279:null) Seq 6-1249116579: Response Received:
> 2013-07-30 15:40:35,969 DEBUG [storage.motion.AncientDataMotionStrategy]
> (Job-Executor-40:job-40 = [ 24eb5a75-590f-43a9-bb90-88be177fa359 ]) copyAsync
> inspecting src type VOLUME copyAsync inspecting dest type VOLUME
> 2013-07-30 15:40:35,970 DEBUG [agent.transport.Request]
> (StatsCollector-1:null) Seq 6-1249116579: Received: { Ans: , MgmtId:
> 7280707764394, via: 6, Ver: v1, Flags: 10, { GetHostStatsAnswer } }
> 2013-07-30 15:40:35,974 DEBUG [cache.allocator.StorageCacheRandomAllocator]
> (Job-Executor-40:job-40 = [ 24eb5a75-590f-43a9-bb90-88be177fa359 ]) Can't
> find staging storage in zone: 1
> 2013-07-30 15:40:36,033 DEBUG [agent.transport.Request]
> (Job-Executor-40:job-40 = [ 24eb5a75-590f-43a9-bb90-88be177fa359 ]) Seq
> 3-46924274: Sending { Cmd , MgmtId: 7280707764394, via: 3, Ver: v1, Flags:
> 100011,
> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"8e27413a-9904-4b04-88e2-4646f9e2012e","volumeType":"DATADISK","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"5458182e-bfcb-351c-97ed-e7223bca2b8e","id":1,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/swamy/primary.campo.kvm.1.zone","port":2049}},"name":"adminDataDisk2","size":0,"path":"a8923a0d-401c-453a-bf77-0e4496711f0d","volumeId":13,"accountId":2,"format":"QCOW2","id":13,"hypervisorType":"None"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"8e27413a-9904-4b04-88e2-4646f9e2012e","volumeType":"DATADISK","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/swamy/secondary.campo.kvm.1","_role":"Image"}},"name":"adminDataDisk2","size":0,"path":"volumes/2/13","volumeId":13,"accountId":2,"format":"QCOW2","id":13,"hypervisorType":"None"}},"executeInSequence":false,"wait":10800}}]
> }
> 2013-07-30 15:40:36,039 DEBUG [agent.transport.Request]
> (AgentManager-Handler-4:null) Seq 3-46924274: Processing: { Ans: , MgmtId:
> 7280707764394, via: 3, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.UnsupportedAnswer":{"result":false,"details":"Unsupported
> command issued:org.apache.cloudstack.storage.command.CopyCommand. Are you
> sure you got the right type of server?","wait":0}}] }
> 2013-07-30 15:40:36,039 DEBUG [agent.transport.Request]
> (Job-Executor-40:job-40 = [ 24eb5a75-590f-43a9-bb90-88be177fa359 ]) Seq
> 3-46924274: Received: { Ans: , MgmtId: 7280707764394, via: 3, Ver: v1,
> Flags: 10, { UnsupportedAnswer } }
> 2013-07-30 15:40:36,039 WARN [agent.manager.AgentManagerImpl]
> (Job-Executor-40:job-40 = [ 24eb5a75-590f-43a9-bb90-88be177fa359 ])
> Unsupported Command: Unsupported command
> issued:org.apache.cloudstack.storage.command.CopyCommand. Are you sure you
> got the right type of server?
> 2013-07-30 15:40:36,039 DEBUG [storage.motion.AncientDataMotionStrategy]
> (Job-Executor-40:job-40 = [ 24eb5a75-590f-43a9-bb90-88be177fa359 ]) copy to
> image store failed: Unsupported command
> issued:org.apache.cloudstack.storage.command.CopyCommand. Are you sure you
> got the right type of server?
> 2013-07-30 15:40:36,062 DEBUG [storage.image.BaseImageStoreDriverImpl]
> (Job-Executor-40:job-40 = [ 24eb5a75-590f-43a9-bb90-88be177fa359 ]) Unable to
> destoy VOLUME: 13
> java.lang.NullPointerException
> at
> org.apache.cloudstack.storage.volume.VolumeObject.getPath(VolumeObject.java:338)
> at
> org.apache.cloudstack.storage.to.VolumeObjectTO.<init>(VolumeObjectTO.java:53)
> at
> org.apache.cloudstack.storage.volume.VolumeObject.getTO(VolumeObject.java:460)
> at
> org.apache.cloudstack.storage.image.BaseImageStoreDriverImpl.deleteAsync(BaseImageStoreDriverImpl.java:225)
> at
> org.apache.cloudstack.storage.image.store.ImageStoreImpl.delete(ImageStoreImpl.java:148)
> at
> org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyVolumeBetweenPools(AncientDataMotionStrategy.java:302)
> at
> org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:381)
> at
> org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
> at
> org.apache.cloudstack.storage.volume.VolumeServiceImpl.copyVolume(VolumeServiceImpl.java:782)
> at
> com.cloud.storage.VolumeManagerImpl.migrateVolume(VolumeManagerImpl.java:2210)
> at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at
> com.cloud.storage.VolumeManagerImpl.migrateVolume(VolumeManagerImpl.java:2202)
> at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at
> org.apache.cloudstack.api.command.user.volume.MigrateVolumeCmd.execute(MigrateVolumeCmd.java:102)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
> at
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:679)
> 2013-07-30 15:40:36,064 WARN
> [storage.datastore.ObjectInDataStoreManagerImpl] (Job-Executor-40:job-40 = [
> 24eb5a75-590f-43a9-bb90-88be177fa359 ]) Volume 13 is not found on image store
> 1, so no need to delete
> 2013-07-30 15:40:36,087 WARN
> [storage.datastore.ObjectInDataStoreManagerImpl] (Job-Executor-40:job-40 = [
> 24eb5a75-590f-43a9-bb90-88be177fa359 ]) Unsupported data object (VOLUME,
> org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@75103941)
> 2013-07-30 15:40:36,101 WARN
> [storage.datastore.ObjectInDataStoreManagerImpl] (Job-Executor-40:job-40 = [
> 24eb5a75-590f-43a9-bb90-88be177fa359 ]) Unsupported data object (VOLUME,
> org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@3af3cacd)
> 2013-07-30 15:40:36,193 DEBUG [agent.transport.Request]
> (Job-Executor-40:job-40 = [ 24eb5a75-590f-43a9-bb90-88be177fa359 ]) Seq
> 3-46924275: Sending { Cmd , MgmtId: 7280707764394, via: 3, Ver: v1, Flags:
> 100011,
> [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"05d76cf3-7df5-459f-a9a9-7a0a271ff5d6","volumeType":"DATADISK","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"5458182e-bfcb-351c-97ed-e7223bca2b8e","id":1,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/swamy/primary.campo.kvm.1.zone","port":2049}},"name":"adminDataDisk2","size":0,"path":"a8923a0d-401c-453a-bf77-0e4496711f0d","volumeId":15,"accountId":2,"id":15,"hypervisorType":"None"}},"wait":0}}]
> }
> 2013-07-30 15:40:36,338 DEBUG [agent.transport.Request]
> (AgentManager-Handler-9:null) Seq 3-46924275: Processing: { Ans: , MgmtId:
> 7280707764394, via: 3, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
> 2013-07-30 15:40:36,338 DEBUG [agent.transport.Request]
> (Job-Executor-40:job-40 = [ 24eb5a75-590f-43a9-bb90-88be177fa359 ]) Seq
> 3-46924275: Received: { Ans: , MgmtId: 7280707764394, via: 3, Ver: v1,
> Flags: 10, { Answer } }
> 2013-07-30 15:40:36,358 INFO [storage.volume.VolumeServiceImpl]
> (Job-Executor-40:job-40 = [ 24eb5a75-590f-43a9-bb90-88be177fa359 ]) Volume 15
> is not referred anywhere, remove it from volumes table
> 2013-07-30 15:40:36,374 ERROR [cloud.storage.VolumeManagerImpl]
> (Job-Executor-40:job-40 = [ 24eb5a75-590f-43a9-bb90-88be177fa359 ]) migrate
> volume failed:Unsupported command
> issued:org.apache.cloudstack.storage.command.CopyCommand. Are you sure you
> got the right type of server?
> 2013-07-30 15:40:36,376 DEBUG [cloud.async.AsyncJobManagerImpl]
> (Job-Executor-40:job-40 = [ 24eb5a75-590f-43a9-bb90-88be177fa359 ]) Complete
> async job-40 = [ 24eb5a75-590f-43a9-bb90-88be177fa359 ], jobStatus: 2,
> resultCode: 530, result: Error Code: 530 Error text: Failed to migrate volume
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
