[
https://issues.apache.org/jira/browse/CLOUDSTACK-4652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13765309#comment-13765309
]
sadhu suresh commented on CLOUDSTACK-4652:
------------------------------------------
fail to attach a volume after migrating from primary to another primary.
- GET
command=attachVolume&id=3d67606b-eeaf-474e-a7f4-95e24a4da7fd&virtualMachineId=42c31b97-d376-4e04-b6c3-08cecda7c13f&response=json&sessionkey=W%2F%2BBolWn%2BDoUEK1P92X71fVmknc%3D&_=1378979512588
2013-09-12 20:44:19,627 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-9:job-23 = [ 3158826e-9c85-4210-bc1a-d97ce9865a3d ]) Executing
org.apache.cloudstack.api.command.user.volume.AttachVolumeCmd for job-23 = [
3158826e-9c85-4210-bc1a-d97ce9865a3d ]
2013-09-12 20:44:19,956 ERROR [cloud.async.AsyncJobManagerImpl]
(Job-Executor-9:job-23 = [ 3158826e-9c85-4210-bc1a-d97ce9865a3d ]) Unexpected
exception while executing
org.apache.cloudstack.api.command.user.volume.AttachVolumeCmd
com.cloud.utils.exception.CloudRuntimeException: The VM VM33 has more than one
ROOT volume and is in an invalid state.
at
com.cloud.storage.VolumeManagerImpl.attachVolumeToVM(VolumeManagerImpl.java:1815)
at
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at
org.apache.cloudstack.api.command.user.volume.AttachVolumeCmd.execute(AttachVolumeCmd.java:122)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
at
com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
2013-09-12 20:44:19,980 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-9:job-23 = [ 3158826e-9c85-4210-bc1a-d97ce9865a3d ]) Complete
async job-23 = [ 3158826e-9c85-4210-bc1a-d97ce9865a3d ], jobStatus: 2,
resultCode: 530, result: Error Code: 530 Error text: The VM VM33 has more than
one ROOT volume and is in an invalid state.
2013-09-12 20:44:22,270 DEBUG [agent.manager.AgentManagerImpl]
(AgentManager-Handler-3:null) SeqA 2-1802: Processing Seq 2-1802: { Cmd ,
MgmtId: -1, via: 2, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2013-09-12 20:44:22,279 DEBUG [agent.manager.AgentManagerImpl]
(AgentManager-Handler-3:null) SeqA 2-1802: Sending Seq 2-1802: { Ans: ,
MgmtId: 7175246184473, via: 2, Ver: v1, Flags: 100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2013-09-12 20:44:22,497 DEBUG [cloud.api.ApiServlet] (catalina-exec-14:null)
===START=== 10.252.192.100 -- GET
command=queryAsyncJobResult&jobId=3158826e-9c85-4210-bc1a-d97ce9865a3d&response=json&sessionkey=W%2F%2BBolWn%2BDoUEK1P92X71fVmknc%3D&_=1378979516168
2013-09-12 20:44:22,508 DEBUG [cloud.async.AsyncJobManagerImpl]
(catalina-exec-14:null) Async job-23 = [ 3158826e-9c85-4210-bc1a-d97ce9865a3d ]
completed
2013-09-12 20:44:22,514 DEBUG [cloud.api.ApiServlet] (catalina-exec-14:null)
===END=== 10.252.192.100 -- GET
command=queryAsyncJobResult&jobId=3158826e-9c85-4210-bc1a-d97ce9865a3d&response=json&sessionkey=W%2F%2BBolWn%2BDoUEK1P92X71fVmknc%3D&_=1378979516168
> ceph:UI:Noticed 2 records for same volume after migrating instance from one
> primary to another primary
> ------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-4652
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4652
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Management Server
> Affects Versions: 4.2.0
> Reporter: sadhu suresh
>
> 1.deploy advanced zone with kvm cluster(single cluster with ubuntu host+nfs
> as primary storage)
> 2.add RBD based primary storage
> 3.create compute offering and create a instance using this offering
> 4.stop the running VM which was created in step 3
> 5.perform migrate instance from one primary to another primary
> Actual result:
> migrate instance successful but UI shows 2 records for same Volume(newly
> created one and expunged one).it looks like volume was successfuly copied
> from primary1 to secondary and secondary to primary2 .once its copied to
> primry2 successfully and it tries to delete the volume exits in primary1
> and its fail to delete the object.due to this reason it shows 2 records for
> same volume.
> [root@cen62307 ~]# grep -i "job-21"
> /var/log/cloudstack/management/management-server.log
> 2013-09-12 19:54:55,954 DEBUG [cloud.async.AsyncJobManagerImpl]
> (catalina-exec-21:null) submit async job-21 = [
> ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ], details: AsyncJobVO {id:21, userId:
> 2, accountId: 2, sessionKey: null, instanceType: None, instanceId: null, cmd:
> org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd, cmdOriginator: null,
> cmdInfo:
> {"response":"json","sessionkey":"W/+BolWn+DoUEK1P92X71fVmknc\u003d","virtualmachineid":"42c31b97-d376-4e04-b6c3-08cecda7c13f","cmdEventType":"VM.MIGRATE","ctxUserId":"2","storageid":"59b50b13-75b2-3385-87c7-a452d56e2ee0","httpmethod":"GET","_":"1378976549445","ctxAccountId":"2","ctxStartEventId":"133"},
> cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0,
> processStatus: 0, resultCode: 0, result: null, initMsid: 7175246184473,
> completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
> 2013-09-12 19:54:56,290 DEBUG [cloud.async.AsyncJobManagerImpl]
> (Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Executing
> org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd for job-21 = [
> ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]
> 2013-09-12 19:54:56,332 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) VM state
> transitted from :Stopped to Migrating with event:
> StorageMigrationRequestedvm's original host id: 1 new host id: null host id
> before state transition: null
> 2013-09-12 19:54:56,860 DEBUG [storage.motion.AncientDataMotionStrategy]
> (Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) copyAsync
> inspecting src type VOLUME copyAsync inspecting dest type VOLUME
> 2013-09-12 19:54:56,957 DEBUG [cache.allocator.StorageCacheRandomAllocator]
> (Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Can't find
> staging storage in zone: 1
> 2013-09-12 19:54:57,691 DEBUG [agent.transport.Request]
> (Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Seq
> 1-1872036399: Sending { Cmd , MgmtId: 7175246184473, via: 1, Ver: v1, Flags:
> 100111,
> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"1b5f2f3e-6b11-43aa-9d18-aa02d7e00aa1","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"2c19704e-55d4-31aa-b0e7-a52f1fffab9e","id":2,"poolType":"RBD","host":"10.147.41.3","path":"sadhu1","port":6789}},"name":"ROOT-7","size":8589934592,"path":"48d50558-02f0-48d3-839f-bc73d119d190","volumeId":8,"vmName":"i-2-7-VM","accountId":2,"format":"RAW","id":8,"hypervisorType":"KVM"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"1b5f2f3e-6b11-43aa-9d18-aa02d7e00aa1","volumeType":"ROOT","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sadhu/asf/kvmsec","_role":"Image"}},"name":"ROOT-7","size":8589934592,"path":"volumes/2/8","volumeId":8,"vmName":"i-2-7-VM","accountId":2,"format":"RAW","id":8,"hypervisorType":"KVM"}},"executeInSequence":true,"wait":10800}}]
> }
> 2013-09-12 20:02:03,636 DEBUG [agent.transport.Request]
> (Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Seq
> 1-1872036399: Received: { Ans: , MgmtId: 7175246184473, via: 1, Ver: v1,
> Flags: 110, { CopyCmdAnswer } }
> 2013-09-12 20:02:03,759 DEBUG [agent.transport.Request]
> (Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Seq
> 1-1872036425: Sending { Cmd , MgmtId: 7175246184473, via: 1, Ver: v1, Flags:
> 100111,
> [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"1b5f2f3e-6b11-43aa-9d18-aa02d7e00aa1","volumeType":"ROOT","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sadhu/asf/kvmsec","_role":"Image"}},"name":"ROOT-7","size":8589934592,"path":"volumes/2/8/5a4a7d49-87d2-4e85-aa65-3d3afe7068e3.raw","volumeId":8,"vmName":"i-2-7-VM","accountId":2,"format":"RAW","id":8,"hypervisorType":"KVM"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"1639db5a-f6f3-47cf-9b34-191ef3bf4f75","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"59b50b13-75b2-3385-87c7-a452d56e2ee0","id":3,"poolType":"RBD","host":"10.147.41.3","path":"sadhu22","port":6789}},"name":"ROOT-7","size":8589934592,"path":"48d50558-02f0-48d3-839f-bc73d119d190","volumeId":12,"vmName":"i-2-7-VM","accountId":2,"format":"RAW","id":12,"hypervisorType":"KVM"}},"executeInSequence":true,"wait":10800}}]
> }
> 2013-09-12 20:07:24,455 DEBUG [agent.transport.Request]
> (Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Seq
> 1-1872036425: Received: { Ans: , MgmtId: 7175246184473, via: 1, Ver: v1,
> Flags: 110, { CopyCmdAnswer } }
> 2013-09-12 20:07:24,602 DEBUG [agent.transport.Request]
> (Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Seq
> 3-362741906: Sending { Cmd , MgmtId: 7175246184473, via: 3, Ver: v1, Flags:
> 100011,
> [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"1b5f2f3e-6b11-43aa-9d18-aa02d7e00aa1","volumeType":"ROOT","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sadhu/asf/kvmsec","_role":"Image"}},"name":"ROOT-7","size":8589934592,"path":"volumes/2/8/5a4a7d49-87d2-4e85-aa65-3d3afe7068e3.raw","volumeId":8,"vmName":"i-2-7-VM","accountId":2,"format":"RAW","id":8,"hypervisorType":"KVM"}},"wait":0}}]
> }
> 2013-09-12 20:07:25,268 DEBUG [agent.transport.Request]
> (Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Seq
> 3-362741906: Received: { Ans: , MgmtId: 7175246184473, via: 3, Ver: v1,
> Flags: 10, { Answer } }
> 2013-09-12 20:07:26,262 DEBUG [agent.transport.Request]
> (Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Seq
> 1-1872036442: Sending { Cmd , MgmtId: 7175246184473, via: 1, Ver: v1, Flags:
> 100011,
> [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"1b5f2f3e-6b11-43aa-9d18-aa02d7e00aa1","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"2c19704e-55d4-31aa-b0e7-a52f1fffab9e","id":2,"poolType":"RBD","host":"10.147.41.3","path":"sadhu1","port":6789}},"name":"ROOT-7","size":8589934592,"path":"48d50558-02f0-48d3-839f-bc73d119d190","volumeId":8,"vmName":"i-2-7-VM","accountId":2,"format":"RAW","id":8,"hypervisorType":"KVM"}},"wait":0}}]
> }
> 2013-09-12 20:07:26,720 DEBUG [agent.transport.Request]
> (Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Seq
> 1-1872036442: Received: { Ans: , MgmtId: 7175246184473, via: 1, Ver: v1,
> Flags: 10, { Answer } }
> 2013-09-12 20:07:26,736 WARN
> [storage.datastore.ObjectInDataStoreManagerImpl] (Job-Executor-7:job-21 = [
> ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Unsupported data object (VOLUME,
> org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@7e80dcb5), no
> need to delete from object in store ref table
> 2013-09-12 20:07:27,047 DEBUG [cloud.capacity.CapacityManagerImpl]
> (Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) VM state
> transitted from :Migrating to Stopped with event: AgentReportStoppedvm's
> original host id: null new host id: null host id before state transition: null
> 2013-09-12 20:07:27,174 DEBUG [cloud.async.AsyncJobManagerImpl]
> (Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Complete
> async job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ], jobStatus: 1,
> resultCode: 0, result:
> org.apache.cloudstack.api.response.UserVmResponse@4db78d11
> 2013-09-12 20:07:27,212 DEBUG [cloud.async.AsyncJobManagerImpl]
> (Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Done
> executing org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd for job-21
> = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]
> 2013-09-12 20:07:28,412 DEBUG [cloud.async.AsyncJobManagerImpl]
> (catalina-exec-21:null) Async job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa
> ] completed
> [root@cen62307 ~]#
> mysql> select * from volumes where instance_id=7\G;
> *************************** 1. row ***************************
> id: 8
> account_id: 2
> domain_id: 1
> pool_id: 2
> last_pool_id: NULL
> instance_id: 7
> device_id: 0
> name: ROOT-7
> uuid: 1b5f2f3e-6b11-43aa-9d18-aa02d7e00aa1
> size: 8589934592
> folder: NULL
> path: 48d50558-02f0-48d3-839f-bc73d119d190
> pod_id: NULL
> data_center_id: 1
> iscsi_name: NULL
> host_ip: NULL
> volume_type: ROOT
> pool_type: NULL
> disk_offering_id: 12
> template_id: 4
> first_snapshot_backup_uuid: NULL
> recreatable: 0
> created: 2013-09-12 11:22:10
> attached: NULL
> updated: 2013-09-12 14:37:26
> removed: NULL
> state: Expunging
> chain_info: NULL
> update_count: 8
> disk_type: NULL
> vm_snapshot_chain_size: NULL
> iso_id: 0
> display_volume: 0
> format: RAW
> min_iops: NULL
> max_iops: NULL
> *************************** 2. row ***************************
> id: 12
> account_id: 2
> domain_id: 1
> pool_id: 3
> last_pool_id: 2
> instance_id: 7
> device_id: 0
> name: ROOT-7
> uuid: 1639db5a-f6f3-47cf-9b34-191ef3bf4f75
> size: 8589934592
> folder: sadhu22
> path: ad21045a-0d7a-442a-97cf-f402cbf6c8e8
> pod_id: 1
> data_center_id: 1
> iscsi_name: NULL
> host_ip: NULL
> volume_type: ROOT
> pool_type: NULL
> disk_offering_id: 12
> template_id: 4
> first_snapshot_backup_uuid: NULL
> recreatable: 0
> created: 2013-09-12 14:24:56
> attached: NULL
> updated: 2013-09-12 14:37:25
> removed: NULL
> state: Ready
> chain_info: NULL
> update_count: 2
> disk_type: NULL
> vm_snapshot_chain_size: NULL
> iso_id: 0
> display_volume: 0
> format: RAW
> min_iops: NULL
> max_iops: NULL
> 2 rows in set (0.00 sec)
> ERROR:
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
