sadhu suresh created CLOUDSTACK-4652:
----------------------------------------
Summary: ceph:Noticed 2 records(one is in expunge state and other
is in ready state)for same volume after migrating instance from one primary to
another primary
Key: CLOUDSTACK-4652
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4652
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: Management Server
Affects Versions: 4.2.0
Reporter: sadhu suresh
1.deploy advanced zone with kvm cluster(single cluster with ubuntu host+nfs as
primary storage)
2.add RBD based primary storage
3.create compute offering and create a instance using this offering
4.stop the running VM which was created in step 3
5.perform migrate instance from one primary to another primary
Actual result:
migrate instance successful but UI shows 2 records for same Volume(newly
created one and expunged one).it looks like volume was successfuly copied from
primary1 to secondary and secondary to primary2 .once its copied to primry2
successfully and it tries to delete the volume exits in primary1 and its
fail to delete the object.due to this reason it shows 2 records for same volume.
[root@cen62307 ~]# grep -i "job-21"
/var/log/cloudstack/management/management-server.log
2013-09-12 19:54:55,954 DEBUG [cloud.async.AsyncJobManagerImpl]
(catalina-exec-21:null) submit async job-21 = [
ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ], details: AsyncJobVO {id:21, userId: 2,
accountId: 2, sessionKey: null, instanceType: None, instanceId: null, cmd:
org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd, cmdOriginator: null,
cmdInfo:
{"response":"json","sessionkey":"W/+BolWn+DoUEK1P92X71fVmknc\u003d","virtualmachineid":"42c31b97-d376-4e04-b6c3-08cecda7c13f","cmdEventType":"VM.MIGRATE","ctxUserId":"2","storageid":"59b50b13-75b2-3385-87c7-a452d56e2ee0","httpmethod":"GET","_":"1378976549445","ctxAccountId":"2","ctxStartEventId":"133"},
cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0,
processStatus: 0, resultCode: 0, result: null, initMsid: 7175246184473,
completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
2013-09-12 19:54:56,290 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Executing
org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd for job-21 = [
ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]
2013-09-12 19:54:56,332 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) VM state
transitted from :Stopped to Migrating with event: StorageMigrationRequestedvm's
original host id: 1 new host id: null host id before state transition: null
2013-09-12 19:54:56,860 DEBUG [storage.motion.AncientDataMotionStrategy]
(Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) copyAsync
inspecting src type VOLUME copyAsync inspecting dest type VOLUME
2013-09-12 19:54:56,957 DEBUG [cache.allocator.StorageCacheRandomAllocator]
(Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Can't find
staging storage in zone: 1
2013-09-12 19:54:57,691 DEBUG [agent.transport.Request] (Job-Executor-7:job-21
= [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Seq 1-1872036399: Sending { Cmd ,
MgmtId: 7175246184473, via: 1, Ver: v1, Flags: 100111,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"1b5f2f3e-6b11-43aa-9d18-aa02d7e00aa1","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"2c19704e-55d4-31aa-b0e7-a52f1fffab9e","id":2,"poolType":"RBD","host":"10.147.41.3","path":"sadhu1","port":6789}},"name":"ROOT-7","size":8589934592,"path":"48d50558-02f0-48d3-839f-bc73d119d190","volumeId":8,"vmName":"i-2-7-VM","accountId":2,"format":"RAW","id":8,"hypervisorType":"KVM"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"1b5f2f3e-6b11-43aa-9d18-aa02d7e00aa1","volumeType":"ROOT","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sadhu/asf/kvmsec","_role":"Image"}},"name":"ROOT-7","size":8589934592,"path":"volumes/2/8","volumeId":8,"vmName":"i-2-7-VM","accountId":2,"format":"RAW","id":8,"hypervisorType":"KVM"}},"executeInSequence":true,"wait":10800}}]
}
2013-09-12 20:02:03,636 DEBUG [agent.transport.Request] (Job-Executor-7:job-21
= [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Seq 1-1872036399: Received: { Ans:
, MgmtId: 7175246184473, via: 1, Ver: v1, Flags: 110, { CopyCmdAnswer } }
2013-09-12 20:02:03,759 DEBUG [agent.transport.Request] (Job-Executor-7:job-21
= [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Seq 1-1872036425: Sending { Cmd ,
MgmtId: 7175246184473, via: 1, Ver: v1, Flags: 100111,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"1b5f2f3e-6b11-43aa-9d18-aa02d7e00aa1","volumeType":"ROOT","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sadhu/asf/kvmsec","_role":"Image"}},"name":"ROOT-7","size":8589934592,"path":"volumes/2/8/5a4a7d49-87d2-4e85-aa65-3d3afe7068e3.raw","volumeId":8,"vmName":"i-2-7-VM","accountId":2,"format":"RAW","id":8,"hypervisorType":"KVM"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"1639db5a-f6f3-47cf-9b34-191ef3bf4f75","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"59b50b13-75b2-3385-87c7-a452d56e2ee0","id":3,"poolType":"RBD","host":"10.147.41.3","path":"sadhu22","port":6789}},"name":"ROOT-7","size":8589934592,"path":"48d50558-02f0-48d3-839f-bc73d119d190","volumeId":12,"vmName":"i-2-7-VM","accountId":2,"format":"RAW","id":12,"hypervisorType":"KVM"}},"executeInSequence":true,"wait":10800}}]
}
2013-09-12 20:07:24,455 DEBUG [agent.transport.Request] (Job-Executor-7:job-21
= [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Seq 1-1872036425: Received: { Ans:
, MgmtId: 7175246184473, via: 1, Ver: v1, Flags: 110, { CopyCmdAnswer } }
2013-09-12 20:07:24,602 DEBUG [agent.transport.Request] (Job-Executor-7:job-21
= [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Seq 3-362741906: Sending { Cmd ,
MgmtId: 7175246184473, via: 3, Ver: v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"1b5f2f3e-6b11-43aa-9d18-aa02d7e00aa1","volumeType":"ROOT","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sadhu/asf/kvmsec","_role":"Image"}},"name":"ROOT-7","size":8589934592,"path":"volumes/2/8/5a4a7d49-87d2-4e85-aa65-3d3afe7068e3.raw","volumeId":8,"vmName":"i-2-7-VM","accountId":2,"format":"RAW","id":8,"hypervisorType":"KVM"}},"wait":0}}]
}
2013-09-12 20:07:25,268 DEBUG [agent.transport.Request] (Job-Executor-7:job-21
= [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Seq 3-362741906: Received: { Ans:
, MgmtId: 7175246184473, via: 3, Ver: v1, Flags: 10, { Answer } }
2013-09-12 20:07:26,262 DEBUG [agent.transport.Request] (Job-Executor-7:job-21
= [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Seq 1-1872036442: Sending { Cmd ,
MgmtId: 7175246184473, via: 1, Ver: v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"1b5f2f3e-6b11-43aa-9d18-aa02d7e00aa1","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"2c19704e-55d4-31aa-b0e7-a52f1fffab9e","id":2,"poolType":"RBD","host":"10.147.41.3","path":"sadhu1","port":6789}},"name":"ROOT-7","size":8589934592,"path":"48d50558-02f0-48d3-839f-bc73d119d190","volumeId":8,"vmName":"i-2-7-VM","accountId":2,"format":"RAW","id":8,"hypervisorType":"KVM"}},"wait":0}}]
}
2013-09-12 20:07:26,720 DEBUG [agent.transport.Request] (Job-Executor-7:job-21
= [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Seq 1-1872036442: Received: { Ans:
, MgmtId: 7175246184473, via: 1, Ver: v1, Flags: 10, { Answer } }
2013-09-12 20:07:26,736 WARN [storage.datastore.ObjectInDataStoreManagerImpl]
(Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Unsupported
data object (VOLUME,
org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@7e80dcb5), no need
to delete from object in store ref table
2013-09-12 20:07:27,047 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) VM state
transitted from :Migrating to Stopped with event: AgentReportStoppedvm's
original host id: null new host id: null host id before state transition: null
2013-09-12 20:07:27,174 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Complete
async job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ], jobStatus: 1,
resultCode: 0, result:
org.apache.cloudstack.api.response.UserVmResponse@4db78d11
2013-09-12 20:07:27,212 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-7:job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]) Done
executing org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd for job-21 =
[ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]
2013-09-12 20:07:28,412 DEBUG [cloud.async.AsyncJobManagerImpl]
(catalina-exec-21:null) Async job-21 = [ ca7b08f2-a7a7-4797-8ed7-9c11c15e34aa ]
completed
[root@cen62307 ~]#
mysql> select * from volumes where instance_id=7\G;
*************************** 1. row ***************************
id: 8
account_id: 2
domain_id: 1
pool_id: 2
last_pool_id: NULL
instance_id: 7
device_id: 0
name: ROOT-7
uuid: 1b5f2f3e-6b11-43aa-9d18-aa02d7e00aa1
size: 8589934592
folder: NULL
path: 48d50558-02f0-48d3-839f-bc73d119d190
pod_id: NULL
data_center_id: 1
iscsi_name: NULL
host_ip: NULL
volume_type: ROOT
pool_type: NULL
disk_offering_id: 12
template_id: 4
first_snapshot_backup_uuid: NULL
recreatable: 0
created: 2013-09-12 11:22:10
attached: NULL
updated: 2013-09-12 14:37:26
removed: NULL
state: Expunging
chain_info: NULL
update_count: 8
disk_type: NULL
vm_snapshot_chain_size: NULL
iso_id: 0
display_volume: 0
format: RAW
min_iops: NULL
max_iops: NULL
*************************** 2. row ***************************
id: 12
account_id: 2
domain_id: 1
pool_id: 3
last_pool_id: 2
instance_id: 7
device_id: 0
name: ROOT-7
uuid: 1639db5a-f6f3-47cf-9b34-191ef3bf4f75
size: 8589934592
folder: sadhu22
path: ad21045a-0d7a-442a-97cf-f402cbf6c8e8
pod_id: 1
data_center_id: 1
iscsi_name: NULL
host_ip: NULL
volume_type: ROOT
pool_type: NULL
disk_offering_id: 12
template_id: 4
first_snapshot_backup_uuid: NULL
recreatable: 0
created: 2013-09-12 14:24:56
attached: NULL
updated: 2013-09-12 14:37:25
removed: NULL
state: Ready
chain_info: NULL
update_count: 2
disk_type: NULL
vm_snapshot_chain_size: NULL
iso_id: 0
display_volume: 0
format: RAW
min_iops: NULL
max_iops: NULL
2 rows in set (0.00 sec)
ERROR:
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
