Likitha Shetty created CLOUDSTACK-8118:
------------------------------------------
Summary: Root volume migration fails with 'No such disk device' in
case of vCenter 5.5 setup
Key: CLOUDSTACK-8118
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-8118
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Affects Versions: 4.5.0
Reporter: Likitha Shetty
Assignee: Likitha Shetty
Priority: Critical
Fix For: Future
+Steps to reproduce+
1. Setup CS with vCenter 5.5.
2. Deploy a VM in cluster. Say ROOT volume is in primary storage P1
3. Add another primary storage to the same cluster (P2).
4. Migrate ROOT volume to primary storage P2.
Migration fails with -
{noformat}
2014-12-24 15:13:29,299 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-325:ctx-0dbf332a) Seq 5-6870804181507113730: Executing request
2014-11-24 15:13:29,299 INFO [c.c.h.v.r.VmwareResource]
(DirectAgent-325:ctx-0dbf332a 10.147.40.14, job-760/job-761, cmd:
MigrateVolumeCommand) Executing resource Mi grateVolumeCommand:
{"volumeId":106,"volumePath":"ROOT-40-000001","pool":
{"id":8,"uuid":"251a42e1-7316-30a8-ad11-bf03421f39f2","host":"10.147.28.7","path":"/export/home/manasa/priVMw2","port":2049,"type":"NetworkFilesystem"}
,"attachedVmName":"i-2-40-VM","wait":3600}
2014-12-24 15:13:29,422 INFO [c.c.h.v.m.VirtualMachineMO]
(DirectAgent-325:ctx-0dbf332a 10.147.40.14, job-760/job-761, cmd:
MigrateVolumeCommand) Look for disk devic e info from volume :
ROOT-40-000001.vmdk with trimmed base name: ROOT-40
2014-12-24 15:13:29,422 INFO [c.c.h.v.m.VirtualMachineMO]
(DirectAgent-325:ctx-0dbf332a 10.147.40.14, job-760/job-761, cmd:
MigrateVolumeCommand) Test against disk d evice, controller key: 1000, unit
number: 0
2014-11-24 15:13:29,422 INFO [c.c.h.v.m.VirtualMachineMO]
(DirectAgent-325:ctx-0dbf332a 10.147.40.14, job-760/job-761, cmd:
MigrateVolumeCommand) Test against disk b acking :
[f159abf3a6863ff1ad6a24d59100bdb9] vm-test/ROOT-40-000001.vmdk
2014-12-24 15:13:29,422 ERROR [c.c.h.v.r.VmwareResource]
(DirectAgent-325:ctx-0dbf332a 10.147.40.14, job-760/job-761, cmd:
MigrateVolumeCommand) Catch Exception java. lang.Exception due to
java.lang.Exception: No such disk device: ROOT-40-000001.vmdk
java.lang.Exception: No such disk device: ROOT-40-000001.vmdk
at
com.cloud.hypervisor.vmware.resource.VmwareResource.getVirtualDiskInfo(VmwareResource.java:3366)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:3319)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:403)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2014-11-24 15:13:29,424 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-325:ctx-0dbf332a) Seq 5-6870804181507113730: Response Received:
2014-11-24 15:13:29,425 DEBUG [c.c.a.t.Request] (DirectAgent-325:ctx-0dbf332a)
Seq 5-6870804181507113730: Processing: { Ans: , MgmtId: 6662467354709, via: 5,
Ver: v1 , Flags: 110,
[{"com.cloud.agent.api.storage.MigrateVolumeAnswer":{"result":false,"details":"Catch
Exception java.lang.Exception due to java.lang.Exception: No such d isk
device: ROOT-40-000001.vmdk","wait":0}}] }
2014-12-24 15:13:29,425 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-151:ctx-1199aeb3 job-760/job-761 ctx-63b18747) Seq
5-6870804181507113730: Received: { Ans: , MgmtI d: 6662467354709, via: 5, Ver:
v1, Flags: 110,
{ MigrateVolumeAnswer }
}
2014-11-24 15:13:29,425 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
(Work-Job-Executor-151:ctx-1199aeb3 job-760/job-761 ctx-63b18747) copy failed
com.cloud.utils.exception.CloudRuntimeException: Failed to migrate volume
org.apache.cloudstack.storage.volume.VolumeObject@58ac8d1b to storage pool
org.apache.clouds tack.storage.datastore.PrimaryDataStoreImpl@28641518
at
org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.migrateVolumeToPool(AncientDataMotionStrategy.java:390)
at
org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:430)
at
org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:68)
at
org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:73)
at
org.apache.cloudstack.storage.volume.VolumeServiceImpl.migrateVolume(VolumeServiceImpl.java:1081)
at
com.cloud.storage.VolumeApiServiceImpl.liveMigrateVolume(VolumeApiServiceImpl.java:1835)
at
com.cloud.storage.VolumeApiServiceImpl.orchestrateMigrateVolume(VolumeApiServiceImpl.java:1821)
at
com.cloud.storage.VolumeApiServiceImpl.orchestrateMigrateVolume(VolumeApiServiceImpl.java:2611)
at sun.reflect.GeneratedMethodAccessor900.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.storage.VolumeApiServiceImpl.handleVmWorkJob(VolumeApiServiceImpl.java:2627)
at sun.reflect.GeneratedMethodAccessor633.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy185.handleVmWorkJob(Unknown Source)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:103)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:536)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:493)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2014-11-24 15:13:29,427 DEBUG [c.c.a.m.AgentAttache]
(DirectAgent-325:ctx-0dbf332a) Seq 5-6870804181507113730: No more commands found
2014-12-24 15:13:29,480 WARN [o.a.c.s.d.ObjectInDataStoreManagerImpl]
(Work-Job-Executor-151:ctx-1199aeb3 job-760/job-761 ctx-63b18747) Unsupported
data object (VOLU ME,
org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@5fadb7eb), no need
to delete from object in store ref table
2014-12-24 15:13:29,481 DEBUG [c.c.s.VolumeApiServiceImpl]
(Work-Job-Executor-151:ctx-1199aeb3 job-760/job-761 ctx-63b18747) migrate
volume failed:com.cloud.utils.exc eption.CloudRuntimeException: Failed to
migrate volume org.apache.cloudstack.storage.volume.VolumeObject@58ac8d1b to
storage pool org.apache.cloudstack.storage.datast
ore.PrimaryDataStoreImpl@28641518
2014-11-24 15:13:29,481 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-151:ctx-1199aeb3 job-760/job-761 ctx-63b18747) Invocation
exception, caused by: java.la ng.NullPointerException
2014-12-24 15:13:29,481 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-151:ctx-1199aeb3 job-760/job-761 ctx-63b18747) Rethrow
exception java.lang.NullPointerE xception
2014-12-24 15:13:29,481 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-151:ctx-1199aeb3 job-760/job-761) Done with run of VM work
job: com.cloud.vm.VmWorkMigrat eVolume for VM 40, job origin: 760
2014-12-24 15:13:29,481 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-151:ctx-1199aeb3 job-760/job-761) Unable to complete
AsyncJobVO
{id:761, userId: 2, accou ntId: 2, instanceType: null, instanceId: null, cmd:
com.cloud.vm.VmWorkMigrateVolume, cmdInfo:
rO0ABXNyACBjb20uY2xvdWQudm0uVm1Xb3JrTWlncmF0ZVZvbHVtZfgl82-871PmAgADSgA
KZGVzdFBvb2xJZFoAC2xpdmVNaWdyYXRlSgAIdm9sdW1lSWR4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1ldAASTGphdmEvbGFuZ
y9TdHJpbmc7eHAAAAAAAAAAAgAAAAAAAAACAAAAAAAAACh0ABRWb2x1bWVBcGlTZXJ2aWNlSW1wbAAAAAAAAAAIAQAAAAAAAABq,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 6662467354709, completeMsid: null, lastUpdated: null,
lastPolled: null, created: Mon Nov 24 15:13:27 IST 2014}
, job origin:760
java.lang.NullPointerException
at
com.cloud.storage.VolumeApiServiceImpl.orchestrateMigrateVolume(VolumeApiServiceImpl.java:2612)
at sun.reflect.GeneratedMethodAccessor900.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.storage.VolumeApiServiceImpl.handleVmWorkJob(VolumeApiServiceImpl.java:2627)
at sun.reflect.GeneratedMethodAccessor633.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy185.handleVmWorkJob(Unknown Source)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:103)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:536)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
