Snapshots work fine. I can make a snapshot -> convert it to template
and start the VM on a new node from that template. When I needed to
move one VM for balance purposes. But I want to fix the migration
process. I have attached the error log to this email. Maybe I'm
looking at the wrong place were I get the error?
On Wed, May 9, 2018 at 9:23 PM, Marc-Aurèle Brothier <ma...@exoscale.ch> wrote:
> Can you try to perform a snapshot of the volume on VM's that are on your
> host, to see if they get copied correctly other the NFS too.
>
> Otherwise you need to look into the management logs to catch the exception
> (stack trace) to have a better understanding of the issue.
>
> On Wed, May 9, 2018 at 1:58 PM, Daznis <daz...@gmail.com> wrote:
>
>> Hello,
>>
>>
>> Yeah it's offline. I'm running 4.9.2 version. Running it on the same
>> zone with the only NFS secondary storage.
>>
>> On Wed, May 9, 2018 at 10:49 AM, Marc-Aurèle Brothier <ma...@exoscale.ch>
>> wrote:
>> > Hi Darius,
>> >
>> > Are you trying to perform an offline migration within the same zone,
>> > meaning that the source and destination hosts have the same set of NFS
>> > secondary storage ?
>> >
>> > Marc-Aurèle
>> >
>> > On Tue, May 8, 2018 at 3:37 PM, Daznis <daz...@gmail.com> wrote:
>> >
>> >> Hi,
>> >>
>> >>
>> >> I'm having an issue while migrating offline vm disk within local
>> >> storages. The particular error that has be baffled is "Can't find
>> >> staging storage in zone". From what I have gather "staging storage"
>> >> referred to secondary storage in cloudstack and it's working perfectly
>> >> fine with both the source and destination node. Not sure where to go
>> >> next. Any help would be appreciated.
>> >>
>> >>
>> >> Regards,
>> >> Darius
>> >>
>>
2018-05-10 09:41:38,555 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149) (logid:132e1fe7) Run VM
work job: com.cloud.vm.VmWorkStorageMigration for VM 19, job origin: 2148
2018-05-10 09:41:38,556 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149 ctx-cfe56a50)
(logid:132e1fe7) Execute VM work job:
com.cloud.vm.VmWorkStorageMigration{"destPoolId":1,"userId":2,"accountId":2,"vmId":19,"handl
erName":"VirtualMachineManagerImpl"}
2018-05-10 09:41:38,569 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149 ctx-cfe56a50)
(logid:132e1fe7) VM state transitted from :Stopped to Migrating with event:
StorageMigrationRequestedvm's original host id: 5 new ho
st id: null host id before state transition: null
2018-05-10 09:41:38,594 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149 ctx-cfe56a50)
(logid:132e1fe7) copyAsync inspecting src type VOLUME copyAsync inspecting dest
type VOLUME
2018-05-10 09:41:38,598 DEBUG [o.a.c.s.c.a.StorageCacheRandomAllocator]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149 ctx-cfe56a50)
(logid:132e1fe7) Can't find staging storage in zone: 2
2018-05-10 09:41:38,608 DEBUG [o.a.c.s.v.VolumeObject]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149 ctx-cfe56a50)
(logid:132e1fe7) Failed to update state
com.cloud.utils.fsm.NoTransitionException: Unable to transition to a new state
from Creating via CreateOnlyRequested
at
com.cloud.utils.fsm.StateMachine2.getTransition(StateMachine2.java:108)
at com.cloud.utils.fsm.StateMachine2.getNextState(StateMachine2.java:94)
at com.cloud.utils.fsm.StateMachine2.transitTo(StateMachine2.java:124)
at
org.apache.cloudstack.storage.datastore.ObjectInDataStoreManagerImpl.update(ObjectInDataStoreManagerImpl.java:307)
at
org.apache.cloudstack.storage.volume.VolumeObject.processEvent(VolumeObject.java:292)
at
org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyVolumeBetweenPools(AncientDataMotionStrategy.java:317)
at
org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:440)
at
org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:68)
at
org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:73)
at
org.apache.cloudstack.storage.volume.VolumeServiceImpl.copyVolume(VolumeServiceImpl.java:1372)
at
org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.migrateVolume(VolumeOrchestrator.java:952)
at
org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.storageMigration(VolumeOrchestrator.java:1056)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStorageMigration(VirtualMachineManagerImpl.java:1791)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStorageMigration(VirtualMachineManagerImpl.java:5073)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5080)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:554)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:502)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-05-10 09:41:38,609 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149 ctx-cfe56a50)
(logid:132e1fe7) copy failed
com.cloud.utils.exception.CloudRuntimeException: Failed to update
state:com.cloud.utils.fsm.NoTransitionException: Unable to transition to a new
state from Creating via CreateOnlyRequested
at
org.apache.cloudstack.storage.volume.VolumeObject.processEvent(VolumeObject.java:332)
at
org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyVolumeBetweenPools(AncientDataMotionStrategy.java:317)
at
org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:440)
at
org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:68)
at
org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:73)
at
org.apache.cloudstack.storage.volume.VolumeServiceImpl.copyVolume(VolumeServiceImpl.java:1372)
at
org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.migrateVolume(VolumeOrchestrator.java:952)
at
org.apache.cloudstack.engine.orchestration.VolumeOrchestrator.storageMigration(VolumeOrchestrator.java:1056)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStorageMigration(VirtualMachineManagerImpl.java:1791)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStorageMigration(VirtualMachineManagerImpl.java:5073)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5080)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:554)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:502)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-05-10 09:41:38,622 WARN [o.a.c.s.d.ObjectInDataStoreManagerImpl]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149 ctx-cfe56a50)
(logid:132e1fe7) Unsupported data object (VOLUME,
org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@47384c14), no need
to delete from object in store ref table
2018-05-10 09:41:38,645 DEBUG [o.a.c.s.v.VolumeServiceImpl]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149 ctx-cfe56a50)
(logid:132e1fe7) Marking volume that was never created as destroyed:
Vol[70|vm=null|ROOT]
2018-05-10 09:41:38,650 ERROR [o.a.c.e.o.VolumeOrchestrator]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149 ctx-cfe56a50)
(logid:132e1fe7) Migrate volume
failed:com.cloud.utils.exception.CloudRuntimeException: Failed to update
state:com.cloud.utils.fsm.NoTransitionException: Unable to transition to a new
state from Creating via CreateOnlyRequested
2018-05-10 09:41:38,651 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149 ctx-cfe56a50)
(logid:132e1fe7) Failed to migration:
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is
unreachable: Migrate volume failed:
com.cloud.utils.exception.CloudRuntimeException: Failed to update
state:com.cloud.utils.fsm.NoTransitionException: Unable to transition to a new
state from Creating via CreateOnlyRequested
2018-05-10 09:41:38,657 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149 ctx-cfe56a50)
(logid:132e1fe7) VM state transitted from :Migrating to Stopped with event:
AgentReportStoppedvm's original host id: 5 new host id: null host id before
state transition: null
2018-05-10 09:41:38,663 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149 ctx-cfe56a50)
(logid:132e1fe7) Hosts's actual total CPU: 57600 and CPU after applying
overprovisioning: 1728000
2018-05-10 09:41:38,663 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149 ctx-cfe56a50)
(logid:132e1fe7) Hosts's actual total RAM: 66299768832 and RAM after applying
overprovisioning: 66299768832
2018-05-10 09:41:38,663 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149 ctx-cfe56a50)
(logid:132e1fe7) release cpu from host: 5, old used: 66800,reserved: 7600,
actual total: 57600, total with overprovisioning: 1728000; new used:
59200,reserved:7600; movedfromreserved: false,moveToReserveredfalse
2018-05-10 09:41:38,663 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149 ctx-cfe56a50)
(logid:132e1fe7) release mem from host: 5, old used: 55834574848,reserved:
4294967296, total: 66299768832; new used: 51539607552,reserved:4294967296;
movedfromreserved: false,moveToReserveredfalse
2018-05-10 09:41:38,674 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149 ctx-cfe56a50)
(logid:132e1fe7) Invocation exception, caused by:
com.cloud.utils.exception.CloudRuntimeException: Failed to migration:
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is
unreachable: Migrate volume failed:
com.cloud.utils.exception.CloudRuntimeException: Failed to update
state:com.cloud.utils.fsm.NoTransitionException: Unable to transition to a new
state from Creating via CreateOnlyRequested
2018-05-10 09:41:38,674 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149 ctx-cfe56a50)
(logid:132e1fe7) Rethrow exception
com.cloud.utils.exception.CloudRuntimeException: Failed to migration:
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is
unreachable: Migrate volume failed:
com.cloud.utils.exception.CloudRuntimeException: Failed to update
state:com.cloud.utils.fsm.NoTransitionException: Unable to transition to a new
state from Creating via CreateOnlyRequested
2018-05-10 09:41:38,674 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149) (logid:132e1fe7) Done with
run of VM work job: com.cloud.vm.VmWorkStorageMigration for VM 19, job origin:
2148
2018-05-10 09:41:38,674 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149) (logid:132e1fe7) Unable to
complete AsyncJobVO {id:2149, userId: 2, accountId: 2, instanceType: null,
instanceId: null, cmd: com.cloud.vm.VmWorkStorageMigration, cmdInfo:
rO0ABXNyACNjb20uY2xvdWQudm0uVm1Xb3JrU3RvcmFnZU1pZ3JhdGlvboeRn3LBtueeAgABTAAKZGVzdFBvb2xJZHQAEExqYXZhL2xhbmcvTG9uZzt4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1ldAASTGphdmEvbGFuZy9TdHJpbmc7eHAAAAAAAAAAAgAAAAAAAAACAAAAAAAAABN0ABlWaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsc3IADmphdmEubGFuZy5Mb25nO4vkkMyPI98CAAFKAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAAAAAAAAB,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 159304575802479, completeMsid: null, lastUpdated: null,
lastPolled: null, created: Thu May 10 09:41:38 SAST 2018}, job origin:2148
com.cloud.utils.exception.CloudRuntimeException: Failed to migration:
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is
unreachable: Migrate volume failed:
com.cloud.utils.exception.CloudRuntimeException: Failed to update
state:com.cloud.utils.fsm.NoTransitionException: Unable to transition to a new
state from Creating via CreateOnlyRequested
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStorageMigration(VirtualMachineManagerImpl.java:1845)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStorageMigration(VirtualMachineManagerImpl.java:5073)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5080)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:554)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:502)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-05-10 09:41:38,682 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149) (logid:132e1fe7) Complete
async job-2149, jobStatus: FAILED, resultCode: 0, result:
rO0ABXNyAC9jb20uY2xvdWQudXRpbHMuZXhjZXB0aW9uLkNsb3VkUnVudGltZUV4Y2VwdGlvbgAAAABWTU9yAwABSQALY3NFcnJvckNvZGV4cgAaamF2YS5sYW5nLlJ1bnRpbWVFeGNlcHRpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0aW9u0P0fPho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0ABVMamF2YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJpbmc7WwAKc3RhY2tUcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHByZXNzZWRFeGNlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0AUJGYWlsZWQgdG8gbWlncmF0aW9uOiBjb20uY2xvdWQuZXhjZXB0aW9uLlN0b3JhZ2VVbmF2YWlsYWJsZUV4Y2VwdGlvbjogUmVzb3VyY2UgW1N0b3JhZ2VQb29sOjFdIGlzIHVucmVhY2hhYmxlOiBNaWdyYXRlIHZvbHVtZSBmYWlsZWQ6IGNvbS5jbG91ZC51dGlscy5leGNlcHRpb24uQ2xvdWRSdW50aW1lRXhjZXB0aW9uOiBGYWlsZWQgdG8gdXBkYXRlIHN0YXRlOmNvbS5jbG91ZC51dGlscy5mc20uTm9UcmFuc2l0aW9uRXhjZXB0aW9uOiBVbmFibGUgdG8gdHJhbnNpdGlvbiB0byBhIG5ldyBzdGF0ZSBmcm9tIENyZWF0aW5nIHZpYSBDcmVhdGVPbmx5UmVxdWVzdGVkdXIAHltMamF2YS5sYW5nLlN0YWNrVHJhY2VFbGVtZW50OwJGKjw8_SI5AgAAeHAAAAAVc3IAG2phdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudGEJxZomNt2FAgAESQAKbGluZU51bWJlckwADmRlY2xhcmluZ0NsYXNzcQB-AAVMAAhmaWxlTmFtZXEAfgAFTAAKbWV0aG9kTmFtZXEAfgAFeHAAAAc1dAAmY29tLmNsb3VkLnZtLlZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGx0AB5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsLmphdmF0ABtvcmNoZXN0cmF0ZVN0b3JhZ2VNaWdyYXRpb25zcQB-AAwAABPRcQB-AA5xAH4AD3EAfgAQc3EAfgAM_____nQAJHN1bi5yZWZsZWN0Lk5hdGl2ZU1ldGhvZEFjY2Vzc29ySW1wbHQAHU5hdGl2ZU1ldGhvZEFjY2Vzc29ySW1wbC5qYXZhdAAHaW52b2tlMHNxAH4ADAAAAD5xAH4AE3EAfgAUdAAGaW52b2tlc3EAfgAMAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AF3NxAH4ADAAAAfJ0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAXc3EAfgAMAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4ADAAAE9hxAH4ADnEAfgAPcQB-ACFzcQB-AAwAAABmdAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-AAwAAAIqdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgAMAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgAMAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4ADAAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADF0AA9jYWxsV2l0aENvbnRleHRzcQB-AAwAAAA1cQB-ADRxAH4AMXQADnJ1bldpdGhDb250ZXh0c3EAfgAMAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAtcQB-AC5zcQB-AAwAAAH2cQB-AChxAH4AKXEAfgAuc3EAfgAMAAAB_3QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgAyc3EAfgAMAAABCnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4ALnNxAH4ADAAABH10ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4ADAAAAnB0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AENxAH4ALnNxAH4ADAAAAux0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4ALnNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAHeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AE94AAAQmncIAAAAAAAAAAB4
2018-05-10 09:41:38,683 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149) (logid:132e1fe7) Publish
async job-2149 complete on message bus
2018-05-10 09:41:38,683 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149) (logid:132e1fe7) Wake up
jobs related to job-2149
2018-05-10 09:41:38,683 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149) (logid:132e1fe7) Update db
status for job-2149
2018-05-10 09:41:38,685 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149) (logid:132e1fe7) Wake up
jobs joined with job-2149 and disjoin all subjobs created from job- 2149
2018-05-10 09:41:38,691 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149) (logid:132e1fe7) Done
executing com.cloud.vm.VmWorkStorageMigration for job-2149
2018-05-10 09:41:38,693 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-3:ctx-69344343 job-2148/job-2149) (logid:132e1fe7) Remove
job-2149 from job monitoring
2018-05-10 09:41:38,703 ERROR [c.c.a.ApiAsyncJobDispatcher]
(API-Job-Executor-3:ctx-64fdce4d job-2148) (logid:132e1fe7) Unexpected
exception while executing
org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd
com.cloud.utils.exception.CloudRuntimeException: Failed to migration:
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is
unreachable: Migrate volume failed:
com.cloud.utils.exception.CloudRuntimeException: Failed to update
state:com.cloud.utils.fsm.NoTransitionException: Unable to transition to a new
state from Creating via CreateOnlyRequested
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStorageMigration(VirtualMachineManagerImpl.java:1845)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStorageMigration(VirtualMachineManagerImpl.java:5073)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5080)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:554)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:502)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-05-10 09:41:38,704 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-3:ctx-64fdce4d job-2148) (logid:132e1fe7) Complete async
job-2148, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Failed
to migration: com.cloud.exception.StorageUnavailableException: Resource
[StoragePool:1] is unreachable: Migrate volume failed:
com.cloud.utils.exception.CloudRuntimeException: Failed to update
state:com.cloud.utils.fsm.NoTransitionException: Unable to transition to a new
state from Creating via CreateOnlyRequested"}
