Re: [Users] oVirt 3.1 - VM Migration Issue
-7c7f48b60adf`::Failed to migrate Traceback (most recent call last): File /usr/share/vdsm/vm.py, line 245, in run self._startUnderlyingMigration() File /usr/share/vdsm/libvirtvm.py, line 474, in _startUnderlyingMigration None, maxBandwidth) File /usr/share/vdsm/libvirtvm.py, line 510, in f ret = attr(*args, **kwargs) File /usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py, line 83, in wrapper ret = f(*args, **kwargs) File /usr/lib64/python2.6/site-packages/libvirt.py, line 1103, in migrateToURI2 if ret == -1: raise libvirtError ('virDomainMigrateToURI2() failed', dom=self) libvirtError: internal error Process exited while reading console log output: any chance you attach libvirtd.log and qemu log (/var/log/libvirt/qemu/{}.log? Danken - any insights? - Original Message - From: Tom Brown t...@ng23.net To: Roy Golan rgo...@redhat.com Cc: Haim Ateya hat...@redhat.com, users@ovirt.org Sent: Tuesday, January 8, 2013 11:50:26 AM Subject: Re: [Users] oVirt 3.1 - VM Migration Issue can you attach the same snip from the src VDSM 10.192.42.196 as well? The log is pretty chatty therefore i did another migration attempt and snipd'd the new log from both sides. see attached ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] oVirt 3.1 - VM Migration Issue
libvirtError: internal error Process exited while reading console log outpu could this be related to selinux? can you try disabling it and see if migration succeeds? It was indeed the case! my src node was set to disabled and my destination node was enforcing, this was due to the destination being the first HV built and therefore provisioned slightly differently, my kickstart server is a VM in the pool. Its interesting that a VM can be provisioned onto a node that is set to enforcing and yet not migrated to. thanks ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] oVirt 3.1 - VM Migration Issue
- Original Message - From: Dan Kenigsberg dan...@redhat.com To: Tom Brown t...@ng23.net Cc: Simon Grinberg sgrin...@redhat.com, users@ovirt.org Sent: Wednesday, January 9, 2013 2:11:14 PM Subject: Re: [Users] oVirt 3.1 - VM Migration Issue On Wed, Jan 09, 2013 at 10:06:12AM +, Tom Brown wrote: libvirtError: internal error Process exited while reading console log outpu could this be related to selinux? can you try disabling it and see if migration succeeds? It was indeed the case! my src node was set to disabled and my destination node was enforcing, this was due to the destination being the first HV built and therefore provisioned slightly differently, my kickstart server is a VM in the pool. Its interesting that a VM can be provisioned onto a node that is set to enforcing and yet not migrated to. I have (only a vague) memory of discussing this already... Shouldn't oVirt-Engine be aware of selinux enforcement? If a cluster has disabled hosts, an enforcing host should not be operational (or at least warn the admin about that). I recall something like that, but I don't recall we ever converged and can't find the thread ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] oVirt 3.1 - VM Migration Issue
- Original Message - From: Dan Kenigsberg dan...@redhat.com To: Simon Grinberg si...@redhat.com Cc: users@ovirt.org, Tom Brown t...@ng23.net Sent: Wednesday, January 9, 2013 6:20:02 PM Subject: Re: [Users] oVirt 3.1 - VM Migration Issue On Wed, Jan 09, 2013 at 09:05:37AM -0500, Simon Grinberg wrote: - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Tom Brown t...@ng23.net Cc: Simon Grinberg sgrin...@redhat.com, users@ovirt.org Sent: Wednesday, January 9, 2013 2:11:14 PM Subject: Re: [Users] oVirt 3.1 - VM Migration Issue On Wed, Jan 09, 2013 at 10:06:12AM +, Tom Brown wrote: libvirtError: internal error Process exited while reading console log outpu could this be related to selinux? can you try disabling it and see if migration succeeds? It was indeed the case! my src node was set to disabled and my destination node was enforcing, this was due to the destination being the first HV built and therefore provisioned slightly differently, my kickstart server is a VM in the pool. Its interesting that a VM can be provisioned onto a node that is set to enforcing and yet not migrated to. I have (only a vague) memory of discussing this already... Shouldn't oVirt-Engine be aware of selinux enforcement? If a cluster has disabled hosts, an enforcing host should not be operational (or at least warn the admin about that). I recall something like that, but I don't recall we ever converged and can't find the thread What is your opinion on the subject? I think that at the least, the scheduler must be aware of selinux enforcement when it chooses migration destination. Either all or non in the same cluster - that is the default. On a mixed environment, the non enforced hosts should be move to non-operational, but VM should not be migrated off due to this, we don't want them moved to protected hosts without the admin awareness. As exception to the above, have a config parameter that allows in a mixed environment to migrate VMs from an insecure onto a secure host never the other way around. This is to support transition from non-enabled system to enabled. I think this is the closest I can get to the agreement (or at least concerns) raised in that old thread I can't find. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] oVirt 3.1 - VM Migration Issue
On Wed, Jan 09, 2013 at 09:05:37AM -0500, Simon Grinberg wrote: - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Tom Brown t...@ng23.net Cc: Simon Grinberg sgrin...@redhat.com, users@ovirt.org Sent: Wednesday, January 9, 2013 2:11:14 PM Subject: Re: [Users] oVirt 3.1 - VM Migration Issue On Wed, Jan 09, 2013 at 10:06:12AM +, Tom Brown wrote: libvirtError: internal error Process exited while reading console log outpu could this be related to selinux? can you try disabling it and see if migration succeeds? It was indeed the case! my src node was set to disabled and my destination node was enforcing, this was due to the destination being the first HV built and therefore provisioned slightly differently, my kickstart server is a VM in the pool. Its interesting that a VM can be provisioned onto a node that is set to enforcing and yet not migrated to. I have (only a vague) memory of discussing this already... Shouldn't oVirt-Engine be aware of selinux enforcement? If a cluster has disabled hosts, an enforcing host should not be operational (or at least warn the admin about that). I recall something like that, but I don't recall we ever converged and can't find the thread What is your opinion on the subject? I think that at the least, the scheduler must be aware of selinux enforcement when it chooses migration destination. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] oVirt 3.1 - VM Migration Issue
On 01/07/2013 01:18 PM, Tom Brown wrote: VDSM is the virtualization agent. look at /var/log/vdsm/vdsm.log many thanks - attached is the snip of that log when i try the migration thanks can you attach the same snip from the src VDSM 10.192.42.196 as well? ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] oVirt 3.1 - VM Migration Issue
, line 474, in _startUnderlyingMigration None, maxBandwidth) File /usr/share/vdsm/libvirtvm.py, line 510, in f ret = attr(*args, **kwargs) File /usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py, line 83, in wrapper ret = f(*args, **kwargs) File /usr/lib64/python2.6/site-packages/libvirt.py, line 1103, in migrateToURI2 if ret == -1: raise libvirtError ('virDomainMigrateToURI2() failed', dom=self) libvirtError: internal error Process exited while reading console log output: any chance you attach libvirtd.log and qemu log (/var/log/libvirt/qemu/{}.log? Danken - any insights? - Original Message - From: Tom Brown t...@ng23.net To: Roy Golan rgo...@redhat.com Cc: Haim Ateya hat...@redhat.com, users@ovirt.org Sent: Tuesday, January 8, 2013 11:50:26 AM Subject: Re: [Users] oVirt 3.1 - VM Migration Issue can you attach the same snip from the src VDSM 10.192.42.196 as well? The log is pretty chatty therefore i did another migration attempt and snipd'd the new log from both sides. see attached ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] oVirt 3.1 - VM Migration Issue
On 01/09/2013 09:35 AM, Haim Ateya wrote: libvirtError: internal error Process exited while reading console log outpu could this be related to selinux? can you try disabling it and see if migration succeeds? ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] oVirt 3.1 - VM Migration Issue
On 01/03/2013 05:07 PM, Tom Brown wrote: interesting, please search for migrationCreate command on desination host and search for ERROR afterwords, what do you see? - Original Message - From: Tom Brown t...@ng23.net To: users@ovirt.org Sent: Thursday, January 3, 2013 4:12:05 PM Subject: [Users] oVirt 3.1 - VM Migration Issue Hi I seem to have an issue with a single VM and migration, other VM's can migrate OK - When migrating from the GUI it appears to just hang but in the engine.log i see the following 2013-01-03 14:03:10,359 INFO [org.ovirt.engine.core.bll.VdsSelector] (ajp--0.0.0.0-8009-59) Checking for a specific VDS only - id:a2d84a1e-3e18-11e2-8851-3cd92b4c8e89, name:ovirt-node.domain-name, host_name(ip):10.192.42.165 2013-01-03 14:03:10,411 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-3-thread-48) [4d32917d] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 9dc63ce4-0f76-4963-adfe-6f8eb1a44806 Type: VM 2013-01-03 14:03:10,413 INFO [org.ovirt.engine.core.bll.VdsSelector] (pool-3-thread-48) [4d32917d] Checking for a specific VDS only - id:a2d84a1e-3e18-11e2-8851-3cd92b4c8e89, name:ovirt-node.domain-name, host_name(ip):10.192.42.165 2013-01-03 14:03:11,028 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-3-thread-48) [4d32917d] START, MigrateVDSCommand(vdsId = 1a52b722-43a1-11e2-af96-3cd92b4c8e89, vmId=9dc63ce4-0f76-4963-adfe-6f8eb1a44806, srcHost=10.192.42.196, dstVdsId=a2d84a1e-3e18-11e2-8851-3cd92b4c8e89, dstHost=10.192.42.165:54321, migrationMethod=ONLINE), log id: 5011789b 2013-01-03 14:03:11,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-3-thread-48) [4d32917d] VdsBroker::migrate::Entered (vm_guid=9dc63ce4-0f76-4963-adfe-6f8eb1a44806, srcHost=10.192.42.196, dstHost=10.192.42.165:54321, method=online 2013-01-03 14:03:11,031 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-3-thread-48) [4d32917d] START, MigrateBrokerVDSCommand(vdsId = 1a52b722-43a1-11e2-af96-3cd92b4c8e89, vmId=9dc63ce4-0f76-4963-adfe-6f8eb1a44806, srcHost=10.192.42.196, dstVdsId=a2d84a1e-3e18-11e2-8851-3cd92b4c8e89, dstHost=10.192.42.165:54321, migrationMethod=ONLINE), log id: 7cd53864 2013-01-03 14:03:11,041 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-3-thread-48) [4d32917d] FINISH, MigrateBrokerVDSCommand, log id: 7cd53864 2013-01-03 14:03:11,086 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-3-thread-48) [4d32917d] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 5011789b 2013-01-03 14:03:11,606 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-29) vds::refreshVmList vm id 9dc63ce4-0f76-4963-adfe-6f8eb1a44806 is migrating to vds ovirt-node.domain-name ignoring it in the refresh till migration is done 2013-01-03 14:03:12,836 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-36) VM test002.domain-name 9dc63ce4-0f76-4963-adfe-6f8eb1a44806 moved from MigratingFrom -- Up 2013-01-03 14:03:12,837 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-36) adding VM 9dc63ce4-0f76-4963-adfe-6f8eb1a44806 to re-run list 2013-01-03 14:03:12,852 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-36) Rerun vm 9dc63ce4-0f76-4963-adfe-6f8eb1a44806. Called from vds ovirt-node002.domain-name 2013-01-03 14:03:12,855 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (pool-3-thread-48) START, MigrateStatusVDSCommand(vdsId = 1a52b722-43a1-11e2-af96-3cd92b4c8e89, vmId=9dc63ce4-0f76-4963-adfe-6f8eb1a44806), log id: 4721a1f3 2013-01-03 14:03:12,864 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-48) Failed in MigrateStatusVDS method 2013-01-03 14:03:12,865 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-48) Error code migrateErr and error message VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration 2013-01-03 14:03:12,865 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-48) Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc mStatus Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc mCode 12 mMessage Fatal error during migration 2013-01-03 14:03:12,866 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-48) Vds: ovirt-node002.itvonline.ads 2013-01-03 14:03:12,867 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (pool-3-thread-48) Command MigrateStatusVDS execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration
Re: [Users] oVirt 3.1 - VM Migration Issue
interesting, please search for migrationCreate command on desination host and search for ERROR afterwords, what do you see? - Original Message - From: Tom Brown t...@ng23.net To: users@ovirt.org Sent: Thursday, January 3, 2013 4:12:05 PM Subject: [Users] oVirt 3.1 - VM Migration Issue Hi I seem to have an issue with a single VM and migration, other VM's can migrate OK - When migrating from the GUI it appears to just hang but in the engine.log i see the following 2013-01-03 14:03:10,359 INFO [org.ovirt.engine.core.bll.VdsSelector] (ajp--0.0.0.0-8009-59) Checking for a specific VDS only - id:a2d84a1e-3e18-11e2-8851-3cd92b4c8e89, name:ovirt-node.domain-name, host_name(ip):10.192.42.165 2013-01-03 14:03:10,411 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-3-thread-48) [4d32917d] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 9dc63ce4-0f76-4963-adfe-6f8eb1a44806 Type: VM 2013-01-03 14:03:10,413 INFO [org.ovirt.engine.core.bll.VdsSelector] (pool-3-thread-48) [4d32917d] Checking for a specific VDS only - id:a2d84a1e-3e18-11e2-8851-3cd92b4c8e89, name:ovirt-node.domain-name, host_name(ip):10.192.42.165 2013-01-03 14:03:11,028 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-3-thread-48) [4d32917d] START, MigrateVDSCommand(vdsId = 1a52b722-43a1-11e2-af96-3cd92b4c8e89, vmId=9dc63ce4-0f76-4963-adfe-6f8eb1a44806, srcHost=10.192.42.196, dstVdsId=a2d84a1e-3e18-11e2-8851-3cd92b4c8e89, dstHost=10.192.42.165:54321, migrationMethod=ONLINE), log id: 5011789b 2013-01-03 14:03:11,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-3-thread-48) [4d32917d] VdsBroker::migrate::Entered (vm_guid=9dc63ce4-0f76-4963-adfe-6f8eb1a44806, srcHost=10.192.42.196, dstHost=10.192.42.165:54321, method=online 2013-01-03 14:03:11,031 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-3-thread-48) [4d32917d] START, MigrateBrokerVDSCommand(vdsId = 1a52b722-43a1-11e2-af96-3cd92b4c8e89, vmId=9dc63ce4-0f76-4963-adfe-6f8eb1a44806, srcHost=10.192.42.196, dstVdsId=a2d84a1e-3e18-11e2-8851-3cd92b4c8e89, dstHost=10.192.42.165:54321, migrationMethod=ONLINE), log id: 7cd53864 2013-01-03 14:03:11,041 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-3-thread-48) [4d32917d] FINISH, MigrateBrokerVDSCommand, log id: 7cd53864 2013-01-03 14:03:11,086 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-3-thread-48) [4d32917d] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 5011789b 2013-01-03 14:03:11,606 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-29) vds::refreshVmList vm id 9dc63ce4-0f76-4963-adfe-6f8eb1a44806 is migrating to vds ovirt-node.domain-name ignoring it in the refresh till migration is done 2013-01-03 14:03:12,836 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-36) VM test002.domain-name 9dc63ce4-0f76-4963-adfe-6f8eb1a44806 moved from MigratingFrom -- Up 2013-01-03 14:03:12,837 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-36) adding VM 9dc63ce4-0f76-4963-adfe-6f8eb1a44806 to re-run list 2013-01-03 14:03:12,852 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-36) Rerun vm 9dc63ce4-0f76-4963-adfe-6f8eb1a44806. Called from vds ovirt-node002.domain-name 2013-01-03 14:03:12,855 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (pool-3-thread-48) START, MigrateStatusVDSCommand(vdsId = 1a52b722-43a1-11e2-af96-3cd92b4c8e89, vmId=9dc63ce4-0f76-4963-adfe-6f8eb1a44806), log id: 4721a1f3 2013-01-03 14:03:12,864 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-48) Failed in MigrateStatusVDS method 2013-01-03 14:03:12,865 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-48) Error code migrateErr and error message VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration 2013-01-03 14:03:12,865 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-48) Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc mStatus Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc mCode 12 mMessage Fatal error during migration 2013-01-03 14:03:12,866 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-48) Vds: ovirt-node002.itvonline.ads 2013-01-03 14:03:12,867 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (pool-3-thread-48) Command MigrateStatusVDS execution failed. Exception: VDSErrorException: VDSGenericException:
Re: [Users] oVirt 3.1 - VM Migration Issue
interesting, please search for migrationCreate command on desination host and search for ERROR afterwords, what do you see? - Original Message - From: Tom Brown t...@ng23.net To: users@ovirt.org Sent: Thursday, January 3, 2013 4:12:05 PM Subject: [Users] oVirt 3.1 - VM Migration Issue Hi I seem to have an issue with a single VM and migration, other VM's can migrate OK - When migrating from the GUI it appears to just hang but in the engine.log i see the following 2013-01-03 14:03:10,359 INFO [org.ovirt.engine.core.bll.VdsSelector] (ajp--0.0.0.0-8009-59) Checking for a specific VDS only - id:a2d84a1e-3e18-11e2-8851-3cd92b4c8e89, name:ovirt-node.domain-name, host_name(ip):10.192.42.165 2013-01-03 14:03:10,411 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-3-thread-48) [4d32917d] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 9dc63ce4-0f76-4963-adfe-6f8eb1a44806 Type: VM 2013-01-03 14:03:10,413 INFO [org.ovirt.engine.core.bll.VdsSelector] (pool-3-thread-48) [4d32917d] Checking for a specific VDS only - id:a2d84a1e-3e18-11e2-8851-3cd92b4c8e89, name:ovirt-node.domain-name, host_name(ip):10.192.42.165 2013-01-03 14:03:11,028 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-3-thread-48) [4d32917d] START, MigrateVDSCommand(vdsId = 1a52b722-43a1-11e2-af96-3cd92b4c8e89, vmId=9dc63ce4-0f76-4963-adfe-6f8eb1a44806, srcHost=10.192.42.196, dstVdsId=a2d84a1e-3e18-11e2-8851-3cd92b4c8e89, dstHost=10.192.42.165:54321, migrationMethod=ONLINE), log id: 5011789b 2013-01-03 14:03:11,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-3-thread-48) [4d32917d] VdsBroker::migrate::Entered (vm_guid=9dc63ce4-0f76-4963-adfe-6f8eb1a44806, srcHost=10.192.42.196, dstHost=10.192.42.165:54321, method=online 2013-01-03 14:03:11,031 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-3-thread-48) [4d32917d] START, MigrateBrokerVDSCommand(vdsId = 1a52b722-43a1-11e2-af96-3cd92b4c8e89, vmId=9dc63ce4-0f76-4963-adfe-6f8eb1a44806, srcHost=10.192.42.196, dstVdsId=a2d84a1e-3e18-11e2-8851-3cd92b4c8e89, dstHost=10.192.42.165:54321, migrationMethod=ONLINE), log id: 7cd53864 2013-01-03 14:03:11,041 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-3-thread-48) [4d32917d] FINISH, MigrateBrokerVDSCommand, log id: 7cd53864 2013-01-03 14:03:11,086 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-3-thread-48) [4d32917d] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 5011789b 2013-01-03 14:03:11,606 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-29) vds::refreshVmList vm id 9dc63ce4-0f76-4963-adfe-6f8eb1a44806 is migrating to vds ovirt-node.domain-name ignoring it in the refresh till migration is done 2013-01-03 14:03:12,836 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-36) VM test002.domain-name 9dc63ce4-0f76-4963-adfe-6f8eb1a44806 moved from MigratingFrom -- Up 2013-01-03 14:03:12,837 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-36) adding VM 9dc63ce4-0f76-4963-adfe-6f8eb1a44806 to re-run list 2013-01-03 14:03:12,852 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-36) Rerun vm 9dc63ce4-0f76-4963-adfe-6f8eb1a44806. Called from vds ovirt-node002.domain-name 2013-01-03 14:03:12,855 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (pool-3-thread-48) START, MigrateStatusVDSCommand(vdsId = 1a52b722-43a1-11e2-af96-3cd92b4c8e89, vmId=9dc63ce4-0f76-4963-adfe-6f8eb1a44806), log id: 4721a1f3 2013-01-03 14:03:12,864 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-48) Failed in MigrateStatusVDS method 2013-01-03 14:03:12,865 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-48) Error code migrateErr and error message VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration 2013-01-03 14:03:12,865 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-48) Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc mStatus Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc mCode 12 mMessage Fatal error during migration 2013-01-03 14:03:12,866 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-48) Vds: ovirt-node002.itvonline.ads 2013-01-03 14:03:12,867 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (pool-3-thread-48) Command MigrateStatusVDS execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: