Hi:
I deployed 2 ovirt hosts and an ovirt engine in a nested KVM server. I've a
windows vm setup and tried to perform live migration but failed. I checked on
the hosts and found them meeting the live migration requirements, or at least
that's what I thought. I took the requirement from the below document.
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.5/html/Administration_Guide/sect-Migrating_Virtual_Machines_Between_Hosts.html
The hosts, both source and destination are quite empty, with only the hosted
engine, 1 centos vm and the windows VM in the cluster. I can do a live
migration for the centos vm successfully. But when I tried live migration on
the hosted-engine vm it failed immediately with a message "No available host to
migrate VMs to". When I tried to migrate the windows VM the message box that
let me choose the destination host popped up but failed after a while.
I'd like to ask where can I get more information with regards to
live-migration apart from /var/log/ovirt-engine/engine.log ? I also checked on
the ovirt hosts' /var/log/vdsm/vdsm.log but found nothing pointing to the
reason why it failed.
Below is the extract from /var/log/ovirt-engine/engine.log when the
live-migration took place
2019-03-12 14:37:58,159+08 INFO
[org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default task-131) []
User admin@internal successfully logged in with scopes: ovirt-app-api
ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search
ovirt-ext=token-info:validate ovirt-ext=token:password-access
2019-03-12 14:37:58,450+08 INFO
[org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand]
(EE-ManagedThreadFactory-engineScheduled-Thread-59) [77888830] Lock freed to
object
'EngineLock:{exclusiveLocks='[d113be83-2740-4246-a1f2-b9344889c3cf=PROVIDER]',
sharedLocks=''}'
2019-03-12 14:38:02,544+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
(EE-ManagedThreadFactory-engineScheduled-Thread-50) []
BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363'
(Parent Command 'RemoveDisk', Parameters Type
'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended
successfully.
2019-03-12 14:38:12,677+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
(EE-ManagedThreadFactory-engineScheduled-Thread-16) []
BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363'
(Parent Command 'RemoveDisk', Parameters Type
'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended
successfully.
2019-03-12 14:38:21,650+08 INFO
[org.ovirt.engine.core.bll.aaa.SessionDataContainer]
(EE-ManagedThreadFactory-engineScheduled-Thread-51) [] Not removing session
'xDiHqqa6l+g8cngM26TTCfW7NeLN3WgWChsx28wUM391vAngSxwtyCkLbQxZR1AbJ5I+2bkPZNQijMUk0jLZcA==',
session has running commands for user 'admin@internal-authz'.
2019-03-12 14:38:22,782+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
(EE-ManagedThreadFactory-engineScheduled-Thread-49) []
BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363'
(Parent Command 'RemoveDisk', Parameters Type
'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended
successfully.
2019-03-12 14:38:33,018+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
(EE-ManagedThreadFactory-engineScheduled-Thread-74) []
BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363'
(Parent Command 'RemoveDisk', Parameters Type
'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended
successfully.
2019-03-12 14:38:43,261+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
(EE-ManagedThreadFactory-engineScheduled-Thread-59) [77888830]
BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363'
(Parent Command 'RemoveDisk', Parameters Type
'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended
successfully.
2019-03-12 14:38:53,528+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
(EE-ManagedThreadFactory-engineScheduled-Thread-13) []
BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363'
(Parent Command 'RemoveDisk', Parameters Type
'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended
successfully.
2019-03-12 14:39:03,759+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
(EE-ManagedThreadFactory-engineScheduled-Thread-43) []
BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363'
(Parent Command 'RemoveDisk', Parameters Type
'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended
successfully.
2019-03-12 14:39:14,011+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
(EE-ManagedThreadFactory-engineScheduled-Thread-60) []
BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363'
(Parent Command 'RemoveDisk', Parameters Type
'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended
successfully.
2019-03-12 14:39:21,660+08 INFO
[org.ovirt.engine.core.bll.aaa.SessionDataContainer]
(EE-ManagedThreadFactory-engineScheduled-Thread-79) [] Not removing session
'xDiHqqa6l+g8cngM26TTCfW7NeLN3WgWChsx28wUM391vAngSxwtyCkLbQxZR1AbJ5I+2bkPZNQijMUk0jLZcA==',
session has running commands for user 'admin@internal-authz'.
2019-03-12 14:39:24,122+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
(EE-ManagedThreadFactory-engineScheduled-Thread-85) []
BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363'
(Parent Command 'RemoveDisk', Parameters Type
'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended
successfully.
2019-03-12 14:39:29,773+08 INFO
[org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-131)
[7f0cf113-55e8-4def-9c68-de3b91d6d641] Lock Acquired to object
'EngineLock:{exclusiveLocks='[5cad5c5f-5aab-46ec-a28e-d484abc0401d=VM]',
sharedLocks=''}'
2019-03-12 14:39:29,887+08 INFO
[org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-131)
[7f0cf113-55e8-4def-9c68-de3b91d6d641] Running command:
MigrateVmToServerCommand internal: false. Entities affected : ID:
5cad5c5f-5aab-46ec-a28e-d484abc0401d Type: VMAction group MIGRATE_VM with role
type USER
2019-03-12 14:39:30,019+08 INFO
[org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-131)
[7f0cf113-55e8-4def-9c68-de3b91d6d641] START, MigrateVDSCommand(
MigrateVDSCommandParameters:{hostId='f9014bc4-485c-4eb0-a9bc-42d13ed68f41',
vmId='5cad5c5f-5aab-46ec-a28e-d484abc0401d', srcHost='host2.xxxx.com',
dstVdsId='1bc9b9e9-1e90-4570-9930-08416d1927cc',
dstHost='host3.xxxx.com:54321', migrationMethod='ONLINE',
tunnelMigration='false', migrationDowntime='0', autoConverge='true',
migrateCompressed='false', consoleAddress='null', maxBandwidth='null',
enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2',
convergenceSchedule='[init=[{name=setDowntime, params=[100]}],
stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2,
action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime,
params=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6,
action={name=setDowntime, params=[500]}}, {limit=-1, action={nam
e=abort, params=[]}}]]', dstQemu='192.168.138.135'}), log id: 7eeb678c
2019-03-12 14:39:30,022+08 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default
task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] START,
MigrateBrokerVDSCommand(HostName = host2.xxxx.com,
MigrateVDSCommandParameters:{hostId='f9014bc4-485c-4eb0-a9bc-42d13ed68f41',
vmId='5cad5c5f-5aab-46ec-a28e-d484abc0401d', srcHost='host2.xxxx.com',
dstVdsId='1bc9b9e9-1e90-4570-9930-08416d1927cc',
dstHost='host3.xxxx.com:54321', migrationMethod='ONLINE',
tunnelMigration='false', migrationDowntime='0', autoConverge='true',
migrateCompressed='false', consoleAddress='null', maxBandwidth='null',
enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2',
convergenceSchedule='[init=[{name=setDowntime, params=[100]}],
stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2,
action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime,
params=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6,
action={name=set
Downtime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]',
dstQemu='192.168.138.135'}), log id: 5cef4981
2019-03-12 14:39:30,039+08 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (default
task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] FINISH,
MigrateBrokerVDSCommand, return: , log id: 5cef4981
2019-03-12 14:39:30,048+08 INFO
[org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-131)
[7f0cf113-55e8-4def-9c68-de3b91d6d641] FINISH, MigrateVDSCommand, return:
MigratingFrom, log id: 7eeb678c
2019-03-12 14:39:30,067+08 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default
task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] EVENT_ID:
VM_MIGRATION_START(62), Migration started (VM: Win_2016_1, Source:
host2.xxxx.com, Destination: host3, User: admin@internal-authz).
2019-03-12 14:39:33,901+08 INFO
[org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
(ForkJoinPool-1-worker-3) [] VM '5cad5c5f-5aab-46ec-a28e-d484abc0401d' was
reported as Down on VDS '1bc9b9e9-1e90-4570-9930-08416d1927cc'(host3)
2019-03-12 14:39:33,903+08 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand]
(ForkJoinPool-1-worker-3) [] START, DestroyVDSCommand(HostName = host3,
DestroyVmVDSCommandParameters:{hostId='1bc9b9e9-1e90-4570-9930-08416d1927cc',
vmId='5cad5c5f-5aab-46ec-a28e-d484abc0401d', secondsToWait='0',
gracefully='false', reason='', ignoreNoVm='true'}), log id: c853ba5
2019-03-12 14:39:34,211+08 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
(EE-ManagedThreadFactory-engineScheduled-Thread-73) []
BaseAsyncTask::onTaskEndSuccess: Task '67631cf6-4c75-4681-88ef-fd4af56c0363'
(Parent Command 'RemoveDisk', Parameters Type
'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended
successfully.
2019-03-12 14:39:34,604+08 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand]
(ForkJoinPool-1-worker-3) [] Failed to destroy VM
'5cad5c5f-5aab-46ec-a28e-d484abc0401d' because VM does not exist, ignoring
2019-03-12 14:39:34,605+08 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand]
(ForkJoinPool-1-worker-3) [] FINISH, DestroyVDSCommand, return: , log id:
c853ba5
2019-03-12 14:39:34,605+08 INFO
[org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
(ForkJoinPool-1-worker-3) [] VM
'5cad5c5f-5aab-46ec-a28e-d484abc0401d'(Win_2016_1) was unexpectedly detected as
'Down' on VDS '1bc9b9e9-1e90-4570-9930-08416d1927cc'(ohost3) (expected on
'f9014bc4-485c-4eb0-a9bc-42d13ed68f41')
2019-03-12 14:39:34,605+08 ERROR
[org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
(ForkJoinPool-1-worker-3) [] Migration of VM 'Win_2016_1' to host 'host3'
failed: VM destroyed during the startup.
2019-03-12 14:39:34,615+08 INFO
[org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
(ForkJoinPool-1-worker-10) [] VM
'5cad5c5f-5aab-46ec-a28e-d484abc0401d'(Win_2016_1) moved from 'MigratingFrom'
--> 'Up'
2019-03-12 14:39:34,615+08 INFO
[org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
(ForkJoinPool-1-worker-10) [] Adding VM
'5cad5c5f-5aab-46ec-a28e-d484abc0401d'(Win_2016_1) to re-run list
2019-03-12 14:39:34,621+08 ERROR
[org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring]
(ForkJoinPool-1-worker-10) [] Rerun VM '5cad5c5f-5aab-46ec-a28e-d484abc0401d'.
Called from VDS 'host2.xxxx.com'
2019-03-12 14:39:34,752+08 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-53959) [] START,
MigrateStatusVDSCommand(HostName = host2.xxxx.com,
MigrateStatusVDSCommandParameters:{hostId='f9014bc4-485c-4eb0-a9bc-42d13ed68f41',
vmId='5cad5c5f-5aab-46ec-a28e-d484abc0401d'}), log id: 7ded4ad7
2019-03-12 14:39:34,760+08 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-53959) [] FINISH,
MigrateStatusVDSCommand, return: , log id: 7ded4ad7
2019-03-12 14:39:34,786+08 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-53959) [] EVENT_ID:
VM_MIGRATION_TO_SERVER_FAILED(120), Migration failed (VM: Win_2016_1, Source:
host2.xxxx.com, Destination: host3).
Any help is greatly appreciated.
regards,
Bong SF
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/DWCLF4NYZWWN43K6774YGQKANYHCWFTL/
