Can you please attach the relevant lines from vdsm.log on the hosts? libvirt logs and qemu logs would also be helpful
On Tue, Mar 12, 2019 at 2:50 AM Bong Shau Fui <[email protected]> wrote: > Hi: > I deployed 2 ovirt hosts and an ovirt engine in a nested KVM server. > I've a windows vm setup and tried to perform live migration but failed. I > checked on the hosts and found them meeting the live migration > requirements, or at least that's what I thought. I took the requirement > from the below document. > > https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.5/html/Administration_Guide/sect-Migrating_Virtual_Machines_Between_Hosts.html > The hosts, both source and destination are quite empty, with only the > hosted engine, 1 centos vm and the windows VM in the cluster. I can do a > live migration for the centos vm successfully. But when I tried live > migration on the hosted-engine vm it failed immediately with a message "No > available host to migrate VMs to". When I tried to migrate the windows VM > the message box that let me choose the destination host popped up but > failed after a while. > I'd like to ask where can I get more information with regards to > live-migration apart from /var/log/ovirt-engine/engine.log ? I also > checked on the ovirt hosts' /var/log/vdsm/vdsm.log but found nothing > pointing to the reason why it failed. > Below is the extract from /var/log/ovirt-engine/engine.log when the > live-migration took place > > 2019-03-12 14:37:58,159+08 INFO > [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default task-131) [] > User admin@internal successfully logged in with scopes: ovirt-app-api > ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search > ovirt-ext=token-info:validate ovirt-ext=token:password-access > 2019-03-12 14:37:58,450+08 INFO > [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand] > (EE-ManagedThreadFactory-engineScheduled-Thread-59) [77888830] Lock freed > to object > 'EngineLock:{exclusiveLocks='[d113be83-2740-4246-a1f2-b9344889c3cf=PROVIDER]', > sharedLocks=''}' > 2019-03-12 14:38:02,544+08 INFO > [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] > (EE-ManagedThreadFactory-engineScheduled-Thread-50) [] > BaseAsyncTask::onTaskEndSuccess: Task > '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', > Parameters Type > 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended > successfully. > 2019-03-12 14:38:12,677+08 INFO > [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] > (EE-ManagedThreadFactory-engineScheduled-Thread-16) [] > BaseAsyncTask::onTaskEndSuccess: Task > '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', > Parameters Type > 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended > successfully. > 2019-03-12 14:38:21,650+08 INFO > [org.ovirt.engine.core.bll.aaa.SessionDataContainer] > (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] Not removing session > 'xDiHqqa6l+g8cngM26TTCfW7NeLN3WgWChsx28wUM391vAngSxwtyCkLbQxZR1AbJ5I+2bkPZNQijMUk0jLZcA==', > session has running commands for user 'admin@internal-authz'. > 2019-03-12 14:38:22,782+08 INFO > [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] > (EE-ManagedThreadFactory-engineScheduled-Thread-49) [] > BaseAsyncTask::onTaskEndSuccess: Task > '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', > Parameters Type > 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended > successfully. > 2019-03-12 14:38:33,018+08 INFO > [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] > (EE-ManagedThreadFactory-engineScheduled-Thread-74) [] > BaseAsyncTask::onTaskEndSuccess: Task > '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', > Parameters Type > 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended > successfully. > 2019-03-12 14:38:43,261+08 INFO > [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] > (EE-ManagedThreadFactory-engineScheduled-Thread-59) [77888830] > BaseAsyncTask::onTaskEndSuccess: Task > '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', > Parameters Type > 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended > successfully. > 2019-03-12 14:38:53,528+08 INFO > [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] > (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] > BaseAsyncTask::onTaskEndSuccess: Task > '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', > Parameters Type > 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended > successfully. > 2019-03-12 14:39:03,759+08 INFO > [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] > (EE-ManagedThreadFactory-engineScheduled-Thread-43) [] > BaseAsyncTask::onTaskEndSuccess: Task > '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', > Parameters Type > 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended > successfully. > 2019-03-12 14:39:14,011+08 INFO > [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] > (EE-ManagedThreadFactory-engineScheduled-Thread-60) [] > BaseAsyncTask::onTaskEndSuccess: Task > '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', > Parameters Type > 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended > successfully. > 2019-03-12 14:39:21,660+08 INFO > [org.ovirt.engine.core.bll.aaa.SessionDataContainer] > (EE-ManagedThreadFactory-engineScheduled-Thread-79) [] Not removing session > 'xDiHqqa6l+g8cngM26TTCfW7NeLN3WgWChsx28wUM391vAngSxwtyCkLbQxZR1AbJ5I+2bkPZNQijMUk0jLZcA==', > session has running commands for user 'admin@internal-authz'. > 2019-03-12 14:39:24,122+08 INFO > [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] > (EE-ManagedThreadFactory-engineScheduled-Thread-85) [] > BaseAsyncTask::onTaskEndSuccess: Task > '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', > Parameters Type > 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended > successfully. > 2019-03-12 14:39:29,773+08 INFO > [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-131) > [7f0cf113-55e8-4def-9c68-de3b91d6d641] Lock Acquired to object > 'EngineLock:{exclusiveLocks='[5cad5c5f-5aab-46ec-a28e-d484abc0401d=VM]', > sharedLocks=''}' > 2019-03-12 14:39:29,887+08 INFO > [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (default task-131) > [7f0cf113-55e8-4def-9c68-de3b91d6d641] Running command: > MigrateVmToServerCommand internal: false. Entities affected : ID: > 5cad5c5f-5aab-46ec-a28e-d484abc0401d Type: VMAction group MIGRATE_VM with > role type USER > 2019-03-12 14:39:30,019+08 INFO > [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-131) > [7f0cf113-55e8-4def-9c68-de3b91d6d641] START, MigrateVDSCommand( > MigrateVDSCommandParameters:{hostId='f9014bc4-485c-4eb0-a9bc-42d13ed68f41', > vmId='5cad5c5f-5aab-46ec-a28e-d484abc0401d', srcHost='host2.xxxx.com', > dstVdsId='1bc9b9e9-1e90-4570-9930-08416d1927cc', dstHost=' > host3.xxxx.com:54321', migrationMethod='ONLINE', tunnelMigration='false', > migrationDowntime='0', autoConverge='true', migrateCompressed='false', > consoleAddress='null', maxBandwidth='null', enableGuestEvents='true', > maxIncomingMigrations='2', maxOutgoingMigrations='2', > convergenceSchedule='[init=[{name=setDowntime, params=[100]}], > stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, > action={name=setDowntime, params=[200]}}, {limit=3, > action={name=setDowntime, params=[300]}}, {limit=4, > action={name=setDowntime, params=[400]}}, {limit=6, > action={name=setDowntime, params=[500]}}, {limit=-1, action={nam > e=abort, params=[]}}]]', dstQemu='192.168.138.135'}), log id: 7eeb678c > 2019-03-12 14:39:30,022+08 INFO > [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] > (default task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] START, > MigrateBrokerVDSCommand(HostName = host2.xxxx.com, > MigrateVDSCommandParameters:{hostId='f9014bc4-485c-4eb0-a9bc-42d13ed68f41', > vmId='5cad5c5f-5aab-46ec-a28e-d484abc0401d', srcHost='host2.xxxx.com', > dstVdsId='1bc9b9e9-1e90-4570-9930-08416d1927cc', dstHost=' > host3.xxxx.com:54321', migrationMethod='ONLINE', tunnelMigration='false', > migrationDowntime='0', autoConverge='true', migrateCompressed='false', > consoleAddress='null', maxBandwidth='null', enableGuestEvents='true', > maxIncomingMigrations='2', maxOutgoingMigrations='2', > convergenceSchedule='[init=[{name=setDowntime, params=[100]}], > stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, > action={name=setDowntime, params=[200]}}, {limit=3, > action={name=setDowntime, params=[300]}}, {limit=4, > action={name=setDowntime, params=[400]}}, {limit=6, action={name=set > Downtime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]', > dstQemu='192.168.138.135'}), log id: 5cef4981 > 2019-03-12 14:39:30,039+08 INFO > [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] > (default task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] FINISH, > MigrateBrokerVDSCommand, return: , log id: 5cef4981 > 2019-03-12 14:39:30,048+08 INFO > [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (default task-131) > [7f0cf113-55e8-4def-9c68-de3b91d6d641] FINISH, MigrateVDSCommand, return: > MigratingFrom, log id: 7eeb678c > 2019-03-12 14:39:30,067+08 INFO > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > (default task-131) [7f0cf113-55e8-4def-9c68-de3b91d6d641] EVENT_ID: > VM_MIGRATION_START(62), Migration started (VM: Win_2016_1, Source: > host2.xxxx.com, Destination: host3, User: admin@internal-authz). > 2019-03-12 14:39:33,901+08 INFO > [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] > (ForkJoinPool-1-worker-3) [] VM '5cad5c5f-5aab-46ec-a28e-d484abc0401d' was > reported as Down on VDS '1bc9b9e9-1e90-4570-9930-08416d1927cc'(host3) > 2019-03-12 14:39:33,903+08 INFO > [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] > (ForkJoinPool-1-worker-3) [] START, DestroyVDSCommand(HostName = host3, > DestroyVmVDSCommandParameters:{hostId='1bc9b9e9-1e90-4570-9930-08416d1927cc', > vmId='5cad5c5f-5aab-46ec-a28e-d484abc0401d', secondsToWait='0', > gracefully='false', reason='', ignoreNoVm='true'}), log id: c853ba5 > 2019-03-12 14:39:34,211+08 INFO > [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] > (EE-ManagedThreadFactory-engineScheduled-Thread-73) [] > BaseAsyncTask::onTaskEndSuccess: Task > '67631cf6-4c75-4681-88ef-fd4af56c0363' (Parent Command 'RemoveDisk', > Parameters Type > 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended > successfully. > 2019-03-12 14:39:34,604+08 INFO > [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] > (ForkJoinPool-1-worker-3) [] Failed to destroy VM > '5cad5c5f-5aab-46ec-a28e-d484abc0401d' because VM does not exist, ignoring > 2019-03-12 14:39:34,605+08 INFO > [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] > (ForkJoinPool-1-worker-3) [] FINISH, DestroyVDSCommand, return: , log id: > c853ba5 > 2019-03-12 14:39:34,605+08 INFO > [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] > (ForkJoinPool-1-worker-3) [] VM > '5cad5c5f-5aab-46ec-a28e-d484abc0401d'(Win_2016_1) was unexpectedly > detected as 'Down' on VDS '1bc9b9e9-1e90-4570-9930-08416d1927cc'(ohost3) > (expected on 'f9014bc4-485c-4eb0-a9bc-42d13ed68f41') > 2019-03-12 14:39:34,605+08 ERROR > [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] > (ForkJoinPool-1-worker-3) [] Migration of VM 'Win_2016_1' to host 'host3' > failed: VM destroyed during the startup. > 2019-03-12 14:39:34,615+08 INFO > [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] > (ForkJoinPool-1-worker-10) [] VM > '5cad5c5f-5aab-46ec-a28e-d484abc0401d'(Win_2016_1) moved from > 'MigratingFrom' --> 'Up' > 2019-03-12 14:39:34,615+08 INFO > [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] > (ForkJoinPool-1-worker-10) [] Adding VM > '5cad5c5f-5aab-46ec-a28e-d484abc0401d'(Win_2016_1) to re-run list > 2019-03-12 14:39:34,621+08 ERROR > [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring] > (ForkJoinPool-1-worker-10) [] Rerun VM > '5cad5c5f-5aab-46ec-a28e-d484abc0401d'. Called from VDS 'host2.xxxx.com' > 2019-03-12 14:39:34,752+08 INFO > [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] > (EE-ManagedThreadFactory-engine-Thread-53959) [] START, > MigrateStatusVDSCommand(HostName = host2.xxxx.com, > MigrateStatusVDSCommandParameters:{hostId='f9014bc4-485c-4eb0-a9bc-42d13ed68f41', > vmId='5cad5c5f-5aab-46ec-a28e-d484abc0401d'}), log id: 7ded4ad7 > 2019-03-12 14:39:34,760+08 INFO > [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] > (EE-ManagedThreadFactory-engine-Thread-53959) [] FINISH, > MigrateStatusVDSCommand, return: , log id: 7ded4ad7 > 2019-03-12 14:39:34,786+08 ERROR > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > (EE-ManagedThreadFactory-engine-Thread-53959) [] EVENT_ID: > VM_MIGRATION_TO_SERVER_FAILED(120), Migration failed (VM: Win_2016_1, > Source: host2.xxxx.com, Destination: host3). > > Any help is greatly appreciated. > > > regards, > Bong SF > _______________________________________________ > Users mailing list -- [email protected] > To unsubscribe send an email to [email protected] > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/[email protected]/message/DWCLF4NYZWWN43K6774YGQKANYHCWFTL/ > -- Ryan Barry Associate Manager - RHV Virt/SLA [email protected] M: +16518159306 IM: rbarry <https://red.ht/sig>
_______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/PB2OLTT2WCCR6HBUJJGEIRSD7ASX654O/
