On Tue, Jun 19, 2018 at 11:39 AM <r...@chef.net> wrote:
> Hello,
>
> I'm using a 4 node cluster oVirt 4.1. After updating the Engine to 4.2.3.8
> (and ContOS 7.5), I tried to update the nodes too.
> Unfortunally, the live migration is not working (any more). Maybe it's
> related to following error found in engine.log
>
> 2018-06-19 09:32:53,469+02 WARN
> [org.ovirt.engine.core.bll.provider.network.openstack.CustomizedRESTEasyConnector]
> (EE-ManagedThreadFactory-engineScheduled-Thread-7) [4d6a7dc4] Cannot
> register external providers trust store: java.io.IOException: Keystore was
> tampered with, or password was incorrect
> 2018-06-19 09:32:53,475+02 ERROR
> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand]
> (EE-ManagedThreadFactory-engineScheduled-Thread-7) [4d6a7dc4] Command
> 'org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand'
> failed: EngineException: (Failed with error unable to find valid
> certification path to requested target and code 5050)
>
>
Didi, do have an idea what can be the issue?
> I'm using the selfsigned CA provided with oVirt.
>
> Nodes (source and destination)
> OS-Version:RHEL - 7 - 4.1708.el7.centos
> Kernelversion: 3.10.0 - 693.21.1.el7.x86_64
> KVM-Version:2.9.0 - 16.el7_4.14.1
> LIBVIRT-Version:libvirt-3.2.0-14.el7_4.9
> VDSM-Version:vdsm-4.19.45-1.el7.centos
> SPICE-Version:0.12.8 - 2.el7.1
> CEPH-Version:librbd1-0.94.5-2.el7
> Kernel Features:PTI: 1, IBPB: 0, IBRS: 0
>
> engine.log for the mirgration:
> 2018-06-14 13:13:30,300+02 INFO
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (EE-ManagedThreadFactory-engine-Thread-246968) [] EVENT_ID:
> VM_MIGRATION_START_SYSTEM_INITIATED(67), Migration initiated by system (VM:
> vm_to_migrate, Source: SOURCE, Destination: DESTINATION, Reason: Host
> preparing for maintenance).
> 2018-06-14 13:13:32,886+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmsStatisticsFetcher]
> (EE-ManagedThreadFactory-engineScheduled-Thread-94) [] Fetched 8 VMs from
> VDS 'f3749f1e-a68d-424f-aa3a-f0ec996df498'
> 2018-06-14 13:13:32,887+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-94) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111'(vm_to_migrate) was unexpectedly
> detected as 'MigratingTo' on VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) (expected on
> 'ea7bae8f-d966-4baa-b8b4-a5522b88ec3a')
> 2018-06-14 13:13:32,887+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-94) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111' is migrating to VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) ignoring it in the
> refresh until migration is done
> 2018-06-14 13:13:47,939+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-42) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111'(vm_to_migrate) was unexpectedly
> detected as 'MigratingTo' on VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) (expected on
> 'ea7bae8f-d966-4baa-b8b4-a5522b88ec3a')
> 2018-06-14 13:13:47,939+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-42) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111' is migrating to VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) ignoring it in the
> refresh until migration is done
> 2018-06-14 13:14:02,993+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111'(vm_to_migrate) was unexpectedly
> detected as 'MigratingTo' on VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) (expected on
> 'ea7bae8f-d966-4baa-b8b4-a5522b88ec3a')
> 2018-06-14 13:14:02,993+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111' is migrating to VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) ignoring it in the
> refresh until migration is done
> 2018-06-14 13:14:18,042+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111'(vm_to_migrate) was unexpectedly
> detected as 'MigratingTo' on VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) (expected on
> 'ea7bae8f-d966-4baa-b8b4-a5522b88ec3a')
> 2018-06-14 13:14:18,042+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111' is migrating to VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) ignoring it in the
> refresh until migration is done
> 2018-06-14 13:14:33,090+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-85) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111'(vm_to_migrate) was unexpectedly
> detected as 'MigratingTo' on VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) (expected on
> 'ea7bae8f-d966-4baa-b8b4-a5522b88ec3a')
> 2018-06-14 13:14:33,090+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-85) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111' is migrating to VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) ignoring it in the
> refresh until migration is done
> 2018-06-14 13:14:38,356+02 INFO
> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand]
> (EE-ManagedThreadFactory-engineScheduled-Thread-43) [6d991f10] Lock
> Acquired to object
> 'EngineLock:{exclusiveLocks='[d1457e7e-f29d-49ac-b635-2457a7cbc59f=PROVIDER]',
> sharedLocks=''}'
> 2018-06-14 13:14:38,369+02 INFO
> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand]
> (EE-ManagedThreadFactory-engineScheduled-Thread-43) [6d991f10] Running
> command: SyncNetworkProviderCommand internal: true.
> 2018-06-14 13:14:38,370+02 WARN
> [org.ovirt.engine.core.bll.provider.network.openstack.CustomizedRESTEasyConnector]
> (EE-ManagedThreadFactory-engineScheduled-Thread-43) [6d991f10] Cannot
> register external providers trust store: java.io.IOException: Keystore was
> tampered with, or password was incorrect
> 2018-06-14 13:14:38,379+02 ERROR
> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand]
> (EE-ManagedThreadFactory-engineScheduled-Thread-43) [6d991f10] Command
> 'org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand'
> failed: EngineException: (Failed with error unable to find valid
> certification path to requested target and code 5050)
> 2018-06-14 13:14:38,382+02 INFO
> [org.ovirt.engine.core.bll.provider.network.SyncNetworkProviderCommand]
> (EE-ManagedThreadFactory-engineScheduled-Thread-43) [6d991f10] Lock freed
> to object
> 'EngineLock:{exclusiveLocks='[d1457e7e-f29d-49ac-b635-2457a7cbc59f=PROVIDER]',
> sharedLocks=''}'
> 2018-06-14 13:14:48,139+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-92) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111'(vm_to_migrate) was unexpectedly
> detected as 'MigratingTo' on VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) (expected on
> 'ea7bae8f-d966-4baa-b8b4-a5522b88ec3a')
> 2018-06-14 13:14:48,139+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-92) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111' is migrating to VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) ignoring it in the
> refresh until migration is done
> 2018-06-14 13:15:03,193+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-71) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111'(vm_to_migrate) was unexpectedly
> detected as 'MigratingTo' on VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) (expected on
> 'ea7bae8f-d966-4baa-b8b4-a5522b88ec3a')
> 2018-06-14 13:15:03,193+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-71) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111' is migrating to VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) ignoring it in the
> refresh until migration is done
> 2018-06-14 13:15:18,242+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111'(vm_to_migrate) was unexpectedly
> detected as 'MigratingTo' on VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) (expected on
> 'ea7bae8f-d966-4baa-b8b4-a5522b88ec3a')
> 2018-06-14 13:15:18,242+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111' is migrating to VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) ignoring it in the
> refresh until migration is done
> 2018-06-14 13:15:33,293+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-17) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111'(vm_to_migrate) was unexpectedly
> detected as 'MigratingTo' on VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) (expected on
> 'ea7bae8f-d966-4baa-b8b4-a5522b88ec3a')
> 2018-06-14 13:15:33,293+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-17) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111' is migrating to VDS
> 'f3749f1e-a68d-424f-aa3a-f0ec996df498'(DESTINATION) ignoring it in the
> refresh until migration is done
> 2018-06-14 13:15:45,377+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-10) [] VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111'(vm_to_migrate) moved from
> 'MigratingFrom' --> 'Up'
> 2018-06-14 13:15:45,377+02 INFO
> [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer]
> (EE-ManagedThreadFactory-engineScheduled-Thread-10) [] Adding VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111'(vm_to_migrate) to re-run list
> 2018-06-14 13:15:45,400+02 ERROR
> [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring]
> (EE-ManagedThreadFactory-engineScheduled-Thread-10) [] Rerun VM
> '4aff4193-ba75-481d-92b3-59b62cd8b111'. Called from VDS 'SOURCE'
> 2018-06-14 13:15:45,451+02 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand]
> (EE-ManagedThreadFactory-engine-Thread-247023) [] START,
> MigrateStatusVDSCommand(HostName = SOURCE,
> MigrateStatusVDSCommandParameters:{hostId='ea7bae8f-d966-4baa-b8b4-a5522b88ec3a',
> vmId='4aff4193-ba75-481d-92b3-59b62cd8b111'}), log id: 25f95334
> 2018-06-14 13:15:45,453+02 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand]
> (EE-ManagedThreadFactory-engine-Thread-247023) [] FINISH,
> MigrateStatusVDSCommand, log id: 25f95334
> 2018-06-14 13:15:45,478+02 WARN
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (EE-ManagedThreadFactory-engine-Thread-247023) [] EVENT_ID:
> VM_MIGRATION_TRYING_RERUN(128), Failed to migrate VM vm_to_migrate to Host
> DESTINATION . Trying to migrate to another Host.
> 2018-06-14 13:15:45,557+02 INFO
> [org.ovirt.engine.core.bll.MigrateVmCommand]
> (EE-ManagedThreadFactory-engine-Thread-247023) [] Running command:
> MigrateVmCommand internal: true. Entities affected : ID:
> 4aff4193-ba75-481d-92b3-59b62cd8b111 Type: VMAction group MIGRATE_VM with
> role type USER
> 2018-06-14 13:15:45,595+02 INFO
> [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand]
> (EE-ManagedThreadFactory-engine-Thread-247023) [] START, MigrateVDSCommand(
> MigrateVDSCommandParameters:{hostId='ea7bae8f-d966-4baa-b8b4-a5522b88ec3a',
> vmId='4aff4193-ba75-481d-92b3-59b62cd8b111', srcHost='10.157.8.37',
> dstVdsId='69ebacd7-ee55-4c52-abf1-437a14d5fb0d', dstHost='
> 10.157.8.42:54321', migrationMethod='ONLINE', tunnelMigration='false',
> migrationDowntime='0', autoConverge='false', migrateCompressed='false',
> consoleAddress='null', maxBandwidth='500', enableGuestEvents='false',
> maxIncomingMigrations='2', maxOutgoingMigrations='2',
> convergenceSchedule='null', dstQemu='192.168.1.113'}), log id: 736d023f
> 2018-06-14 13:15:45,595+02 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand]
> (EE-ManagedThreadFactory-engine-Thread-247023) [] START,
> MigrateBrokerVDSCommand(HostName = SOURCE,
> MigrateVDSCommandParameters:{hostId='ea7bae8f-d966-4baa-b8b4-a5522b88ec3a',
> vmId='4aff4193-ba75-481d-92b3-59b62cd8b111', srcHost='10.157.8.37',
> dstVdsId='69ebacd7-ee55-4c52-abf1-437a14d5fb0d', dstHost='
> 10.157.8.42:54321', migrationMethod='ONLINE', tunnelMigration='false',
> migrationDowntime='0', autoConverge='false', migrateCompressed='false',
> consoleAddress='null', maxBandwidth='500', enableGuestEvents='false',
> maxIncomingMigrations='2', maxOutgoingMigrations='2',
> convergenceSchedule='null', dstQemu='192.168.1.113'}), log id: 58572c5b
> 2018-06-14 13:15:45,601+02 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand]
> (EE-ManagedThreadFactory-engine-Thread-247023) [] FINISH,
> MigrateBrokerVDSCommand, log id: 58572c5b
> 2018-06-14 13:15:45,604+02 INFO
> [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand]
> (EE-ManagedThreadFactory-engine-Thread-247023) [] FINISH,
> MigrateVDSCommand, return: MigratingFrom, log id: 736d023f
>
> _______________________________________________
> Users mailing list -- users@ovirt.org
> To unsubscribe send an email to users-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/users@ovirt.org/message/B6IEVX5XEBFHJCWCHGRMPOJ27VUJI2TL/
>
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/LKLSSLJJTUTUHJDKZDMRUGTUGU6YZCDI/
