On Tue, May 14, 2019 at 12:25 PM Yedidyah Bar David <[email protected]> wrote:
> On Tue, May 14, 2019 at 12:25 PM <[email protected]> wrote: > > > > > I'd personally first try to fix the engine<->vdsm comm issue. > > > > I fully agree. I might then take exports of all VMs and do a fresh start > over. I'm not sure the upgrade instructions work, as the upgrade for > 3.6->4.0 already activates much newer content. For example the quoted > --upgrade-appliance options does not even exist, probably because a later > package removed it. One would have to manually versionlock the 4.0 > hypervisior packages. > > > > > How did you upgrade the host? yum update? Is is ovirt-node? > > > > Currently the state is as follows: engine is running on the 3.6 branch, > all updates applied [1]. The hosting RHEL host is on the RHV4 branch with > all updats in, but the gnutls mentioned [2]. > > > > I managed to flag the host as in maintenance mode, so I could ask it to > re-enroll the certs. > > Did you? Please do. > > > I can see from the hosts logs, that the ssh connection works, and the > certs have been updated. But now I see in the host's vdsm log that the SSL > connection is being dropped. > > This can also be because a 3.6 engine defaults xmlrpc and a new 4.3 > host talks only jsonrpc. Not sure, perhaps you can change the host (on > engine side) to use json rpc and it would work, although a 4.3 host is > definitely not supposed to support a 3.6 engine. If all you want is to > export the VMs, I'd downgrade the host to 3.6. > > > > > > If you are ok debugging/fixing yourself and only need the occasional > > > tip, fine. Otherwise, please provide more details - exact versions of > > > engine/vdsm, last version that worked, whether you can reinstall the > > > host or need anything on it, etc. > > > > I'm infinitely grateful for any assistance! I didn't want to spam the > list with my first mail. I hope I provided some better information now. > Thanks! > > > > In the long run I will decommission this host. I need the VMs on it to > migrate to a fresh 4.3 3 host hyperconverged setup which is yet to be > built. Actually I am abusing one of the three nodes at the moment as a > backup system for this failed upgrade, so I'm blocked with going forward > with that setup as well. ... :( > > Another option you might attempt is to import the data domain directly > from the new engine. Obviously, do this first on a test copy... > > Re [2]: I'll use this opportunity to clarify something which is not > obvious, since you use RHV and not oVirt. > > There is a significant difference between oVirt and RHV regarding > repos/channels. In oVirt, each minor version has its own repos, and > users can choose freely which one they want. In RHV, this the same for > the engine channels, but not for hosts. For hosts, there is only a > single channel per each major version. So if you use RHV 4 host > channel, you get latest. Older versions are still available there, but > you have to play with yum to choose them, and in RHV this is > considered not supported, other than using RHVH (RHV downstream of > oVirt node) which can be used for specific cases (perhaps like yours, > but not sure it would help). Yes, for this specific case the best option is to use latest RHV-H from 4.2 time. > This has the advantage, for RHV > customers, of not having to add a new repo to get updates, and the > disadvantage of not being able to stay on a specific minor version > when upgrading. > > Good luck and best regards, > > > > > > [1] > > [root@engine ~]# engine-upgrade-check > > VERB: queue package rhevm-setup for update > > VERB: Downloading: repomdtDvMe7tmp.xml (0%) > > VERB: Downloading: repomdtDvMe7tmp.xml 3.4 k(100%) > > VERB: Downloading: jb-eap-6-for-rhel-6-server-rpms/primary_db (0%) > > VERB: Downloading: jb-eap-6-for-rhel-6-server-rpms/primary_db 1.2 M(100%) > > VERB: Downloading: repomd__rFIUtmp.xml (0%) > > VERB: Downloading: repomd__rFIUtmp.xml 3.4 k(100%) > > VERB: Downloading: repomdXw2WPVtmp.xml (0%) > > VERB: Downloading: repomdXw2WPVtmp.xml 4.0 k(100%) > > VERB: Downloading: repomdF3AGBOtmp.xml (0%) > > VERB: Downloading: repomdF3AGBOtmp.xml 3.5 k(100%) > > VERB: Downloading: repomd9DvU9vtmp.xml (0%) > > VERB: Downloading: repomd9DvU9vtmp.xml 3.4 k(100%) > > VERB: Downloading: rhel-6-server-supplementary-rpms/primary_db (0%) > > VERB: Downloading: rhel-6-server-supplementary-rpms/primary_db 851 > k(100%) > > VERB: Building transaction > > VERB: Empty transaction > > VERB: Transaction Summary: > > No upgrade > > > > [2] > > # subscription-manager repos --list-enabled; yum check-update > > +----------------------------------------------------------+ > > Available Repositories in /etc/yum.repos.d/redhat.repo > > +----------------------------------------------------------+ > > Repo ID: rhel-7-server-ansible-2-rpms > > Repo Name: Red Hat Ansible Engine 2 RPMs for Red Hat Enterprise Linux 7 > Server > > Repo URL: > https://cdn.redhat.com/content/dist/rhel/server/7/7Server/$basearch/a > > nsible/2/os > > Enabled: 1 > > > > Repo ID: rhel-7-server-rhev-mgmt-agent-rpms > > Repo Name: Red Hat Enterprise Virtualization Management Agents for RHEL > 7 (RPMs) > > Repo URL: > https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basear > > ch/rhev-mgmt-agent/3/os > > Enabled: 1 > > > > Repo ID: rhel-7-server-rhv-4-mgmt-agent-rpms > > Repo Name: Red Hat Virtualization 4 Management Agents for RHEL 7 (RPMs) > > Repo URL: > https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basear > > ch/rhv-mgmt-agent/4/os > > Enabled: 1 > > > > Repo ID: rhel-7-server-rpms > > Repo Name: Red Hat Enterprise Linux 7 Server (RPMs) > > Repo URL: > https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basear > > ch/os > > Enabled: 1 > > > > Repo ID: rhel-7-server-optional-rpms > > Repo Name: Red Hat Enterprise Linux 7 Server - Optional (RPMs) > > Repo URL: > https://cdn.redhat.com/content/dist/rhel/server/7/$releasever/$basear > > ch/optional/os > > Enabled: 1 > > > > Loaded plugins: enabled_repos_upload, package_upload, product-id, search- > > : disabled-repos, subscription-manager, vdsmupgrade > > > > gnutls.x86_64 > 3.3.29-9.el7_6 > > rhel-7-server-rpms > > gnutls-dane.x86_64 > 3.3.29-9.el7_6 > > rhel-7-server-rpms > > gnutls-utils.x86_64 > 3.3.29-9.el7_6 > > rhel-7-server-rpms > > Uploading Enabled Repositories Report > > Loaded plugins: product-id, subscription-manager > > Unable to upload Enabled Repositories Report > > > > [3] > > May 14 10:57:18 hetzner-XXXXXX systemd[1]: Starting Virtual Desktop > Server Manager... > > May 14 10:57:18 hetzner-XXXXXX vdsmd_init_common.sh[8177]: vdsm: Running > mkdirs > > May 14 10:57:18 hetzner-XXXXXX vdsmd_init_common.sh[8177]: vdsm: Running > configure_coredump > > May 14 10:57:18 hetzner-XXXXXX vdsmd_init_common.sh[8177]: vdsm: Running > configure_vdsm_logs > > May 14 10:57:18 hetzner-XXXXXX vdsmd_init_common.sh[8177]: vdsm: Running > wait_for_network > > May 14 10:57:18 hetzner-XXXXXX vdsmd_init_common.sh[8177]: vdsm: Running > run_init_hooks > > May 14 10:57:19 hetzner-XXXXXX vdsmd_init_common.sh[8177]: vdsm: Running > check_is_configured > > May 14 10:57:19 hetzner-XXXXXX sasldblistusers2[8203]: DIGEST-MD5 common > mech free > > May 14 10:57:20 hetzner-XXXXXX vdsmd_init_common.sh[8177]: abrt is > already configured for vdsm > > May 14 10:57:20 hetzner-XXXXXX vdsmd_init_common.sh[8177]: Managed > volume database is already configured > > May 14 10:57:20 hetzner-XXXXXX vdsmd_init_common.sh[8177]: lvm is > configured for vdsm > > May 14 10:57:20 hetzner-XXXXXX vdsmd_init_common.sh[8177]: libvirt is > already configured for vdsm > > May 14 10:57:20 hetzner-XXXXXX vdsmd_init_common.sh[8177]: Current > revision of multipath.conf detected, preserving > > May 14 10:57:20 hetzner-XXXXXX vdsmd_init_common.sh[8177]: vdsm: Running > validate_configuration > > May 14 10:57:20 hetzner-XXXXXX vdsmd_init_common.sh[8177]: SUCCESS: ssl > configured to true. No conflicts > > May 14 10:57:20 hetzner-XXXXXX vdsmd_init_common.sh[8177]: vdsm: Running > prepare_transient_repository > > May 14 10:57:21 hetzner-XXXXXX vdsmd_init_common.sh[8177]: vdsm: Running > syslog_available > > May 14 10:57:21 hetzner-XXXXXX vdsmd_init_common.sh[8177]: vdsm: Running > nwfilter > > May 14 10:57:21 hetzner-XXXXXX vdsmd_init_common.sh[8177]: vdsm: Running > dummybr > > May 14 10:57:21 hetzner-XXXXXX vdsmd_init_common.sh[8177]: vdsm: Running > tune_system > > May 14 10:57:21 hetzner-XXXXXX vdsmd_init_common.sh[8177]: vdsm: Running > test_space > > May 14 10:57:21 hetzner-XXXXXX vdsmd_init_common.sh[8177]: vdsm: Running > test_lo > > May 14 10:57:21 hetzner-XXXXXX systemd[1]: Started Virtual Desktop > Server Manager. > > May 14 10:57:22 hetzner-XXXXXX vdsm[8252]: WARN unhandled write event > > May 14 10:57:22 hetzner-XXXXXX vdsm[8252]: WARN Not ready yet, ignoring > event '|virt|VM_status|4f28af23-dd7e-413e-a331-1875f4dd18b3' > args={'4f28af23-dd7e-413e-a331-1875f4dd18b3': {'status': 'Down', > 'displayInfo': [{'tlsPort': '-1', 'ipAddress': '0', 'type': 'vnc', 'port': > '-1'}], 'hash': '-8231387692555228201', 'exitMessage': 'VM terminated with > error', 'cpuUser': '0.00', 'monitorResponse': '0', 'vmId': > '4f28af23-dd7e-413e-a331-1875f4dd18b3', 'exitReason': 1, 'cpuUsage': > '0.00', 'elapsedTime': '8420', 'cpuSys': '0.00', 'timeOffset': '0', > 'clientIp': '', 'exitCode': 1}} > > May 14 10:57:22 hetzner-XXXXXX vdsm[8252]: WARN MOM not available. > > May 14 10:57:22 hetzner-XXXXXX vdsm[8252]: WARN MOM not available, KSM > stats will be missing. > > May 14 10:58:47 hetzner-XXXXXX vdsm[8252]: WARN File: > /var/lib/libvirt/qemu/channels/4f28af23-dd7e-413e-a331-1875f4dd18b3.com.redhat.rhevm.vdsm > already removed > > May 14 10:58:47 hetzner-XXXXXX vdsm[8252]: WARN File: > /var/lib/libvirt/qemu/channels/4f28af23-dd7e-413e-a331-1875f4dd18b3.org.qemu.guest_agent.0 > already removed > > May 14 11:00:52 hetzner-XXXXXX vdsm[8252]: ERROR ssl handshake: > SSLError, address: ::ffff:192.168.111.10 > > May 14 11:01:54 hetzner-XXXXXX vdsm[8252]: ERROR ssl handshake: > SSLError, address: ::ffff:192.168.111.10 > > May 14 11:02:06 hetzner-XXXXXX vdsm[8252]: ERROR ssl handshake: > SSLError, address: ::ffff:192.168.111.10 > > May 14 11:05:18 hetzner-XXXXXX vdsm[8252]: ERROR ssl handshake: > SSLError, address: ::ffff:192.168.111.10 > > May 14 11:05:29 hetzner-XXXXXX vdsm[8252]: ERROR ssl handshake: > SSLError, address: ::ffff:192.168.111.10 > > May 14 11:08:41 hetzner-XXXXXX vdsm[8252]: ERROR ssl handshake: > SSLError, address: ::ffff:192.168.111.10 > > May 14 11:08:53 hetzner-XXXXXX vdsm[8252]: ERROR ssl handshake: > SSLError, address: ::ffff:192.168.111.10 > > May 14 11:12:04 hetzner-XXXXXX vdsm[8252]: ERROR ssl handshake: > SSLError, address: ::ffff:192.168.111.10 > > May 14 11:12:16 hetzner-XXXXXX vdsm[8252]: ERROR ssl handshake: > SSLError, address: ::ffff:192.168.111.10 > > May 14 11:15:28 hetzner-XXXXXX vdsm[8252]: ERROR ssl handshake: > SSLError, address: ::ffff:192.168.111.10 > > May 14 11:15:40 hetzner-XXXXXX vdsm[8252]: ERROR ssl handshake: > SSLError, address: ::ffff:192.168.111.10 > > _______________________________________________ > > Users mailing list -- [email protected] > > To unsubscribe send an email to [email protected] > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > > List Archives: > https://lists.ovirt.org/archives/list/[email protected]/message/DOD7T7DL55TR5LTDCAHA64464WQBV5QX/ > > > > -- > Didi > _______________________________________________ > Users mailing list -- [email protected] > To unsubscribe send an email to [email protected] > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/[email protected]/message/VVT3XFYQPSBQUEK76NEQOMSZTARQ4KJR/ > -- Simone Tiraboschi He / Him / His Principal Software Engineer Red Hat <https://www.redhat.com/> [email protected] @redhatjobs <https://twitter.com/redhatjobs> redhatjobs <https://www.facebook.com/redhatjobs> @redhatjobs <https://instagram.com/redhatjobs> <https://red.ht/sig> <https://redhat.com/summit>
_______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/HQNQEB5AVBZARNAXZ5TQMKZIAZPP2OGD/
