[ovirt-users] Re: Hosted-engine restore failing when migrating to new storage domain
Thank you so much, Gianluca! Yes, the source and target environments are the
same version.
I'm not able to find /root/DisableFenceAtStartupInSec.txt anywhere, but maybe
that's because at this point I've reverted to the original hosted_engine?
Here is the output of the commands you sent:
--
[root@lnxvirt-engine ~]# engine-config -g DisableFenceAtStartupInSec
Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false
DisableFenceAtStartupInSec: 300 version: general
[root@lnxvirt-engine ~]# set -euo pipefail && engine-config -g
DisableFenceAtStartupInSec | cut -d' ' -f 2
Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false
300
engine=# select * from vdc_options where
option_name='DisableFenceAtStartupInSec';
option_id |option_name | option_value | version | default_value
---++--+-+---
45 | DisableFenceAtStartupInSec | 300 | general | 300
(1 row)
--
I also tried the following from the host I tried running the restore on.
--
[root@lnxvirt07 ~]# set -euo pipefail && echo "Picked up JAVA_TOOL_OPTIONS:
-Dcom.redhat.fips=false
DisableFenceAtStartupInSec: 300 version: general" | cut -d' ' -f 2
up
300
--
Any additional questions or suggestions would be greatly appreciated.
Thanks again,
Devin
On Oct 14, 2023, at 12:30 PM, Gianluca Cecchi wrote:
On Sat, Oct 14, 2023 at 5:53 PM Devin A. Bougie
mailto:devin.bou...@cornell.edu>> wrote:
Hello,
We have a functioning oVirt 4.5.4 cluster running on fully-updated EL9.2 hosts.
We are trying to migrate the self-hosted engine to a new iSCSI storage domain
using the existing hosts, following the documented procedure:
- set the cluster into global maintenance mode
- backup the engine using "engine-backup --scope=all --mode=backup
--file=backup.bck --log=backuplog.log"
- shutdown the engine
- restore the engine using "hosted-engine --deploy --4
--restore-from-file=backup.bck"
This almost works, but fails with the attached log file. Any help or
suggestions would be greatly appreciated, including alternate procedures for
migrating a self-hosted engine from one domain to another.
Many thanks,
Devin
If I'm right, the starting error seems to be this one:
2023-10-14 11:06:16,529-0400 ERROR
otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:113
fatal: [local
host -> 192.168.1.25]: FAILED! => {"changed": true, "cmd": "set -euo pipefail
&& engine-config -g DisableFenceAtStartupInSec | c
ut -d' ' -f2 > /root/DisableFenceAtStartupInSec.txt", "delta":
"0:00:01.495195", "end": "2023-10-14 11:06:16.184479", "msg": "no
n-zero return code", "rc": 1, "start": "2023-10-14 11:06:14.689284", "stderr":
"Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=f
alse", "stderr_lines": ["Picked up JAVA_TOOL_OPTIONS:
-Dcom.redhat.fips=false"], "stdout": "", "stdout_lines": []}
As the return code is 1 ("rc": 1,) and determines the failure of the playbook,
possibly the old environment doesn't have DisableFenceAtStartupInSec engine
config property correctly set and/or the "cut" command fails... Or some other
problem with that config parameter. Can you verify what it put into
/root/DisableFenceAtStartupInSec.txt?
I have only a 4.4.10 env at hand and on it:
[root@ovengine01 ~]# engine-config -g DisableFenceAtStartupInSec
Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false
DisableFenceAtStartupInSec: 300 version: general
[root@ovengine01 ~]#
[root@ovengine01 ~]# set -euo pipefail && engine-config -g
DisableFenceAtStartupInSec | cut -d' ' -f 2
Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false
300
[root@ovengine01 ~]#
what is the output of this command on your old env:
engine-config -g DisableFenceAtStartupInSec
?
Are the source and target environments the same version?
If you have access to your old env could you also run this query on engine
database:
select * from vdc_options where option_name='DisableFenceAtStartupInSec';
eg this way
[root@ovengine01 ~]# su - postgres
[postgres@ovengine01 ~]$ psql engine
psql (12.9)
Type "help" for help.
engine=# select * from vdc_options where
option_name='DisableFenceAtStartupInSec';
option_id |option_name | option_value | version | default_value
---++--+-+---
40 | DisableFenceAtStartupInSec | 300 | general | 300
(1 row)
engine=#
Gianluca
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/7FPSGAMPCTNYWQIA3MWFJM5QOBYM3VSC/
[ovirt-users] Re: Hosted-engine restore failing when migrating to new storage domain
On Sat, Oct 14, 2023 at 5:53 PM Devin A. Bougie
wrote:
> Hello,
>
> We have a functioning oVirt 4.5.4 cluster running on fully-updated EL9.2
> hosts. We are trying to migrate the self-hosted engine to a new iSCSI
> storage domain using the existing hosts, following the documented procedure:
> - set the cluster into global maintenance mode
> - backup the engine using "engine-backup --scope=all --mode=backup
> --file=backup.bck --log=backuplog.log"
> - shutdown the engine
> - restore the engine using "hosted-engine --deploy --4
> --restore-from-file=backup.bck"
>
> This almost works, but fails with the attached log file. Any help or
> suggestions would be greatly appreciated, including alternate procedures
> for migrating a self-hosted engine from one domain to another.
>
> Many thanks,
> Devin
If I'm right, the starting error seems to be this one:
2023-10-14 11:06:16,529-0400 ERROR
otopi.ovirt_hosted_engine_setup.ansible_utils
ansible_utils._process_output:113 fatal: [local
host -> 192.168.1.25]: FAILED! => {"changed": true, "cmd": "set -euo
pipefail && engine-config -g DisableFenceAtStartupInSec | c
ut -d' ' -f2 > /root/DisableFenceAtStartupInSec.txt", "delta":
"0:00:01.495195", "end": "2023-10-14 11:06:16.184479", "msg": "no
n-zero return code", "rc": 1, "start": "2023-10-14 11:06:14.689284",
"stderr": "Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=f
alse", "stderr_lines": ["Picked up JAVA_TOOL_OPTIONS:
-Dcom.redhat.fips=false"], "stdout": "", "stdout_lines": []}
As the return code is 1 ("rc": 1,) and determines the failure of the
playbook, possibly the old environment doesn't have
DisableFenceAtStartupInSec engine config property correctly set and/or the
"cut" command fails... Or some other problem with that config parameter.
Can you verify what it put into /root/DisableFenceAtStartupInSec.txt?
I have only a 4.4.10 env at hand and on it:
[root@ovengine01 ~]# engine-config -g DisableFenceAtStartupInSec
Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false
DisableFenceAtStartupInSec: 300 version: general
[root@ovengine01 ~]#
[root@ovengine01 ~]# set -euo pipefail && engine-config -g
DisableFenceAtStartupInSec | cut -d' ' -f 2
Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false
300
[root@ovengine01 ~]#
what is the output of this command on your old env:
engine-config -g DisableFenceAtStartupInSec
?
Are the source and target environments the same version?
If you have access to your old env could you also run this query on engine
database:
select * from vdc_options where option_name='DisableFenceAtStartupInSec';
eg this way
[root@ovengine01 ~]# su - postgres
[postgres@ovengine01 ~]$ psql engine
psql (12.9)
Type "help" for help.
engine=# select * from vdc_options where
option_name='DisableFenceAtStartupInSec';
option_id |option_name | option_value | version |
default_value
---++--+-+---
40 | DisableFenceAtStartupInSec | 300 | general | 300
(1 row)
engine=#
Gianluca
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/PC3HGYPF2J3FPSF4CAZQCQ6PHESL64N7/
[ovirt-users] Re: Hosted-engine restore failing
I was able to work around this by editing /usr/share/ovirt-hosted-engine-setup/he_ansible/callback_plugins/2_ovirt_logger.py and changing: from collections import Callable to: from collections.abc import Callable Thanks again for taking a look, Devin > On Oct 11, 2023, at 12:17 PM, Devin A. Bougie > wrote: > > Hi Jorge, > > Please see below for a full package listing. We are running on fully updated > RHEL9.2 hosts, and are simply trying to migrate to a new hosted engine > storage domain (not trying to replace or upgrade any of the underlying hosts). > > If there's a way to accomplish this without the full "engine-backup" from the > hosted engine followed by "hosted-engine --deploy --restore..." from a host, > that would work too. > > Many thanks for taking a look, > Devin > > -- > [root@lnxvirt01 ~]# rpm -qa | grep -e python -e ovirt > python3-pip-wheel-21.2.3-6.el9.noarch > python3-dbus-1.2.18-2.el9.x86_64 > python3-six-1.15.0-9.el9.noarch > python3-dasbus-1.4-5.el9.noarch > python3-idna-2.10-7.el9.noarch > python3-argcomplete-1.12.0-5.el9.noarch > libpeas-loader-python3-1.30.0-4.el9.x86_64 > python3-distro-1.5.0-7.el9.noarch > python3-dateutil-2.8.1-6.el9.noarch > python3-libcomps-0.1.18-1.el9.x86_64 > python3-chardet-4.0.0-5.el9.noarch > python3-ptyprocess-0.6.0-12.el9.noarch > python3-pexpect-4.8.0-7.el9.noarch > python3-pysocks-1.7.1-12.el9.noarch > python3-urllib3-1.26.5-3.el9.noarch > python3-pyyaml-5.4.1-6.el9.x86_64 > python3-systemd-234-18.el9.x86_64 > libcap-ng-python3-0.8.2-7.el9.x86_64 > python3-cups-2.0.1-10.el9.x86_64 > python3-enchant-3.2.0-5.el9.noarch > python3-louis-3.16.1-4.el9.noarch > python3-ply-3.11-14.el9.noarch > python3-pycparser-2.20-6.el9.noarch > python3-cffi-1.14.5-5.el9.x86_64 > python3-pyxdg-0.27-3.el9.noarch > python3-pyatspi-2.38.1-3.el9.noarch > python3-gpg-1.15.1-6.el9.x86_64 > python3-libreport-2.15.2-6.el9.alma.x86_64 > python-srpm-macros-3.9-52.el9.noarch > python3-speechd-0.10.2-4.el9.x86_64 > python3-brlapi-0.8.2-4.el9.x86_64 > ibus-anthy-python-1.5.13-1.el9.noarch > python3-audit-3.0.7-103.el9.x86_64 > python-rpm-macros-3.9-52.el9.noarch > python3-rpm-macros-3.9-52.el9.noarch > python3-pip-21.2.3-6.el9.noarch > python3-pyparsing-2.4.7-9.el9.noarch > python3-packaging-20.9-5.el9.noarch > python3-rpm-generators-12-8.el9.noarch > python3-lxml-4.6.5-3.el9.x86_64 > python3-gobject-base-3.40.1-6.el9.x86_64 > python3-gobject-base-noarch-3.40.1-6.el9.noarch > python3-cairo-1.20.1-1.el9.x86_64 > python3-gobject-3.40.1-6.el9.x86_64 > python3-pyudev-0.22.0-6.el9.noarch > python3-cryptography-36.0.1-2.el9.x86_64 > python3-sanlock-3.8.4-4.el9.x86_64 > python3-docutils-0.16-6.el9.noarch > python3-decorator-4.4.2-6.el9.noarch > python3-pyasn1-0.4.8-6.el9.noarch > python3-pwquality-1.4.4-8.el9.x86_64 > python3-ovirt-setup-lib-1.3.3-1.el9.noarch > python3-augeas-0.5.0-25.el9.noarch > ovirt-vmconsole-1.0.9-1.el9.noarch > python3-otopi-1.10.3-1.el9.noarch > python3-ioprocess-1.4.2-1.202111071752.git53786ff.el9.x86_64 > python3-lockfile-0.12.2-2.el9s.noarch > python3-daemon-2.3.0-1.el9s.noarch > python3-ovirt-engine-lib-4.5.4-1.el9.noarch > python3-pynacl-1.4.0-2.el9s.x86_64 > python3-sortedcontainers-2.3.0-2.el9s.noarch > vdsm-python-4.50.3.4-1.el9.noarch > python3-bcrypt-3.1.7-7.el9s.x86_64 > python3-paramiko-2.7.2-4.el9s.noarch > ovirt-engine-setup-base-4.5.4-1.el9.noarch > python3-pbr-5.6.0-1.el9s.noarch > python3-pytz-2021.1-4.el9.noarch > python3-greenlet-1.1.2-3.el9.x86_64 > python3-qrcode-core-6.1-12.el9.noarch > python3-pyusb-1.0.2-13.el9.noarch > python3-pyasn1-modules-0.4.8-6.el9.noarch > python3-netifaces-0.10.6-15.el9.x86_64 > python3-gssapi-1.6.9-5.el9.x86_64 > python3-msgpack-1.0.3-2.el9s.x86_64 > python3-extras-1.0.0-15.el9s.noarch > python3-fixtures-3.0.0-27.el9s.noarch > python3-testtools-2.5.0-2.el9s.noarch > ovirt-hosted-engine-ha-2.5.0-1.el9.noarch > python3-yubico-1.3.3-7.el9.noarch > python3-babel-2.9.1-2.el9.noarch > python3-inotify-0.9.6-25.el9.noarch > python3-ethtool-0.15-2.el9.x86_64 > python3-resolvelib-0.5.4-5.el9.noarch > python3-prettytable-0.7.2-27.el9.noarch > python3-jwcrypto-0.8-4.el9.noarch > ovirt-vmconsole-host-1.0.9-1.el9.noarch > python3-yappi-1.3.1-2.el9s.x86_64 > python3-wrapt-1.13.3-2.el9s.x86_64 > python3-debtcollector-2.5.0-1.el9s.noarch > python3-oslo-context-4.1.0-1.el9s.noarch > python3-tenacity-6.3.1-1.el9s.noarch > python3-tempita-0.5.2-2.el9s.noarch > python3-stevedore-3.5.2-1.el9s.noarch > python3-rfc3986-1.5.0-1.el9s.noarch > python3-repoze-lru-0.7-10.el9s.noarch > python3-routes-2.5.1-1.el9s.noarch > python3-jmespath-0.10.0-1.el9s.noarch > python3-iso8601-0.1.13-4.el9s.noarch > python-oslo-utils-lang-4.12.3-1.el9s.noarch > python-oslo-privsep-lang-2.7.0-1.el9s.noarch > python-oslo-log-lang-4.7.0-1.el9s.noarch > python-oslo-i18n-lang-5.1.0-1.el9s.noarch > python3-oslo-i18n-5.1.0-1.el9s.noarch > python3-oslo-utils-4.12.3-1.el9s.noarch > python3-oslo-config-8.8.0-1.el9s.noarch >
