[ovirt-users] Re: Hosted-engine restore failing when migrating to new storage domain
This saga is finally over, at least for now. I never succeeded in restoring our hosted engine to a new iSCSI storage domain. For one reason or another, however, I was able to restore it to a new NFS storage domain. Any advice on the benefits or downsides of using an NFS storage domain versus iSCSI for the self hosted engine would be greatly appreciated. At least for now, however, things seem to be stable. Just incase helps anyone else, here's a rough outline of the procedure I finally used. - set the cluster into global maintenance mode - backup the engine using "engine-backup --scope=all --mode=backup --file=backup.bck --log=backuplog.log" - shutdown the engine - restore the engine using "hosted-engine --deploy --4 --restore-from-file=backup.bck" - when the restore pauses waiting for the lock file to be removed, scp backup.bck to the new engine VM. - ssh into the new VM, and restore the DBs using: - engine-backup --mode=restore --file=backup.bck --provision-all-databases --scope=db - engine-backup --mode=restore --file=backup.bck --provision-all-databases --scope=dwhdb - engine-backup --mode=restore --file=backup.bck --provision-all-databases --scope=grafanadb - delete the lock file, and proceed as usual Without manual intervention, the Postgres db on the new engine VM was never initialized or setup. Thanks again for everyones attention and advice. Sincerely, Devin > On Nov 6, 2023, at 12:00 PM, Devin A. Bougie wrote: > > Does anyone know how to regenerate > /etc/ovirt-hosted-engine/hosted-engine.conf? Or where exactly I find each > field to create the file manually? > > After trying to switch from an iscsi storage domain to NFS for the new hosted > engine, I finally have the engine back up and running. However, when trying > to reinstall a host to move it to the new hosted engine domain, I get "Failed > to fetch hosted engine configuration file." > > /etc/ovirt-hosted-engine/hosted-engine.conf doesn't exist on the host I ran > "hosted-engine --deploy" on. And on all other hosts, it still references the > old hosted engine VM and storage domain. > > Thanks, > Devin > >> On Oct 25, 2023, at 3:55 PM, Devin A. Bougie >> wrote: >> >> Thanks again, Gianluca. >> >> I'm currently ssh'd into the new local engine VM, and Postgres is running. >> However, an engine DB doesn't exist? Should it at this point, and do you >> have any other suggestions of where I should look? >> >> Devin >> >> -- >> [root@lnxvirt-engine ~]# su - postgres >> Last login: Wed Oct 25 15:47:18 EDT 2023 on pts/0 >> [postgres@lnxvirt-engine ~]$ psql >> psql (12.12) >> Type "help" for help. >> >> postgres=# \l >> List of databases >> Name| Owner | Encoding | Collate |Ctype| Access >> privileges >> ---+--+--+-+-+--- >> postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | template0 | >> postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres + >> | | | | | >> postgres=CTc/postgres >> template1 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres >> + >> | | | | | >> postgres=CTc/postgres >> (3 rows) >> >> postgres=# >> -- >> >> >>> On Oct 25, 2023, at 12:40 PM, Gianluca Cecchi >>> wrote: >>> >>> >>> On Wed, Oct 25, 2023 at 5:50 PM Devin A. Bougie >>> wrote: >>> I've had a chance to try this restore again, and this time login to the >>> local (new) hosted-engine VM to verify that >>> /root/DisableFenceAtStartupInSec.txt just contains: >>> to >>> >>> And if I try the "engine-config -g DisableFenceAtStartupInSec" from the new >>> hosted-engine VM, my connection closes. >>> [root@lnxvirt-engine ~]# cat /root/DisableFenceAtStartupInSec.txt >>> to >>> [root@lnxvirt-engine ~]# set -euo pipefail && engine-config -g >>> DisableFenceAtStartupInSec >>> Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false >>> Connection to the Database failed. Please check that the hostname and port >>> number are correct and that the Database service is up and running. >>> Connection to 192.168.222.25 closed. >>> >>> Any new suggestions or more tests I can run would be greatly appreciated. >>> >>> Thanks, >>> Devin >>> >>> >>> The key thing here is that for some reason it seems it is not able to >>> connect to the database and so when "engine-config" command tries to get >>> the second field of the output (the " | cut -d' ' -f2" part) it gets the >>> "to" string here: >>> >>> Connection to the Database failed >>> and anyway it returns error with failure of the overall playbook >>> It should be investigated if there is a problem with the database itself on >>> the new engine or if for some reason the "engine-config" command is not >>> able to implicitly connect to the database >>> >> >>
[ovirt-users] Re: Hosted-engine restore failing when migrating to new storage domain
Does anyone know how to regenerate /etc/ovirt-hosted-engine/hosted-engine.conf? Or where exactly I find each field to create the file manually? After trying to switch from an iscsi storage domain to NFS for the new hosted engine, I finally have the engine back up and running. However, when trying to reinstall a host to move it to the new hosted engine domain, I get "Failed to fetch hosted engine configuration file." /etc/ovirt-hosted-engine/hosted-engine.conf doesn't exist on the host I ran "hosted-engine --deploy" on. And on all other hosts, it still references the old hosted engine VM and storage domain. Thanks, Devin > On Oct 25, 2023, at 3:55 PM, Devin A. Bougie wrote: > > Thanks again, Gianluca. > > I'm currently ssh'd into the new local engine VM, and Postgres is running. > However, an engine DB doesn't exist? Should it at this point, and do you > have any other suggestions of where I should look? > > Devin > > -- > [root@lnxvirt-engine ~]# su - postgres > Last login: Wed Oct 25 15:47:18 EDT 2023 on pts/0 > [postgres@lnxvirt-engine ~]$ psql > psql (12.12) > Type "help" for help. > > postgres=# \l > List of databases > Name| Owner | Encoding | Collate |Ctype| Access > privileges > ---+--+--+-+-+--- > postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | template0 | > postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres + > | | | | | > postgres=CTc/postgres > template1 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres > + > | | | | | > postgres=CTc/postgres > (3 rows) > > postgres=# > -- > > >> On Oct 25, 2023, at 12:40 PM, Gianluca Cecchi >> wrote: >> >> >> On Wed, Oct 25, 2023 at 5:50 PM Devin A. Bougie >> wrote: >> I've had a chance to try this restore again, and this time login to the >> local (new) hosted-engine VM to verify that >> /root/DisableFenceAtStartupInSec.txt just contains: >> to >> >> And if I try the "engine-config -g DisableFenceAtStartupInSec" from the new >> hosted-engine VM, my connection closes. >> [root@lnxvirt-engine ~]# cat /root/DisableFenceAtStartupInSec.txt >> to >> [root@lnxvirt-engine ~]# set -euo pipefail && engine-config -g >> DisableFenceAtStartupInSec >> Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false >> Connection to the Database failed. Please check that the hostname and port >> number are correct and that the Database service is up and running. >> Connection to 192.168.222.25 closed. >> >> Any new suggestions or more tests I can run would be greatly appreciated. >> >> Thanks, >> Devin >> >> >> The key thing here is that for some reason it seems it is not able to >> connect to the database and so when "engine-config" command tries to get the >> second field of the output (the " | cut -d' ' -f2" part) it gets the "to" >> string here: >> >> Connection to the Database failed >> and anyway it returns error with failure of the overall playbook >> It should be investigated if there is a problem with the database itself on >> the new engine or if for some reason the "engine-config" command is not able >> to implicitly connect to the database >> > > ___ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/XJ6PEGB2LUCIOXJ5RKA35IUMIGR6LPIF/ ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/472WX3JWJNGD4NKTF4IOUTKXOTQTHORE/
[ovirt-users] Re: Hosted-engine restore failing when migrating to new storage domain
Thanks again, Gianluca. I'm currently ssh'd into the new local engine VM, and Postgres is running. However, an engine DB doesn't exist? Should it at this point, and do you have any other suggestions of where I should look? Devin -- [root@lnxvirt-engine ~]# su - postgres Last login: Wed Oct 25 15:47:18 EDT 2023 on pts/0 [postgres@lnxvirt-engine ~]$ psql psql (12.12) Type "help" for help. postgres=# \l List of databases Name| Owner | Encoding | Collate |Ctype| Access privileges ---+--+--+-+-+--- postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | template0 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres + | | | | | postgres=CTc/postgres template1 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres + | | | | | postgres=CTc/postgres (3 rows) postgres=# -- > On Oct 25, 2023, at 12:40 PM, Gianluca Cecchi > wrote: > > > On Wed, Oct 25, 2023 at 5:50 PM Devin A. Bougie > wrote: > I've had a chance to try this restore again, and this time login to the local > (new) hosted-engine VM to verify that /root/DisableFenceAtStartupInSec.txt > just contains: > to > > And if I try the "engine-config -g DisableFenceAtStartupInSec" from the new > hosted-engine VM, my connection closes. > [root@lnxvirt-engine ~]# cat /root/DisableFenceAtStartupInSec.txt > to > [root@lnxvirt-engine ~]# set -euo pipefail && engine-config -g > DisableFenceAtStartupInSec > Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false > Connection to the Database failed. Please check that the hostname and port > number are correct and that the Database service is up and running. > Connection to 192.168.222.25 closed. > > Any new suggestions or more tests I can run would be greatly appreciated. > > Thanks, > Devin > > > The key thing here is that for some reason it seems it is not able to connect > to the database and so when "engine-config" command tries to get the second > field of the output (the " | cut -d' ' -f2" part) it gets the "to" string > here: > > Connection to the Database failed > and anyway it returns error with failure of the overall playbook > It should be investigated if there is a problem with the database itself on > the new engine or if for some reason the "engine-config" command is not able > to implicitly connect to the database > ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/XJ6PEGB2LUCIOXJ5RKA35IUMIGR6LPIF/
[ovirt-users] Re: Hosted-engine restore failing when migrating to new storage domain
On Wed, Oct 25, 2023 at 5:50 PM Devin A. Bougie wrote: > I've had a chance to try this restore again, and this time login to the > local (new) hosted-engine VM to verify that > /root/DisableFenceAtStartupInSec.txt just contains: > to > > And if I try the "engine-config -g DisableFenceAtStartupInSec" from the > new hosted-engine VM, my connection closes. > [root@lnxvirt-engine ~]# cat /root/DisableFenceAtStartupInSec.txt > to > [root@lnxvirt-engine ~]# set -euo pipefail && engine-config -g > DisableFenceAtStartupInSec > Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false > Connection to the Database failed. Please check that the hostname and port > number are correct and that the Database service is up and running. > Connection to 192.168.222.25 closed. > > Any new suggestions or more tests I can run would be greatly appreciated. > > Thanks, > Devin > > The key thing here is that for some reason it seems it is not able to connect to the database and so when "engine-config" command tries to get the second field of the output (the " | cut -d' ' -f2" part) it gets the "to" string here: Connection to the Database failed and anyway it returns error with failure of the overall playbook It should be investigated if there is a problem with the database itself on the new engine or if for some reason the "engine-config" command is not able to implicitly connect to the database ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/FR7KU3LAMEDTAZYBWJM4JO22M3FUVO65/
[ovirt-users] Re: Hosted-engine restore failing when migrating to new storage domain
I've had a chance to try this restore again, and this time login to the local (new) hosted-engine VM to verify that /root/DisableFenceAtStartupInSec.txt just contains: to And if I try the "engine-config -g DisableFenceAtStartupInSec" from the new hosted-engine VM, my connection closes. [root@lnxvirt-engine ~]# cat /root/DisableFenceAtStartupInSec.txt to [root@lnxvirt-engine ~]# set -euo pipefail && engine-config -g DisableFenceAtStartupInSec Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false Connection to the Database failed. Please check that the hostname and port number are correct and that the Database service is up and running. Connection to 192.168.222.25 closed. Any new suggestions or more tests I can run would be greatly appreciated. Thanks, Devin > On Oct 15, 2023, at 9:10 AM, Devin A. Bougie wrote: > > Hi Gianluca, > > Thanks for taking another look. I'm not sure what to make of > /var/log/ovirt-hosted-engine-setup/engine-logs-2023-10-14T14:30:39Z/log/ovirt-engine/setup/restore-backup-20231014150412.log, > but here it is. Does that explain anything to you, or give an idea of where > to look next? > > Thanks again! > Devin > > -- > 2023-10-14 11:04:12 7680: Start of engine-backup mode restore scope all file > /root/engine_backup > 2023-10-14 11:04:12 7680: OUTPUT: Start of engine-backup with mode 'restore' > 2023-10-14 11:04:12 7680: OUTPUT: scope: all > 2023-10-14 11:04:12 7680: OUTPUT: archive file: /root/engine_backup > 2023-10-14 11:04:12 7680: OUTPUT: log file: > /var/log/ovirt-engine/setup/restore-backup-20231014150412.log > 2023-10-14 11:04:12 7680: OUTPUT: Preparing to restore: > 2023-10-14 11:04:12 7680: OUTPUT: - Unpacking file '/root/engine_backup' > 2023-10-14 11:04:12 7680: Opening tarball /root/engine_backup to > /tmp/engine-backup.Onm6LsDR0g > 2023-10-14 11:04:13 7680: Verifying hash > 2023-10-14 11:04:13 7680: Verifying version > 2023-10-14 11:04:13 7680: Reading config > 2023-10-14 11:04:13 7680: Scope after checking backup content: > 2023-10-14 11:04:13 7680: SCOPE_FILES:1 > 2023-10-14 11:04:13 7680: SCOPE_ENGINE_DB:1 > 2023-10-14 11:04:13 7680: SCOPE_DWH_DB:1 > 2023-10-14 11:04:13 7680: SCOPE_CINDERLIB_DB: > 2023-10-14 11:04:13 7680: SCOPE_KEYCLOAK_DB: > 2023-10-14 11:04:13 7680: SCOPE_GRAFANA_DB:1 > 2023-10-14 11:04:13 7680: OUTPUT: Restoring: > 2023-10-14 11:04:13 7680: OUTPUT: - Files > 2023-10-14 11:04:13 7680: Restoring files > tar: var/lib/grafana/plugins/performancecopilot-pcp-app: Cannot open: File > exists > tar: Exiting with failure status due to previous errors > 2023-10-14 11:04:13 7680: FATAL: Failed restoring /etc/ovirt-engine > /etc/ovirt-engine-dwh > /etc/ovirt-provider-ovn/conf.d > /etc/ovirt-provider-ovn/logger.conf > /etc/ovirt-vmconsole > /etc/pki/ovirt-engine > /etc/pki/ovirt-vmconsole > /etc/ovirt-engine-setup.conf.d > /etc/httpd/conf.d/internalsso-openidc.conf > /etc/httpd/conf.d/ovirt-engine-grafana-proxy.conf > /etc/httpd/conf.d/ovirt-engine-root-redirect.conf > /etc/httpd/conf.d/ssl.conf > /etc/httpd/conf.d/z-ovirt-engine-proxy.conf > /etc/httpd/conf.d/z-ovirt-engine-keycloak-proxy.conf > /etc/httpd/http.keytab > /etc/httpd/conf.d/ovirt-sso.conf > /etc/yum/pluginconf.d/versionlock.list > /etc/dnf/plugins/versionlock.list > /etc/firewalld/services/ovirt-https.xml > /etc/firewalld/services/ovirt-http.xml > /etc/firewalld/services/ovirt-postgres.xml > /etc/firewalld/services/ovirt-provider-ovn.xml > /etc/firewalld/services/ovn-central-firewall-service.xml > /var/lib/openvswitch > /etc/grafana > /var/lib/ovirt-engine/content > /var/lib/ovirt-engine/setup > /var/lib/grafana/plugins > -- > >> On Oct 15, 2023, at 5:59 AM, Gianluca Cecchi >> wrote: >> >> On Sat, Oct 14, 2023 at 7:05 PM Devin A. Bougie >> wrote: >> [snip] >> Any additional questions or suggestions would be greatly appreciated. >> >> Thanks again, >> Devin >> >> >> There is another FATAL line regarding restore itself, before the message I >> pointed out in my previous message. >> Can you analyze and/or share the contents of >> /var/log/ovirt-engine/setup/restore-backup-20231014150412.log? >> >> Gianluca > ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/D7LPPXNHQV3YCVSA7CSJHRXYU3T2FWQF/
[ovirt-users] Re: Hosted-engine restore failing when migrating to new storage domain
Hi Gianluca, Thanks for taking another look. I'm not sure what to make of /var/log/ovirt-hosted-engine-setup/engine-logs-2023-10-14T14:30:39Z/log/ovirt-engine/setup/restore-backup-20231014150412.log, but here it is. Does that explain anything to you, or give an idea of where to look next? Thanks again! Devin -- 2023-10-14 11:04:12 7680: Start of engine-backup mode restore scope all file /root/engine_backup 2023-10-14 11:04:12 7680: OUTPUT: Start of engine-backup with mode 'restore' 2023-10-14 11:04:12 7680: OUTPUT: scope: all 2023-10-14 11:04:12 7680: OUTPUT: archive file: /root/engine_backup 2023-10-14 11:04:12 7680: OUTPUT: log file: /var/log/ovirt-engine/setup/restore-backup-20231014150412.log 2023-10-14 11:04:12 7680: OUTPUT: Preparing to restore: 2023-10-14 11:04:12 7680: OUTPUT: - Unpacking file '/root/engine_backup' 2023-10-14 11:04:12 7680: Opening tarball /root/engine_backup to /tmp/engine-backup.Onm6LsDR0g 2023-10-14 11:04:13 7680: Verifying hash 2023-10-14 11:04:13 7680: Verifying version 2023-10-14 11:04:13 7680: Reading config 2023-10-14 11:04:13 7680: Scope after checking backup content: 2023-10-14 11:04:13 7680: SCOPE_FILES:1 2023-10-14 11:04:13 7680: SCOPE_ENGINE_DB:1 2023-10-14 11:04:13 7680: SCOPE_DWH_DB:1 2023-10-14 11:04:13 7680: SCOPE_CINDERLIB_DB: 2023-10-14 11:04:13 7680: SCOPE_KEYCLOAK_DB: 2023-10-14 11:04:13 7680: SCOPE_GRAFANA_DB:1 2023-10-14 11:04:13 7680: OUTPUT: Restoring: 2023-10-14 11:04:13 7680: OUTPUT: - Files 2023-10-14 11:04:13 7680: Restoring files tar: var/lib/grafana/plugins/performancecopilot-pcp-app: Cannot open: File exists tar: Exiting with failure status due to previous errors 2023-10-14 11:04:13 7680: FATAL: Failed restoring /etc/ovirt-engine /etc/ovirt-engine-dwh /etc/ovirt-provider-ovn/conf.d /etc/ovirt-provider-ovn/logger.conf /etc/ovirt-vmconsole /etc/pki/ovirt-engine /etc/pki/ovirt-vmconsole /etc/ovirt-engine-setup.conf.d /etc/httpd/conf.d/internalsso-openidc.conf /etc/httpd/conf.d/ovirt-engine-grafana-proxy.conf /etc/httpd/conf.d/ovirt-engine-root-redirect.conf /etc/httpd/conf.d/ssl.conf /etc/httpd/conf.d/z-ovirt-engine-proxy.conf /etc/httpd/conf.d/z-ovirt-engine-keycloak-proxy.conf /etc/httpd/http.keytab /etc/httpd/conf.d/ovirt-sso.conf /etc/yum/pluginconf.d/versionlock.list /etc/dnf/plugins/versionlock.list /etc/firewalld/services/ovirt-https.xml /etc/firewalld/services/ovirt-http.xml /etc/firewalld/services/ovirt-postgres.xml /etc/firewalld/services/ovirt-provider-ovn.xml /etc/firewalld/services/ovn-central-firewall-service.xml /var/lib/openvswitch /etc/grafana /var/lib/ovirt-engine/content /var/lib/ovirt-engine/setup /var/lib/grafana/plugins -- > On Oct 15, 2023, at 5:59 AM, Gianluca Cecchi > wrote: > > On Sat, Oct 14, 2023 at 7:05 PM Devin A. Bougie > wrote: > [snip] > Any additional questions or suggestions would be greatly appreciated. > > Thanks again, > Devin > > > There is another FATAL line regarding restore itself, before the message I > pointed out in my previous message. > Can you analyze and/or share the contents of > /var/log/ovirt-engine/setup/restore-backup-20231014150412.log? > > Gianluca ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/UVWBRJNQ3HVKVXQ7T56ERN4MVT5BSSG5/
[ovirt-users] Re: Hosted-engine restore failing when migrating to new storage domain
On Sat, Oct 14, 2023 at 7:05 PM Devin A. Bougie wrote: [snip] > Any additional questions or suggestions would be greatly appreciated. > > Thanks again, > Devin > > There is another FATAL line regarding restore itself, before the message I pointed out in my previous message. Can you analyze and/or share the contents of /var/log/ovirt-engine/setup/restore-backup-20231014150412.log? Gianluca ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/4AWQKUBFC2V5CKHYSQ3VWAU2LFROT7ZF/
[ovirt-users] Re: Hosted-engine restore failing when migrating to new storage domain
Thank you so much, Gianluca! Yes, the source and target environments are the same version. I'm not able to find /root/DisableFenceAtStartupInSec.txt anywhere, but maybe that's because at this point I've reverted to the original hosted_engine? Here is the output of the commands you sent: -- [root@lnxvirt-engine ~]# engine-config -g DisableFenceAtStartupInSec Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false DisableFenceAtStartupInSec: 300 version: general [root@lnxvirt-engine ~]# set -euo pipefail && engine-config -g DisableFenceAtStartupInSec | cut -d' ' -f 2 Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false 300 engine=# select * from vdc_options where option_name='DisableFenceAtStartupInSec'; option_id |option_name | option_value | version | default_value ---++--+-+--- 45 | DisableFenceAtStartupInSec | 300 | general | 300 (1 row) -- I also tried the following from the host I tried running the restore on. -- [root@lnxvirt07 ~]# set -euo pipefail && echo "Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false DisableFenceAtStartupInSec: 300 version: general" | cut -d' ' -f 2 up 300 -- Any additional questions or suggestions would be greatly appreciated. Thanks again, Devin On Oct 14, 2023, at 12:30 PM, Gianluca Cecchi wrote: On Sat, Oct 14, 2023 at 5:53 PM Devin A. Bougie mailto:devin.bou...@cornell.edu>> wrote: Hello, We have a functioning oVirt 4.5.4 cluster running on fully-updated EL9.2 hosts. We are trying to migrate the self-hosted engine to a new iSCSI storage domain using the existing hosts, following the documented procedure: - set the cluster into global maintenance mode - backup the engine using "engine-backup --scope=all --mode=backup --file=backup.bck --log=backuplog.log" - shutdown the engine - restore the engine using "hosted-engine --deploy --4 --restore-from-file=backup.bck" This almost works, but fails with the attached log file. Any help or suggestions would be greatly appreciated, including alternate procedures for migrating a self-hosted engine from one domain to another. Many thanks, Devin If I'm right, the starting error seems to be this one: 2023-10-14 11:06:16,529-0400 ERROR otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:113 fatal: [local host -> 192.168.1.25]: FAILED! => {"changed": true, "cmd": "set -euo pipefail && engine-config -g DisableFenceAtStartupInSec | c ut -d' ' -f2 > /root/DisableFenceAtStartupInSec.txt", "delta": "0:00:01.495195", "end": "2023-10-14 11:06:16.184479", "msg": "no n-zero return code", "rc": 1, "start": "2023-10-14 11:06:14.689284", "stderr": "Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=f alse", "stderr_lines": ["Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false"], "stdout": "", "stdout_lines": []} As the return code is 1 ("rc": 1,) and determines the failure of the playbook, possibly the old environment doesn't have DisableFenceAtStartupInSec engine config property correctly set and/or the "cut" command fails... Or some other problem with that config parameter. Can you verify what it put into /root/DisableFenceAtStartupInSec.txt? I have only a 4.4.10 env at hand and on it: [root@ovengine01 ~]# engine-config -g DisableFenceAtStartupInSec Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false DisableFenceAtStartupInSec: 300 version: general [root@ovengine01 ~]# [root@ovengine01 ~]# set -euo pipefail && engine-config -g DisableFenceAtStartupInSec | cut -d' ' -f 2 Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false 300 [root@ovengine01 ~]# what is the output of this command on your old env: engine-config -g DisableFenceAtStartupInSec ? Are the source and target environments the same version? If you have access to your old env could you also run this query on engine database: select * from vdc_options where option_name='DisableFenceAtStartupInSec'; eg this way [root@ovengine01 ~]# su - postgres [postgres@ovengine01 ~]$ psql engine psql (12.9) Type "help" for help. engine=# select * from vdc_options where option_name='DisableFenceAtStartupInSec'; option_id |option_name | option_value | version | default_value ---++--+-+--- 40 | DisableFenceAtStartupInSec | 300 | general | 300 (1 row) engine=# Gianluca ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/7FPSGAMPCTNYWQIA3MWFJM5QOBYM3VSC/
[ovirt-users] Re: Hosted-engine restore failing when migrating to new storage domain
On Sat, Oct 14, 2023 at 5:53 PM Devin A. Bougie wrote: > Hello, > > We have a functioning oVirt 4.5.4 cluster running on fully-updated EL9.2 > hosts. We are trying to migrate the self-hosted engine to a new iSCSI > storage domain using the existing hosts, following the documented procedure: > - set the cluster into global maintenance mode > - backup the engine using "engine-backup --scope=all --mode=backup > --file=backup.bck --log=backuplog.log" > - shutdown the engine > - restore the engine using "hosted-engine --deploy --4 > --restore-from-file=backup.bck" > > This almost works, but fails with the attached log file. Any help or > suggestions would be greatly appreciated, including alternate procedures > for migrating a self-hosted engine from one domain to another. > > Many thanks, > Devin If I'm right, the starting error seems to be this one: 2023-10-14 11:06:16,529-0400 ERROR otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:113 fatal: [local host -> 192.168.1.25]: FAILED! => {"changed": true, "cmd": "set -euo pipefail && engine-config -g DisableFenceAtStartupInSec | c ut -d' ' -f2 > /root/DisableFenceAtStartupInSec.txt", "delta": "0:00:01.495195", "end": "2023-10-14 11:06:16.184479", "msg": "no n-zero return code", "rc": 1, "start": "2023-10-14 11:06:14.689284", "stderr": "Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=f alse", "stderr_lines": ["Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false"], "stdout": "", "stdout_lines": []} As the return code is 1 ("rc": 1,) and determines the failure of the playbook, possibly the old environment doesn't have DisableFenceAtStartupInSec engine config property correctly set and/or the "cut" command fails... Or some other problem with that config parameter. Can you verify what it put into /root/DisableFenceAtStartupInSec.txt? I have only a 4.4.10 env at hand and on it: [root@ovengine01 ~]# engine-config -g DisableFenceAtStartupInSec Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false DisableFenceAtStartupInSec: 300 version: general [root@ovengine01 ~]# [root@ovengine01 ~]# set -euo pipefail && engine-config -g DisableFenceAtStartupInSec | cut -d' ' -f 2 Picked up JAVA_TOOL_OPTIONS: -Dcom.redhat.fips=false 300 [root@ovengine01 ~]# what is the output of this command on your old env: engine-config -g DisableFenceAtStartupInSec ? Are the source and target environments the same version? If you have access to your old env could you also run this query on engine database: select * from vdc_options where option_name='DisableFenceAtStartupInSec'; eg this way [root@ovengine01 ~]# su - postgres [postgres@ovengine01 ~]$ psql engine psql (12.9) Type "help" for help. engine=# select * from vdc_options where option_name='DisableFenceAtStartupInSec'; option_id |option_name | option_value | version | default_value ---++--+-+--- 40 | DisableFenceAtStartupInSec | 300 | general | 300 (1 row) engine=# Gianluca ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/PC3HGYPF2J3FPSF4CAZQCQ6PHESL64N7/