[ovirt-users] Re: Trouble restoring + upgrading to ovirt 4.5 system after host crashed

2023-08-09 Thread David Johnson 
Thank you for your response.

I powered up the defunct system and confirmed that the keycloak DB was
present in the source database, but the engine-backup tool was not backing
it up.

I was able to recover with the following these steps:

1. start from clean CentOS 9
2. remove a conflicting man page rpm
2. dnf install ovirt-engine, ignore the script error
3. restore the engine database
4. back up the keycloak database with pg_dump, migrate to new engine host,
and restore the keycloak database
5. Add entries to the pg_hba for the keycloak database
6. ensure the keycloak user could connect to the keycloak database
7. run engine-setup (no errors)
8. manually add engin.cer to the java keystore

We are now running on Version 4.5.4-1.el9

*David Johnson*
*Director of Development, Maxis Technology*
844.696.2947 ext 702 (o) | 479.531.3590 (c)




*Follow us:*  

>
>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/LJ3QBRY3KXHA2V5ZUWQUFSGRAMZO7PBY/

[ovirt-users] Re: Trouble restoring + upgrading to ovirt 4.5 system after host crashed

2023-08-08 Thread Yedidyah Bar David 
Hi,

On Tue, Aug 8, 2023 at 9:21 PM David Johnson 
wrote:

> Good afternoon all,
>
> We had a confluence of events hit all at once and need help desperately.
> Our Ovirt engine system recently crashed and is unrecoverable. Due to a
> power maintenance event at the data center, 1/3 of our VM's are offline.
>
> I have recent backups from the engine created with engine-backup.
>

How do you run engine-backup for backups? What version? What OS?


>
> I installed a clean Centos 9 and followed the directions to install
> the ovirt-engine .
>
> After I restore the backup, the engine-setup fails on the keycloak
> configuration.
>
> *From clean system:*
>
> *Install: **(Observe failed scriptlet during install, but rom install
> still succeeds)*
>
> [root@ovirt2 administrator]# dnf install -y ovirt-engine
> Last metadata expiration check: 2:08:15 ago on Tue 08 Aug 2023 10:11:31 AM
> CDT.
> Dependencies resolved.
>
> =
>  Package  Architecture
>   VersionRepository
>  Size
>
> =
> Installing:
>  ovirt-engine noarch
>   4.5.4-1.el9centos-ovirt45
>  13 M
> Installing dependencies:
>  SuperLU  x86_64
>   5.3.0-2.el9epel
> 182 k
>
> (Snip ...)
>
>
>
>
>
> *  Running scriptlet: ovirt-vmconsole-1.0.9-1.el9.noarch
>
> 60/425Failed to resolve allow statement at
> /var/lib/selinux/targeted/tmp/modules/400/ovirt_vmconsole/cil:539Failed to
> resolve AST/usr/sbin/semodule:  Failed!*
>
>
This might cause a problem later on, but I do not think it's related to
your current issue.


>
> (Snip ...)
>  xmlrpc-common-3.1.3-1.1.el9.noarch
>xorg-x11-fonts-ISO8859-1-100dpi-7.5-33.el9.noarch
>   zziplib-0.13.71-9.el9.x86_64
>
> Complete!
>
>
> *Engine-restore (no visible issues):*
>
> [root@ovirt2 administrator]# engine-backup --mode=restore
> --log=restore1.log --file=Downloads/engine-2023-08-06.22.00.02.bak
> --provision-all-databases --restore-permissions
> Start of engine-backup with mode 'restore'
> scope: all
> archive file: Downloads/engine-2023-08-06.22.00.02.bak
> log file: restore1.log
> Preparing to restore:
> - Unpacking file 'Downloads/engine-2023-08-06.22.00.02.bak'
> Restoring:
> - Files
>
> --
> Please note:
>
> Operating system is different from the one used during backup.
> Current operating system: centos9
> Operating system at backup: centos8
>
>
I do not think this is the problem, but you might try as well on centos8.


>
> Apache httpd configuration will not be restored.
> You will be asked about it on the next engine-setup run.
>
> --
> Provisioning PostgreSQL users/databases:
> - user 'engine', database 'engine'
> - user 'ovirt_engine_history', database 'ovirt_engine_history'
> - user 'ovirt_engine_history_grafana' on database 'ovirt_engine_history'
>
>


> Restoring:
> - Engine database 'engine'
>   - Cleaning up temporary tables in engine database 'engine'
>   - Updating DbJustRestored VdcOption in engine database
>   - Resetting DwhCurrentlyRunning in dwh_history_timekeeping in engine
> database
>   - Resetting HA VM status
>
> --
> Please note:
>
> The engine database was backed up at 2023-08-06 22:00:19.0 -0500 .
>
> Objects that were added, removed or changed after this date, such as
> virtual
> machines, disks, etc., are missing in the engine, and will probably require
> recovery or recreation.
>
> --
> - DWH database 'ovirt_engine_history'
> - Grafana database '/var/lib/grafana/grafana.db'
>
>
No Keycloak DB restored. I guess it was not backed up, perhaps not even
configured.


> You should now run engine-setup.
> Done.
> [root@ovirt2 administrator]#
>
>
> *Engine-setup :*
>
> [root@ovirt2 administrator]# engine-setup
> [ INFO  ] Stage: Initializing
> [ INFO  ] Stage: Environment setup
>   Configuration files:
> /etc/ovirt-engine-setup.conf.d/10-packaging-jboss.conf,
> /etc/ovirt-engine-setup.conf.d/10-packaging.conf,
>   /etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf
>   Log file:
> /var/log/ovirt-engine/setup/ovirt-engine-setup-20230808124501-joveku.log
>   Version: otopi-1.10.3 (otopi-1.10.3-1.el9)
> [ INFO  ] The engine DB has been restored from a backup
>
> *[ ERROR ] Failed to execute stage 'Environment

[ovirt-users] Re: Trouble restoring + upgrading to ovirt 4.5 system after host crashed

2023-08-08 Thread David Johnson 
Good evening all,

I was able to work past this by restarting the dying machine briefly,
backing up the keycloak database with pg_dump, and migrating it to the new
system.  setup-engine seems to work ok, but it is not generating the certs
for the keycloak, now.

When I attempt to log in to the web console, I got this message:

Warning alert:PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target


engine.log records:

2023-08-08 17:56:44,507-05 INFO
>  [org.ovirt.engine.core.sso.service.NegotiateAuthService] (default task-2)
> [] User admin@ovirt@internalkeycloak-authz with profile [internalsso]
> successfully logged in with scopes : ovirt-app-admin ovirt-app-api
> ovirt-app-portal ovirt-ext=auth:sequence-priority=~
> ovirt-ext=revoke:revoke-all ovirt-ext=token-info:authz-search
> ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate
> ovirt-ext=token:password-access
> 2023-08-08 17:56:44,623-05 ERROR
> [org.ovirt.engine.core.aaa.servlet.SsoPostLoginServlet] (default task-2) []
> server_error: PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
> valid certification path to requested target
> 2023-08-08 17:56:50,216-05 INFO
>  [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService]
> (EE-ManagedScheduledExecutorService-engineThreadMonitoringThreadPool-Thread-1)
> [] Thread pool 'default' is using 0 threads out of 1, 5 threads waiting for
> tasks.


After adding engine.cer to the java keystore and restarting the engine all
returned to normal.

Thank you!

*David Johnson*




On Tue, Aug 8, 2023 at 3:30 PM David Johnson 
wrote:

> Update:
>
> I have confirmed the original ovirt version has an ovirt_engine_keycloak
> database, but the database was not backed up by the engine-backup command
>
> *David Johnson*
> *Director of Development, Maxis Technology*
> 844.696.2947 ext 702 (o) | 479.531.3590 (c)
> 
> 
> 
>
> *Follow us:*  
>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/EVFSS6RHGYC6WSRT7SL4TXQXM6PSBDA4/