On Mon, Jan 6, 2020 at 4:19 PM Bob Franzke <[email protected]> wrote: > > So I am getting the impression that without a working ovirt engine, you are > sort of cooked from being able to control VMs such that your whole > organization can potentially come down to the availability of a single > machine? Is this really correct?
Correct. This does not mean that the engine itself is necessarily critical - if it's down, your VMs should still be ok. If _controlling_ VMs is considered critical for you, then yes - you do need to make sure your engine is alive and well. > Are there HA options available for the engine server itself? The standard option is using hosted-engine with several hosts - you get HA out-of-the-box. I also heard about people using standalone active/standby clustering/HA solutions for the engine. > > -----Original Message----- > From: Yedidyah Bar David ([email protected]) <[email protected]> > Sent: Monday, January 6, 2020 12:57 AM > To: Bob Franzke <[email protected]> > Cc: users <[email protected]> > Subject: [ovirt-users] Re: OVirt Engine Server Died - Steps for Rebuilding > the Ovirt Engine System > > On Mon, Jan 6, 2020 at 12:00 AM Bob Franzke <[email protected]> wrote: > > > > Thanks for the reply here. Still waiting on a server to rebuild this with. > > Should be here tomorrow. The engine was running on bare metal server, and > > was not a VM. > > > > In the mean time we had a few of the VMs go dark for some reason. I > > discovered the vdsm-client commands and tried figuring out what happened. > > Is there any way I can start a VM via command line on one of the VM hosts? > > Is the vdsm-client command the way to do this without a working engine? > > It is, in principle, but that's not supported and is risky - because the > engine will not know what you do. > > See also e.g.: > > https://www.ovirt.org/develop/release-management/features/integration/cockpit.html > > > > > -----Original Message----- > > From: Yedidyah Bar David ([email protected]) <[email protected]> > > Sent: Tuesday, December 24, 2019 1:50 AM > > To: Bob Franzke <[email protected]> > > Cc: users <[email protected]> > > Subject: [ovirt-users] Re: OVirt Engine Server Died - Steps for > > Rebuilding the Ovirt Engine System > > > > On Mon, Dec 23, 2019 at 7:08 PM Bob Franzke <[email protected]> wrote: > > > > > > > Which nightly backups? Do they run engine-backup? > > > > > > Yes sorry. The backups are the backups created when running the > > > engine-backup script. So I have the files and the DB backed up and off > > > onto different storage. I just grabbed a copy of the entire /etc > > > directory as well just in case there was something needed in there that > > > is not included in the engine-backup solution. > > > > > > > In either case, assuming this is a production env, I suggest to first > > > > test on a separate env to see how it all looks like. > > > > > > This is a production environment. My plan is to get a new server ordered > > > and built, removing the old server from the equation (old server is old > > > and needs to be replaced anyway). Then rebuild the Ovirt bits and restore > > > the data from my backups. > > > > I assume, from your first post, that you refer to the host running the > > engine, and that this is a standalone engine, not hosted-engine. > > Right? Meaning, it's running on bare-metal, not inside a VM managed by > > itself. > > > > For testing you can try stuff on an isolated VM somewhere, no need to wait > > for your new server to arrive. > > > > > > > > I just more needed a quick set up steps to take here. From what I gather > > > I need to basically: > > > > > > 1. reinstall CentOS > > > 2. Reconfigure storage (this server has several ISCSI LUNs its attached > > > to currently. I don’t know if they are required for this or what). > > > > I obviously have no idea what is your storage design and requirements, but > > this is largely a local matter, unrelated to the hosts that run VMs. The > > engine machine's storage is (normally) not used for that, only for the > > engine itself (and its db, etc.). > > > > > 3. Install PostGreSQL (maybe? Or does the ovirt engine script do > > > this for you?) 3. Install Ovirt/run ovirt-engine script maybe? > > > > Add relevant repo, by installing relevant ovirt-releast* package (see the > > web site), and then 'yum install ovirt-engine' - this should grab for you > > postgresql etc. > > > > > 4. Restore DB and data > > > > Yes. Run basically 'engine-backup --mode=restore' and then 'engine-setup'. > > Please check the backup/restore documentation on the web site. > > If your current engine used only defaults (meaning, engine+dwh+their DBs > > all on the engine machine, provisioned by engine-setup), then the restore > > command should be something like: > > > > engine-backup --mode=restore --file=your-backup-file > > --provision-all-databases > > > > Again, please test on a test VM somewhere, and make sure it's isolated > > - that it can't reach your hosts and start to manage them (unless that's > > what you want, of course). > > > > > > > > I am not sure the details of the list outlined above (what to run where, > > > etc.). I am looking for consultants to help me out here as its clear I am > > > a bit behind the curve on this one. So far not much has worked out on > > > that front. Does the above list seem reasonable in terms of needed steps > > > to get this going again? > > > > See above. > > > > For consultants, you might want to check: > > > > https://www.ovirt.org/community/user-stories/users-and-providers.html > > > > And/or post again to the list with a subject line that's more likely to > > attract them ("Looking for an oVirt consultant..."). > > > > Good luck and best regards, > > > > > > > > > > > -----Original Message----- > > > From: Yedidyah Bar David ([email protected]) <[email protected]> > > > Sent: Sunday, December 22, 2019 1:58 AM > > > To: [email protected] > > > Cc: users <[email protected]> > > > Subject: [ovirt-users] Re: OVirt Engine Server Died - Steps for > > > Rebuilding the Ovirt Engine System > > > > > > On Fri, Dec 20, 2019 at 8:55 PM <[email protected]> wrote: > > > > > > > > Full disclosure here.....I am not an Ovirt Expert. I am a network > > > > Engineer that has been forced to take over sysadmin duties for a > > > > departed co-worker. I have little experience with Ovirt so apologies up > > > > front for anything I say that comes across as stupid or "RTM" > > > > questions. Normally I would do just that but I am in a bind and am > > > > trying to figure this out quickly. We have an OVirt installation setup > > > > that consists of 4 nodes and a server that hosts the ovirt-engine all > > > > running CentOS 7. The server that hosts the engine has a pair of > > > > failing hard drives and I need to replace the hardware ASAP. Need to > > > > outline the steps needed to build a new server to serve as and replace > > > > the ovirt engine server. I have backed up the entire /etc directory and > > > > the backups being done nightly by the engine itself. > > > > > > Which nightly backups? Do they run engine-backup? > > > > > > > I also backed up the iscsi info and took a printout of all the disk > > > > arrangement . The disk has gotten so bad at this point that the DB > > > > won't back up any longer. Get fatal:backup failed error when > > > > trying to run the ovirt backup tool. Also the Ovirt management site > > > > is not rendering and I am not sure why. > > > > > > > > Is there anything else I need to make sure I backup in order to migrate > > > > the engine from one server to another? > > > > > > Generally speaking, if you used engine-backup for backups, it should be > > > enough - it backs up all it needs from /etc. > > > > > > If you didn't use that, /etc won't be enough. You also need a database > > > backup. > > > > > > If you do not have a backup of the database, you'll need to create a new > > > engine from scratch. You can then import the existing storage domains and > > > add the hosts. This will require downtime, and you'll loose some stuff, > > > so if you do have an engine-backup backup, better use that. > > > > > > In either case, assuming this is a production env, I suggest to first > > > test on a separate env to see how it all looks like. > > > > > > > Also, until I can get the engine running again, is there any tool > > > > available to manage the VMs on the hosts themselves. The VMs on the > > > > hosts are running but need a way to manage them if needed in case > > > > something happens while the engine is being repaired. > > > > > > Some management is possible via cockpit. It's much less than what the > > > engine allows. > > > > > > If you search the list archives, you can find suggestions by people to > > > directly use libvirt/virsh after poking a bit inside your storage domain. > > > I'd not recommend doing that, unless you know very well what you are > > > doing and have no other solution (e.g. if storage is corrupted enough so > > > that import to a new engine fails). > > > > > > > Any info on this as well as what to backup and the steps to move the > > > > engine from one server to another would be much much appreciated. > > > > > > You can search the site for backup, restore, and import storage domain, > > > and should find the relevant pages. Please note that the pages under > > > /develop are written during development and are usually not updated after > > > a feature is complete. The official documentation is under > > > /documentation. That, in turn, is often outdated as well :-(. > > > You can use RHV docs in addition. These are more up-to-date and should be > > > 99% applicable to oVirt. > > > > > > > Sorry I know this a real RTM type post but I am in a bind and need a > > > > solution rather quickly. Thanks in advance. > > > > > > Good luck! > > > -- > > > Didi > > > _______________________________________________ > > > Users mailing list -- [email protected] To unsubscribe send an email > > > to [email protected] Privacy > > > Statement: https://www.ovirt.org/site/privacy-policy/ > > > oVirt Code of Conduct: > > > https://www.ovirt.org/community/about/community-guidelines/ > > > List Archives: > > > https://lists.ovirt.org/archives/list/[email protected]/message/FU4AIR > > > 7S > > > CTQOQRWLPLPUH5XHDXYI4DD7/ > > > > > > > > > -- > > Didi > > _______________________________________________ > > Users mailing list -- [email protected] > > To unsubscribe send an email to [email protected] Privacy > > Statement: https://www.ovirt.org/site/privacy-policy/ > > oVirt Code of Conduct: > > https://www.ovirt.org/community/about/community-guidelines/ > > List Archives: > > https://lists.ovirt.org/archives/list/[email protected]/message/DX2JFAI6 > > T2MXOVOXUVL4QIVPSHQQBSNP/ > > > > > -- > Didi > _______________________________________________ > Users mailing list -- [email protected] > To unsubscribe send an email to [email protected] Privacy Statement: > https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/[email protected]/message/B56KRGFPAFGYO7MAF43PJXUCLDNDUSBS/ > -- Didi _______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/HHE26SHL6ZE3SVXTWZ5WQ6PSVEVKAXMX/
