On Sat, Apr 9, 2016 at 12:07 AM, Brett I. Holcomb <biholc...@l1049h.com> wrote: > On Fri, 2016-04-08 at 21:50 +0300, Nir Soffer wrote: > > On Fri, Apr 8, 2016 at 9:33 PM, Brett I. Holcomb <biholc...@l1049h.com> > wrote: > > On Fri, 2016-04-08 at 19:25 +0300, Nir Soffer wrote: > > On Fri, Apr 8, 2016 at 7:17 PM, Brett I. Holcomb <biholc...@l1049h.com> > wrote: > > On Fri, 2016-04-08 at 11:31 +0200, Martin Sivak wrote: > > Hi, > > > I set highly available on, did not pin to any host, and also set the > watchdog which should reset if they go down but I'm not sure that will start > them if the host comes up and the VMs are not running. I'll look at the CLI > first. > > > The engine will try to keep the VM running. So if one host goes down, > it will restart the VM on some other host automatically. We will also > migrate the VM (or some other to free resources) when the current host > gets too loaded. We do not require any migration addons, it just > works. But of course we have usually more hosts in a cluster to make > this possible. > > I do not really remember what happens when all hosts are restarted > (power outage) though as that is quite special case. > > Regards > > -- > Martin Sivak > SLA / oVirt > > > > Thanks. I only have one host so who knows what will happen. I'm working on > a script that will basically emulate what VMware does - start VMS in a given > order at startup of the host/engine. I'll also file a feature request. > > > Why do you care about the order? > > Isn't it enough to restart all the vms after a host was restarted? > > Nir > > > Does the Engine start all VMs after the host is powered up and the engine > running if I set a watchdog on a VM? I know it will try to restart the VM > if it goes down once oVirt is up and running. If it will start the VMs at > reboot time that helps. > > > I think only HA vms are restarted. Adding Michal to add more info on this. > > > The reason for the order is that you need some servers such as DNS, file > servers, up first before other systems can resolve addresses or mount > shares. For Windows you need domain controllers running before the other > windows systems that are part of the domain. For applications such as Lotus > Notes the servers had to come up in the correct order. > > > Lets say you have a way to order the vms when some vms are down. > > What will happen to when dns, file servers or domain controller will crash? > Do you have to restart all the vms depending on them or they can > recover and detect that vm they depend on were restarted? > > Seems that what you need is a systemd for your data center - every > host define the host it depends on, and the system generate the > correct order to start the vms, starting vms in the same time when > possible. > > Please reply also to the list, this thread may be useful to others. > > Nir > > > Hopefully this will go to the list as i"ve been making sure users@ovirt.org > is in the To field. > > I think we need to make sure we distinguish between the start up phase and > the running phase which is what happens after everyone is up and running > happily. Based on my experience the crash of a server is considered > separately from startup. > > As mentioned in my response to the list to someone else's comment the > startup sequence is the same as people flipping switches on physical servers > following a documented procedure to start or shutdown a datacenter as we did > in the "old days". With physical boxes we had to shutdown/startup manually > and we did it in a sequence that we had written down. With virtualization > and since we had VMware we could automate that process so as we spun up the > hosts VMware started spinning up the guests in the order specified so we got > our DNS boxes up first, then others. For Active Directory we started the > domain controllers first, then other servers such as file servers, > application servers in the sequence needed for the applications to run. > > Crashing of DNS, File Server, Domain Controllers crashing after the > datacenter is up and running is handled (or should be) by redundancy of > servers or process the service provides. You have multiple DNS servers and > the resolver will try the secondary/tertiary/whatever if the primary is > down. File servers are the same. For Gluster, CephFS, MS DFS you have (or > should have setup) the ability to keep running if one of the servers goes > down. A redundant file server setup will handle a server crash. For Domain > Controllers you should have at least two (we had six in our environment) and > when one goes down the others keep the domain running by shifting the > services to others and continuing to provide authentication, etc. Generally > what we did when a domain controller crashed is fix it if possible and if it > was not fixable pull it's pieces out of the domain and spin up a new one. > Same for DNS or file servers. When they go down find out why, fix it or > replace the server, and get the service redundant again. Also, oVirt has > the watchdog function so if a VM goes down it will try and restart it. If > it can't restart then we're dealing with a crashed server which we should > have provided for in our data center design. > > My wish is to have oVirt allow us to do what VMware does and allow us to say > start/don't start these servers up in the order I specify so that when the > Engine is ready it looks at the list and begins turning on VMs in the order > specified. Shutdown is done in the reverse order). For large datacenters > with many VMS manual startup is a pain. Once it's running rely on good > practices of having redundant servers (i.e. more than one DNS), file servers > that can handle the failure of a server, multiple domain controllers which > is not something we need to burden oVirt with. Handling of failures needs > to be done by the people in charge who are supposed to design the data > center based on their risk/cost analysis. > > If I understand what you said - I'm not sure we have to define a dependancy > list where we define dependancies like we would with systemd or a package > manger. It doesn't need to be complex since just a simple list of priority: > VM id/name pairs will work. All we need is to be able to say "start these > VMs when the Engine is ready". The order or dependency is set by where I > put the VM in the list. If it's at the top start it first, then move to the > second one, and so on. There can be settings for delay between starting VMs > and how long to wait for a VM to come up before assuming it's dead and > moving on to the next. Tasking oVirt with the job of figuring VM > dependancies would be a nightmare for oVirt and whoever had to program it > <G>. We as data center administrators should be handling that task. Yes, > we manually set the list but trying to automate a dependancy chain would be > pretty difficult. > > I envision a web admin portal GUI where we define a simple list of VMs that > we want the Engine to autostart. We need to be able to move them up/down the > list. The list is stored somewhere for the Engine (database?, whatever else > the Engine has as storage areas - not really familiar with the Engine > internals) and when the Engine is up and ready to start VMs it retrieves the > list and starts at the first VM in the list and starts it, waits some time > (0-?? seconds), and then moves on to the next one. If the VM hasn't started > in the set time the Engine moves on to the next one. > > Note that Microsoft's Hyper-V also provides automatic VM startup but it is > done on a VM level where you just tell the VM to start whenthe Hypr-V > starts. If you want sequencing you have to set time delays. Auto start up > is better than nothing but Hyper-V is a nightmare trying to sequence VMs. > > I think VMware did it right in allowing both autostart and being able to > sequence the startup of VMS so it's host independent. As information VMware > also allowed delays between VM startups as does Hyper-V.
Thanks for the detailed description. Did you file a bug for this feature? Adding Moran and Doron. Nir > On Wed, Apr 6, 2016 at 9:10 PM, Brett I. Holcomb <biholc...@l1049h.com> > wrote: > > On Wed, 2016-04-06 at 13:42 -0400, Adam Litke wrote: > > On 06/04/16 01:46 -0400, Brett I. Holcomb wrote: > > In VMware we could setup guests to autostart when the host started and > define the order. Is that doable in oVirt? The only thing I've seen > is the watchdog and tell it to reset but nothing that allows me to > define who starts up when and if they autostart. I assume it's there > but I must be missing it or haven't found it in the web portal. > > > In oVirt guests aren't tied to a host by default (although you can set > them to run only on a specific host if you want). The closest thing I > can think of would be the High Availability features (VM->Edit). > oVirt will try to restart highly available VMs if they go down. You > can also set the priority for migration and restart in that pane. > Hopefully a combination of host pinning and the high availability > settings will get you close enough to where you want to be. > > Otherwise, you could always do some scripting with the ovirt REST API > using the SDK or CLI. > > > If you had the VMware migration extra add-on you could have hosts move as > needed so they were not tied to any host either but we could set a startup > order and specify auto, manual so that once the host started the VMs were > brought up as specified no matter what host they were running on. > > I am running hosted-engine deployment with the Engine VM on the host. > > I set highly available on, did not pin to any host, and also set the > watchdog which should reset if they go down but I'm not sure that will start > them if the host comes up and the VMs are not running. I'll look at the CLI > first. > > It would be nice if oVirt added this feature as it's really required for > large installations and is a help for any size installation, even small > ones. > > > > > _______________________________________________ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > > > _______________________________________________ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > > > _______________________________________________ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
