Hi Jon and Angus, I did not shutdown the VMs as Yiping Zhang said, but I have confirmed this and discussed earlier in the users list that my HA-enabled VMs got started on another suitable available host in the cluster even when I didn't have IPMI-enabled hardware and did no configuration for OOBM and Host-HA. I simply pulled the ethernet cable connecting the host to entire network (I did use just one NIC) and according to the value set in ping timeout event, the HA-enabled VMs were restarted on another available host. I tested the scenario using both the scenarios: the echo command as well as good old plugging out the NIC from the host. My VMs were successfully started on another available host after CS manager confirmed they were not reachable.
I too want to understand how the failover mechanism in CloudStack actually works. I used ACS 4.11 packages available here: http://cloudstack.apt-get.eu/centos/7/4.11/ Regards, Parth Patel On Thu, 24 May 2018 at 10:53 Paul Angus <paul.an...@shapeblue.com> wrote: > I'm afraid that is not a host crash. When shutting down the guest OS, the > CloudStack agent on the host is still able to report to the management > server that the VM has stopped. > > This is my point. VM-HA relies on the management sever communication with > the host agent. > > Kind regards, > > Paul Angus > > paul.an...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > @shapeblue > > > > > -----Original Message----- > From: Yiping Zhang <yzh...@marketo.com> > Sent: 24 May 2018 00:44 > To: users@cloudstack.apache.org > Subject: Re: 4.11 without Host-HA framework > > I can say for fact that VM's using a HA enabled service offering will be > restarted by CS on another host, assuming there are enough > capacity/resources in the cluster, when their original host crashes, > regardless that host comes back or not. > > The simplest way to test VM HA feature with a VM instance using HA enabled > service offering is to issue shutdown command in guest OS, and watching it > gets restarted by CS manager. > > On 5/23/18, 1:23 PM, "Paul Angus" <paul.an...@shapeblue.com> wrote: > > Hi Jon, > > Don't worry, TBH I'm dubious about those claiming to have VM-HA > working when a host crashes (but doesn't restart). > I'll check in with the guys that set values for host-ha when testing, > to see which ones they change and what they set them to. > > paul.an...@shapeblue.com > www.shapeblue.com > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > @shapeblue > > > > > -----Original Message----- > From: Jon Marshall <jms....@hotmail.co.uk> > Sent: 23 May 2018 21:10 > To: users@cloudstack.apache.org > Subject: Re: 4.11 without Host-HA framework > > Rohit / Paul > > > Thanks again for answering. > > > I am a Cisco guy with an ex Unix background but no virtualisation > experience and I can honestly say I have never felt this stupid before 😊 > > > I have Cloudstack working but failover is killing me. > > > When you say VM HA relies on the host telling CS the VM is down how > does that work because if you crash the host how does it tell CS anything ? > And when you say tell CS do you mean the CS manager ? > > > I guess I am just not understanding all the moving parts. I have had > HOST HA working (to an extent) although it takes a long time to failover > even after tweaking the timers but the fact that I keep finding references > to people saying even without HOST HA it should failover (and mine doesn't) > makes me think I have configured it incorrectly somewhere along the line. > > > I have configured a compute offering with HA and I am crashing the > host with the echo command as suggested but still nothing. > > > I understand what you are saying Paul about it not being a good idea > to rely on VM HA so I will go back to Host HA and try to speed up failover > times. > > > Can I ask, from your experiences, what is a realistic fail over time > for CS ie. if a host fails for example ? > > > Jon > > > > > ________________________________ > From: Paul Angus <paul.an...@shapeblue.com> > Sent: 23 May 2018 19:55 > To: users@cloudstack.apache.org > Subject: RE: 4.11 without Host-HA framework > > Jon, > > As Rohit says, it is very important to understand the difference > between VM HA and host HA. > VM HA relies on the HOST telling CloudStack that the VM is down on > order for CloudStack start it again (wherever that ends up being). > Any sequence of events that ends up with VM HA restarting the VM when > CloudStack can't contact the host is luck/fluke/unreliable/bad(tm) > > The purpose of Host HA was to create a reliable mechanism to determine > that a host has 'crashed' and that the VMs within it are inoperative. Then > take appropriate action, including ultimately telling VM HA to restart the > VM elsewhere. > > > > > > paul.an...@shapeblue.com > www.shapeblue.com<http://www.shapeblue.com> > Shapeblue - The CloudStack Company<http://www.shapeblue.com/> > www.shapeblue.com > ShapeBlue are the largest independent integrator of CloudStack > technologies globally and are specialists in the design and implementation > of IaaS cloud infrastructures for both private and public cloud > implementations. > > > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > > > > -----Original Message----- > From: Rohit Yadav <rohit.ya...@shapeblue.com> > Sent: 23 May 2018 10:45 > To: users@cloudstack.apache.org > Subject: Re: 4.11 without Host-HA framework > > Jon, > > > In the VM's compute offering, make sure that HA is ticked/enabled. > Then use that HA-enabled VM offering while deploying a VM. Around testing - > it depends how you're crashing. In case of KVM, you can try to cause host > crash (example: echo c > /proc/sysrq-trigger) and see if HA-enabled VMs > gets started on a different host. > > > - Rohit > > <https://cloudstack.apache.org> > > > > ________________________________ > From: Jon Marshall <jms....@hotmail.co.uk> > Sent: Tuesday, May 22, 2018 8:28:06 PM > To: users@cloudstack.apache.org > Subject: Re: 4.11 without Host-HA framework > > Hi Rohit > > > Thanks for responding. > > > I have not had much luck with HA at all. I crash a server and nothing > happens in terms of VMs migrating to another host. Monitoring the > management log file it seems the management server recognises the host has > stopped responding to pings but doesn't think it has to do anything. > > > I am currently running v4.11 with basic network but 3 separate NICs, > one for management, one for storage and one for VMs themselves. > > > Should it make it any difference ie. would it be worth trying to run > management and storage over the same NIC ? > > > I am just lost as to why I see no failover at all whereas others are > reporting it works fine. > > > Jon > > > ________________________________ > From: Rohit Yadav <rohit.ya...@shapeblue.com> > Sent: 22 May 2018 12:12 > To: users@cloudstack.apache.org > Subject: Re: 4.11 without Host-HA framework > > Hi Jon, > > > Yes, Host-HA is different from VM-HA and without Host HA enabled a HA > enabled VM should be recovered/run on a different host when it crashes. > Historically the term 'HA' in CloudStack is used around high availability > of a VM. > > > Host HA as the name tries to imply is around HA of a physical > hypervisor host by means of out-of-band management technologies such as > ipmi and currently supporting ipmi as OOBM and KVM hosts with NFS storage. > > > - Rohit > > <https://cloudstack.apache.org> > [https://cloudstack.apache.org/images/monkey-144.png]< > https://cloudstack.apache.org/> > > Apache CloudStack: Open Source Cloud Computing< > https://cloudstack.apache.org/> > cloudstack.apache.org > CloudStack is open source cloud computing software for creating, > managing, and deploying infrastructure cloud services > > > > > > > ________________________________ > From: Jon Marshall <jms....@hotmail.co.uk> > Sent: Monday, May 21, 2018 8:36:04 PM > To: users@cloudstack.apache.org > Subject: 4.11 without Host-HA framework > > I keep seeing conflicting information about this in the mailing lists > and in blogs etc. > > If I run 4.11 without enabling Host HA framework should HA still work > if I crash a compute node because my understanding was the new framework > was added for certain cases only. > > It doesn't work for me but I can find a number of people saying you > don't need to enable the new framework for it to work. > > Thanks > > Jon > > rohit.ya...@shapeblue.com > www.shapeblue.com<http://www.shapeblue.com> > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > > > > rohit.ya...@shapeblue.com > www.shapeblue.com<http://www.shapeblue.com> > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue > > > > > >
