Hi,
A couple of months ago I posted a patch for bug 
https://launchpad.net/bugs/1463688. The issue is as follows: the periodic task 
detects that the instance state does not match the state on the hypervisor and 
it shuts down the running VM. There are a number of ways that this may happen 
and I will try and explain:

  1.  Vmware driver example: a host where the instances are running goes down. 
This could be a power outage, host failure, etc. The first iteration of the 
perdioc task will determine that the actual instacne is down. This will update 
the state of the instance to DOWN. The VC has the ability to do HA and it will 
start the instance up and running again. The next iteration of the periodic 
task will determine that the instance is up and the compute manager will stop 
the instance.
  2.  All drivers. The tenant decides to do a reboot of the instance and that 
coincides with the periodic task state validation. At this point in time the 
instance will not be up and the compute node will update the state of the 
instance as DWON. Next iteration the states will differ and the instance will 
be shutdown

Basically the issue hit us with our CI and there was no CI running for a couple 
of hours due to the fact that the compute node decided to shutdown the running 
instances. The hypervisor should be the source of truth and it should not be 
the compute node that decides to shutdown instances. I posted a patch to deal 
with this https://review.openstack.org/#/c/190047/. Which is the reason for 
this mail. The patch is backwards compatible so that the existing deployments 
and random shutdown continues as it works today and the admin now has an 
ability just to do a log if there is a inconsistency.

We do not want to disable the periodic task as knowing the current state of the 
instance is very important and has a ton of value, we just do not want the 
periodic to task to shut down a running instance.

Thanks
Gary
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to