[
https://issues.apache.org/jira/browse/CLOUDSTACK-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13730554#comment-13730554
]
Wido den Hollander commented on CLOUDSTACK-3954:
------------------------------------------------
Fyi, Lennert is my colleague :)
I've been writing some extra code for the HA. It's mainly a PoC for myself, but
I was thinking about running an extra Python webservice on port 8251 on the KVM
Agent which only tells which Instances are still running on the host in case
the Agent doesn't respond.
The Agent can crash due to some unforeseen reason, but the Instances might
still be running just fine.
So I wanted to create a backup where the Investigator can double-check if the
Instance is still alive without doing a ICMP ping.
It's a simple Webservice which gives back some JSON data about what Instances
are running on that host so the Investigator knows that.
> HA with Security Groups and ping disabled will cause split-brian
> ----------------------------------------------------------------
>
> Key: CLOUDSTACK-3954
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3954
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: KVM
> Affects Versions: 4.1.0
> Environment: Tested this with CS 4.1 on Ubuntu, but will probably
> exist in other versions
> Reporter: Lennert den Teuling
> Priority: Critical
> Fix For: 4.2.0
>
>
> We found out that when running CS 4.1 on KVM with Security Groups enabled +
> ping disabled (default) will cause a split-brain when agent crashes.
> How to reproduce:
> 1. Setup a Basic Zone with SG enabled
> 2. Create one or multiple HA-enabled VMs with a security group which does
> not allow ping (by default).
> 3. Kill the agent on one of the hosts
> When you do this, the HA component on the management server will restart all
> VMs on another node, even when they are running and the VM host is still
> pingable. This will likely corrupt all VMs on the host where the agent was
> stopped/killed.
> We had some issues with libvirt causing the agent to disconnect. Luckily some
> VMs allowed ping so nothing bad happened.
> Temporary fix:
> Ensure at least one of the running VMs on each hosts allows ping, so the HA
> manager will be able to ping it and will not HA the host.
> I'm not sure yet why this happens, but wanted to file this bug so people can
> take necessary preparations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
