bhouse-nexthop opened a new pull request, #14745:
URL: https://github.com/apache/cloudstack/pull/14745

   ### Description
   
   When an HA-enabled VM stops unexpectedly while its host stays Up, the HA 
restart first tries the VM's last host, which is usually the host that just 
stopped it. If that host is Up and passes the capacity check, the VM goes 
straight back there, and the planner and `vm.allocation.algorithm` are never 
asked.
   
   That's right when the VM itself failed. It's wrong when the host stopped it, 
for example when the host's kernel OOM-killed the QEMU process. Under memory 
overcommit, the capacity check still counts that host as having room, so the VM 
is restarted on the same full host.
   
   Changing that unconditionally would surprise anyone who relies on HA 
restarting a VM in place. So this adds a cluster-scoped setting, 
`vm.ha.restart.consider.last.host`, defaulting to `true`, which is today's 
behaviour. When it's `false`, the HA restart sets `ConsiderLastHost=false`, so 
the planner and allocators choose the host. They may still choose the same host 
when it's the best fit.
   
   Fixes #14744
   
   #### Changes
   
   - New `ConfigKey<Boolean>` `vm.ha.restart.consider.last.host` in 
`HighAvailabilityManagerImpl`: Advanced category, dynamic, cluster scope, 
default `true`. It's registered in `getConfigKeys()`.
   - `restart()` builds its params in a new `getRestartParams(workType, 
clusterId)`:
     - `HaTag` when `ha.tag` is set, and `HaOperation` for HA work, both as 
before;
     - `ConsiderLastHost=false` when the setting is `false` for the cluster of 
the host the VM ran on.
   - With `ha.tag` set, the last host is never preferred anyway, so the setting 
has no effect there. The description says so.
   
   #### Not changed here
   
   - The default keeps today's behaviour.
   - The last host isn't excluded, only no longer preferred. Excluding it would 
make the restart fail outright when it's the only host that fits.
   - Host failure isn't affected: a host that isn't Up is never used as the 
last host.
   
   ### Types of changes
   
   - [ ] Breaking change (fix or feature that would cause existing 
functionality to change)
   - [ ] New feature (non-breaking change which adds functionality)
   - [x] Bug fix (non-breaking change which fixes an issue)
   - [x] Enhancement (improves an existing feature and functionality)
   - [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
   - [ ] Build/CI
   - [x] Test (unit or integration test code)
   
   ### Feature/Enhancement Scale or Bug Severity
   
   #### Bug Severity
   
   - [ ] BLOCKER
   - [ ] Critical
   - [ ] Major
   - [x] Minor
   - [ ] Trivial
   
   ### How Has This Been Tested?
   
   - New unit tests in `HighAvailabilityManagerImplTest`:
     - by default the params don't set `ConsiderLastHost`;
     - with the setting `false` they set it to `false`;
     - `HaOperation` is set only for HA work.
   - Lab test: a nested KVM cluster of three hosts, with 
`vm.allocation.algorithm=firstfitleastconsumed`. The lab runs a main-based 
(24.0) build with this commit cherry-picked, not the 4.22 branch itself; the 
code changed here is the same on both.
     - Setup: an HA-enabled VM and two filler VMs on host 1, the most consumed 
host. Then the VM's QEMU process is killed with `kill -9` on host 1, as the OOM 
killer does, and HA restarts it.
   
   | `vm.ha.restart.consider.last.host` (cluster) | HA restarted the VM on | 
Planner log |
   |---|---|---|
   | `true` (default) | host 1, its last host | `Last host [...] of VM [...] is 
UP and has enough capacity` |
   | `false` | host 2, the least consumed | no last-host line |
   
   #### How did you try to break this feature and the system with this change?
   
   - With `false` and no other host that fits, the planner still picks the last 
host, so the restart doesn't fail where it used to succeed.
   - The value is read per cluster at restart time, so a cluster override 
applies without a restart.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to