In the context of the HA Manager, resources' downtime is expected to be
minimized as much as possible. Therefore, it is more reasonable to try
other possible node placements if one or more of the HA resources of a
positive affinity rule fail, instead of putting the failed HA resources
in recovery.
This can be improved later to allow temporarily separated positive
affinity "groups", where the failed HA resources tries to find a node,
where it can start, and afterwards the other HA resources are migrated
there too, but this simpler heuristic is enough for the current feature.
Reported-by: Hannes Dürr <h.du...@proxmox.com>
Reported-by: Michael Köppl <m.koe...@proxmox.com>
Signed-off-by: Daniel Kral <d.k...@proxmox.com>
---
src/PVE/HA/Rules/ResourceAffinity.pm | 4 +++
.../README | 9 ++++---
.../log.expect | 26 ++++++++++++++++---
3 files changed, 31 insertions(+), 8 deletions(-)
diff --git a/src/PVE/HA/Rules/ResourceAffinity.pm
b/src/PVE/HA/Rules/ResourceAffinity.pm
index 7327ee08..9bc039ba 100644
--- a/src/PVE/HA/Rules/ResourceAffinity.pm
+++ b/src/PVE/HA/Rules/ResourceAffinity.pm
@@ -596,6 +596,10 @@ resource has not failed running there yet.
sub apply_positive_resource_affinity : prototype($$) {
my ($together, $allowed_nodes) = @_;
+ for my $node (keys %$together) {
+ delete $together->{$node} if !$allowed_nodes->{$node};
+ }
+
my @possible_nodes = sort keys $together->%*
or return; # nothing to do if there is no positive resource affinity
diff --git a/src/test/test-resource-affinity-strict-positive3/README
b/src/test/test-resource-affinity-strict-positive3/README
index a270277b..804d1312 100644
--- a/src/test/test-resource-affinity-strict-positive3/README
+++ b/src/test/test-resource-affinity-strict-positive3/README
@@ -1,7 +1,8 @@
Test whether a strict positive resource affinity rule makes three resources
migrate to the same recovery node in case of a failover of their previously
assigned node. If one of those fail to start on the recovery node (e.g.
-insufficient resources), the failing resource will be kept on the recovery
node.
+insufficient resources), all resources in the positive resource affinity rule
+will be migrated to another available recovery node.
The test scenario is:
- vm:101, vm:102, and fa:120002 must be kept together
@@ -12,6 +13,6 @@ The test scenario is:
The expected outcome is:
- As node3 fails, all resources are migrated to node2
-- Two of those resources will start successfully, but fa:120002 will stay in
- recovery, since it cannot be started on this node, but cannot be relocated to
- another one either due to the strict resource affinity rule
+- Two of those resources will start successfully, but fa:120002 will fail; as
+ there are other available nodes left where it can run, all resources in the
+ positive resource affinity rule are migrated to the next-best fitting node
diff --git a/src/test/test-resource-affinity-strict-positive3/log.expect
b/src/test/test-resource-affinity-strict-positive3/log.expect
index 4a54cb3b..b5d7018f 100644
--- a/src/test/test-resource-affinity-strict-positive3/log.expect
+++ b/src/test/test-resource-affinity-strict-positive3/log.expect
@@ -82,8 +82,26 @@ info 263 node2/lrm: starting service fa:120002
warn 263 node2/lrm: unable to start service fa:120002
err 263 node2/lrm: unable to start service fa:120002 on local node
after 1 retries
warn 280 node1/crm: starting service fa:120002 on node 'node2' failed,
relocating service.
-warn 280 node1/crm: Start Error Recovery: Tried all available nodes for
service 'fa:120002', retry start on current node. Tried nodes: node2
-info 283 node2/lrm: starting service fa:120002
-info 283 node2/lrm: service status fa:120002 started
-info 300 node1/crm: relocation policy successful for 'fa:120002' on node
'node2', failed nodes: node2
+info 280 node1/crm: relocate service 'fa:120002' to node 'node1'
+info 280 node1/crm: service 'fa:120002': state changed from 'started' to
'relocate' (node = node2, target = node1)
+info 283 node2/lrm: service fa:120002 - start relocate to node 'node1'
+info 283 node2/lrm: service fa:120002 - end relocate to node 'node1'
+info 300 node1/crm: service 'fa:120002': state changed from 'relocate'
to 'started' (node = node1)
+info 300 node1/crm: migrate service 'vm:101' to node 'node1' (running)
+info 300 node1/crm: service 'vm:101': state changed from 'started' to
'migrate' (node = node2, target = node1)
+info 300 node1/crm: migrate service 'vm:102' to node 'node1' (running)
+info 300 node1/crm: service 'vm:102': state changed from 'started' to
'migrate' (node = node2, target = node1)
+info 301 node1/lrm: starting service fa:120002
+info 301 node1/lrm: service status fa:120002 started
+info 303 node2/lrm: service vm:101 - start migrate to node 'node1'
+info 303 node2/lrm: service vm:101 - end migrate to node 'node1'
+info 303 node2/lrm: service vm:102 - start migrate to node 'node1'
+info 303 node2/lrm: service vm:102 - end migrate to node 'node1'
+info 320 node1/crm: relocation policy successful for 'fa:120002' on node
'node1', failed nodes: node2
+info 320 node1/crm: service 'vm:101': state changed from 'migrate' to
'started' (node = node1)
+info 320 node1/crm: service 'vm:102': state changed from 'migrate' to
'started' (node = node1)
+info 321 node1/lrm: starting service vm:101
+info 321 node1/lrm: service status vm:101 started
+info 321 node1/lrm: starting service vm:102
+info 321 node1/lrm: service status vm:102 started
info 720 hardware: exit simulation - done
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
