Stephen O'Donnell created HDDS-4062:
---------------------------------------
Summary: Non rack aware pipelines should not be created if
multiple racks are alive
Key: HDDS-4062
URL: https://issues.apache.org/jira/browse/HDDS-4062
Project: Hadoop Distributed Data Store
Issue Type: Improvement
Components: SCM
Affects Versions: 0.7.0
Reporter: Stephen O'Donnell
Assignee: Stephen O'Donnell
If we have a scenario where one rack has more nodes that others, it is possible
for all hosts in the cluster to have reached their pipeline limit, while 3
nodes on the larger rack have not.
The current fallback logic will then allow a pipeline to be created which uses
only the 3 nodes on the same rack, violating the rack placement policy.
There may be other ways this could happen with cluster load too, were the
pipeline capacity has reached its limit on some nodes but not others.
The proposal here, is that if the cluster has multiple racks AND there are
healthy nodes covering at least 2 racks, where healthy is defined as a node
which is registered and not stale or dead, then we should not allow "fallback"
(pipelines which span only 1 rack) pipelines to be created.
This means if you have a badly configured cluster - eg Rack 1 = 10 nodes; Rack
2 = 1 node, the pipeline limit will be constrained by the capacity of that 1
node on rack 2. Even a setup like Rack 1 = 10 nodes, Rack 2 = 5 would be
constrained by this.
This constraint is better than creating non rack aware pipelines, and the rule
above will handle the case when the cluster degrades to 1 rack, as the healthy
node definition will notice only 1 rack is alive.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
