sodonnel opened a new pull request #881:
URL: https://github.com/apache/hadoop-ozone/pull/881
## What changes were proposed in this pull request?
Now we have network topology available in Ozone, we need the ability to
identify and correct containers where the replicas do not meet the replication
policy. For network aware clusters, the rule is that the replicas must be on at
least two racks. In the future, it is possible there will be other policies, eg
node groups and so on.
Ideally, the existing ReplicationManager in SCM should be extended to
detect, and correct these mis-replicated containers.
This change starts by adding a new method to the PlacementPolicy interface:
```
ContainerPlacementStatus validateContainerPlacement(
List<DatanodeDetails> dns, int replicas);
```
This accepts the list of DNs currently hosting the replicas for a container,
and the desired replication factor. It return an instance of
`ContainerPlacementStatus` which has two methods indicating if the policy is
met and if not, how many more replicas are needed:
```
boolean isPolicySatisfied();
int additionalReplicaRequired();
```
Currently, all placement policies extend `SCMCommonPlacementPolicy` and
there is a default impelementation there, which handles both rack aware and non
rack aware clusters. If we have further policies in the future (eg node groups,
upgrade domain) this can be overridden in new subclasses.
Replication manager can then call:
```
placementPolicy.validateContainerPlacement(List<DatanodeDetails>,
replicationFactor)
```
To get the information it needs to check the container.
For over replicated containers, when removing replicas, it is important to
also ensure removing the container does not violate the policy, so some changes
will be needed around the over-replicated code path too.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-3081
## How was this patch tested?
New unit tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
