swamirishi commented on PR #3836:
URL: https://github.com/apache/ozone/pull/3836#issuecomment-1307707255
> The code to group the replicas and select those that can be use for an
additional copy still seems overly complex.
>
> Really, what we need is a list of containerReplicas, where there are more
than 1 of the replicas on the same "group". The order of them doesn't matter,
and we need to be sure we don't try to move them all from a group, as one
should always stay.
>
> I think this code does it, producing a list of containerReplias where any
of them are good to "move":
>
> ```
> Map<Node, List<Pair<ContainerReplica, Node>>> list =
> getSourcesStream(replicas, deletionInFlight).map(Pair::getKey)
> .map(cr -> Pair.of(cr,
((Node)containerPlacement.getPlacementGroup(cr.getDatanodeDetails()))))
> .collect(Collectors.groupingBy(Pair::getRight));
>
> List<ContainerReplica> candidates = new ArrayList<>();
> for (List<Pair<ContainerReplica, Node>> p : list.values()) {
> if (p.size() > 1) {
> // Only groups with more than one entry can help fix
mis-replications
> for (int i = 1; i < p.size(); i++) {
> // Skip the first entry, as we should not move all replicas in a
group
> // only the excess ones. Ideally we skip a random one, but this
is
> // simple and the sets should not be very large anyway.
> candidates.add(p.get(i).getLeft());
> }
> }
> }
> ```
>
> We can discuss this later and see what we can conclude.
Choosing a random replica index might not work:
Conider:
Rack 1: 1,2
Rack 2: 3,4
Rack 3: 1
Copying replica index 1 to another rack 4 will create the following :
Rack 1: 1,2
Rack 2: 3,4
Rack 3: 1
Rack 4: 1
Now the nodes are overreplicated:
Over replication handler cannot delete any of the replica thus this
container will be always over replicated & would be stuck on an infinite loop
@sodonnel Does this make sense?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
