sodonnel commented on code in PR #3963:
URL: https://github.com/apache/ozone/pull/3963#discussion_r1029486024
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ReplicationManager.java:
##########
@@ -325,10 +329,22 @@ public synchronized void processAll() {
containerManager.getContainers();
ReplicationManagerReport report = new ReplicationManagerReport();
ReplicationQueue newRepQueue = new ReplicationQueue();
+ Map<ContainerID, MoveDataNodePair> pendingMoves =
+ moveManaer.getPendingMove();
Review Comment:
> 1 if a ratis container has only two replicas, r1(dn1) , r2(dn2) , now it
is under-replicated. if we schedule a move for r1 and everything goes well,
there will be a r3 in dn3 ultimately and this container is not under-replicated
again.
RM is aware of this via the pending ops. It will see the container as
under-replicated, but with pending ops it will see it as OK and will not
schedule anything further.
> if a container is over-replicated(r1, r2, r3, r4), and we want to move r1
to dn5, then movemanager will send a replication command to dn5. if RM is not
ware of this move and it find this over-replicated container, it may send a
deletion command to r1, which will fail the move.
This example is slightly strange - what you are saying could happen, but
really rather than "moving" a container from the over-used node to a lesser
used node, we should just delete it from the over-used node.
Also, ideally, a replicate command for a move could have any of the source
replicas as the place to read from. If you only have a few over used nodes on
the cluster, then they will be a bottleneck for copying from. You don't have to
read from the source you are going to delete. You can read from any source.
Infact, I think the replicate command lets you pass as many sources as you
want, and then it can retry if the copy fails from one source.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
