JacksonYao287 commented on PR #3963: URL: https://github.com/apache/ozone/pull/3963#issuecomment-1336188924
@sodonnel sorry for not replying in time. >I believe that the Balancer / Move Manager and the Replication Manager need to be aligned in how they work. agree, we`d better have a new design doc for container move, [the old design doc](https://issues.apache.org/jira/secure/attachment/13026798/Container%20Move.pdf) is written for legacyRM. i think scm could send all the commands(replication , deletion ,reconstruction. e.g.) through a separate class (maybe commandSendingManager). this class can limit the command sending to each datanode. it can buffer some command to be sent to a datanode until the num of commands in the datanode command queue is small enough , and it can also be notified by containerPendingOps with the op command status. >If RM is not replicating its pending moves, then why do we need the balancer to replicate its commands too? @siddhantsangwan can you explain why balancer need to replicate its command in more details. i think you are familiar with this part. >So all we need to do, is tell the DN to drop any commands that are from an old SCM leader. now, datanode will drop all the commands from an old scm leader. but i find there is a small problem. the scm leader term at datanoe is updated only before it is close to be handled. it means all the commands in the command queue from the stale scm leader will be handled until a command from new scm leader is handled. we should update the scm leader term at datanode when receiving a command, so that we can make sure those stale command will not be handled. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
