lokeshj1703 commented on a change in pull request #2488:
URL: https://github.com/apache/ozone/pull/2488#discussion_r686731062
##########
File path:
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManager.java
##########
@@ -1712,8 +1745,244 @@ public ReplicationManagerMetrics getMetrics() {
return inflightDeletion;
}
- public Map<ContainerID,
- Pair<DatanodeDetails, DatanodeDetails>> getInflightMove() {
- return inflightMove;
+ public Map<ContainerID, CompletableFuture<MoveResult>> getInflightMove() {
+ return inflightMoveFuture;
+ }
+
+ /**
+ * make move option HA aware.
+ */
+ public interface MoveScheduler {
+ /**
+ * completeMove a move action for a given container.
+ *
+ * @param contianerIDProto Container to which the move option is finished
+ */
+ @Replicate
+ void completeMove(HddsProtos.ContainerID contianerIDProto);
+
+ /**
+ * start a move action for a given container.
+ *
+ * @param contianerIDProto Container to move
+ * @param mp encapsulates the source and target datanode infos
+ */
+ @Replicate
+ void startMove(HddsProtos.ContainerID contianerIDProto,
+ HddsProtos.MoveDataNodePairProto mp) throws IOException;
+
+ /**
+ * get the MoveDataNodePair of the giver container.
+ *
+ * @param cid Container to move
+ * @return null if cid is not found in MoveScheduler,
+ * or the corresponding MoveDataNodePair
+ */
+ MoveDataNodePair getMoveDataNodePair(ContainerID cid);
+
+ /**
+ * Reinitialize the MoveScheduler with DB if become leader.
+ */
+ void reinitialize(Table<ContainerID,
+ MoveDataNodePair> moveTable) throws IOException;
+
+ /**
+ * get all the inflight move info.
+ */
+ Map<ContainerID, MoveDataNodePair> getInflightMove();
+ }
+
+ /**
+ * @return the moveScheduler of RM
+ */
+ public MoveScheduler getMoveScheduler() {
+ return moveScheduler;
+ }
+
+ /**
+ * Ratis based MoveScheduler, db operations are stored in
+ * DBTransactionBuffer until a snapshot is taken.
+ */
+ public static final class MoveSchedulerImpl implements MoveScheduler {
+ private Table<ContainerID, MoveDataNodePair> moveTable;
+ private final DBTransactionBuffer transactionBuffer;
+ /**
+ * This is used for tracking container move commands
+ * which are not yet complete.
+ */
+ private final Map<ContainerID, MoveDataNodePair> inflightMove;
+
+ private MoveSchedulerImpl(Table<ContainerID, MoveDataNodePair> moveTable,
+ DBTransactionBuffer transactionBuffer) throws IOException {
+ this.moveTable = moveTable;
+ this.transactionBuffer = transactionBuffer;
+ this.inflightMove = new ConcurrentHashMap<>();
+ initialize();
+ }
+
+ @Override
+ public void completeMove(HddsProtos.ContainerID contianerIDProto) {
Review comment:
For all these apis which require protobuf, we can create another
function which takes ContainerID as input and redirects to these apis?
##########
File path:
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManager.java
##########
@@ -1712,8 +1745,244 @@ public ReplicationManagerMetrics getMetrics() {
return inflightDeletion;
}
- public Map<ContainerID,
- Pair<DatanodeDetails, DatanodeDetails>> getInflightMove() {
- return inflightMove;
+ public Map<ContainerID, CompletableFuture<MoveResult>> getInflightMove() {
+ return inflightMoveFuture;
+ }
+
+ /**
+ * make move option HA aware.
+ */
+ public interface MoveScheduler {
Review comment:
We can probably move MoveScheduler and Impl to separate classes?
##########
File path:
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManager.java
##########
@@ -1712,8 +1745,244 @@ public ReplicationManagerMetrics getMetrics() {
return inflightDeletion;
}
- public Map<ContainerID,
- Pair<DatanodeDetails, DatanodeDetails>> getInflightMove() {
- return inflightMove;
+ public Map<ContainerID, CompletableFuture<MoveResult>> getInflightMove() {
+ return inflightMoveFuture;
+ }
+
+ /**
+ * make move option HA aware.
+ */
+ public interface MoveScheduler {
+ /**
+ * completeMove a move action for a given container.
+ *
+ * @param contianerIDProto Container to which the move option is finished
+ */
+ @Replicate
+ void completeMove(HddsProtos.ContainerID contianerIDProto);
+
+ /**
+ * start a move action for a given container.
+ *
+ * @param contianerIDProto Container to move
+ * @param mp encapsulates the source and target datanode infos
+ */
+ @Replicate
+ void startMove(HddsProtos.ContainerID contianerIDProto,
+ HddsProtos.MoveDataNodePairProto mp) throws IOException;
+
+ /**
+ * get the MoveDataNodePair of the giver container.
+ *
+ * @param cid Container to move
+ * @return null if cid is not found in MoveScheduler,
+ * or the corresponding MoveDataNodePair
+ */
+ MoveDataNodePair getMoveDataNodePair(ContainerID cid);
+
+ /**
+ * Reinitialize the MoveScheduler with DB if become leader.
+ */
+ void reinitialize(Table<ContainerID,
+ MoveDataNodePair> moveTable) throws IOException;
+
+ /**
+ * get all the inflight move info.
+ */
+ Map<ContainerID, MoveDataNodePair> getInflightMove();
+ }
+
+ /**
+ * @return the moveScheduler of RM
+ */
+ public MoveScheduler getMoveScheduler() {
+ return moveScheduler;
+ }
+
+ /**
+ * Ratis based MoveScheduler, db operations are stored in
+ * DBTransactionBuffer until a snapshot is taken.
+ */
+ public static final class MoveSchedulerImpl implements MoveScheduler {
+ private Table<ContainerID, MoveDataNodePair> moveTable;
+ private final DBTransactionBuffer transactionBuffer;
+ /**
+ * This is used for tracking container move commands
+ * which are not yet complete.
+ */
+ private final Map<ContainerID, MoveDataNodePair> inflightMove;
+
+ private MoveSchedulerImpl(Table<ContainerID, MoveDataNodePair> moveTable,
+ DBTransactionBuffer transactionBuffer) throws IOException {
+ this.moveTable = moveTable;
+ this.transactionBuffer = transactionBuffer;
+ this.inflightMove = new ConcurrentHashMap<>();
+ initialize();
+ }
+
+ @Override
+ public void completeMove(HddsProtos.ContainerID contianerIDProto) {
Review comment:
Even for move api I think we can support source and target as input
parameters.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
