xichen01 commented on code in PR #4939:
URL: https://github.com/apache/ozone/pull/4939#discussion_r1242571991
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/block/SCMBlockDeletingService.java:
##########
@@ -273,4 +279,24 @@ public void stop() {
public ScmBlockDeletingServiceMetrics getMetrics() {
return this.metrics;
}
+
+ /**
+ * Filters and returns a set of healthy datanodes that have not exceeded
+ * the deleteBlocksPendingCommandLimit.
+ *
+ * @param datanodes a list of DatanodeDetails
+ * @return a set of filtered DatanodeDetails
+ */
+ @VisibleForTesting
+ protected Set<DatanodeDetails> getDatanodesWithinCommandLimit(
+ List<DatanodeDetails> datanodes) throws NodeNotFoundException {
+ final Set<DatanodeDetails> included = new HashSet<>();
+ for (DatanodeDetails dn : datanodes) {
+ if (nodeManager.getTotalDatanodeCommandCount(dn,
Review Comment:
By default the SCM generates a delete command every 60s and sends it to the
DN. The DN reports a heartbeat every 30s. So normally SCM can get the newer DN
status.
`getTotalDatanodeCommandCount` returns the number of `DeleteBlocksCommand`,
each time SCM executes `DeletedBlockTransactionScanner`, only one
`DeleteBlocksCommand` is sent to a specific DN.
`DeletedBlockTransactionScanner` execution frequency is fixed, the limit here
is 5, so the SCM must execute at least 5 times before the DN's queue is full,
which needs 5 min. as long as the DN can send a heartbeat of before all these
commands are executed, then the SCM can continue to send delete commands to the
DN.
If the queue of all DNs is full, SCM should not continue to send new
commands to DN
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
