umamaheswararao commented on code in PR #3329:
URL: https://github.com/apache/ozone/pull/3329#discussion_r861990197
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java:
##########
@@ -621,6 +623,48 @@ public void processLayoutVersionReport(DatanodeDetails
datanodeDetails,
}
}
+ /**
+ * Process Command Queue Reports from the Datanode Heartbeat.
+ *
+ * @param datanodeDetails
+ * @param commandQueueReportProto
+ */
+ @Override
+ public void processNodeCommandQueueReport(DatanodeDetails datanodeDetails,
+ CommandQueueReportProto commandQueueReportProto) {
+ LOG.debug("Processing Command Queue Report from [datanode={}]",
+ datanodeDetails.getHostName());
+ if (LOG.isTraceEnabled()) {
+ LOG.trace("Command Queue Report is received from [datanode={}]: " +
+ "<json>{}</json>", datanodeDetails.getHostName(),
+ commandQueueReportProto.toString().replaceAll("\n", "\\\\n"));
+ }
+ try {
+ DatanodeInfo datanodeInfo = nodeStateManager.getNode(datanodeDetails);
+ if (commandQueueReportProto != null) {
+ datanodeInfo.setCommandCounts(commandQueueReportProto);
+ metrics.incNumNodeCommandQueueReportProcessed();
+ }
+ } catch (NodeNotFoundException e) {
Review Comment:
ok. Yeah name is bit misleading a bit.
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DatanodeInfo.java:
##########
@@ -49,6 +57,7 @@ public class DatanodeInfo extends DatanodeDetails {
private List<StorageReportProto> storageReports;
private List<MetadataStorageReportProto> metadataStorageReports;
private LayoutVersionProto lastKnownLayoutVersion;
Review Comment:
Yeah, On thinking a bit, I think it's ok to leave the counts as well (
anyway we will not assign tasks to dead DN). when node rejoined, we should
receive new HB and counts should get refreshed.
This above questions was just for my clarity what happens to DN object.
Thanks for the details
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/DatanodeInfo.java:
##########
@@ -272,6 +282,57 @@ public void setNodeStatus(NodeStatus newNodeStatus) {
}
}
+ /**
+ * Set the current command counts for this datanode, as reported in the last
+ * heartbeat.
+ * @param cmds Proto message containing a list of command count pairs.
+ */
+ public void setCommandCounts(CommandQueueReportProto cmds) {
+ try {
+ int count = cmds.getCommandCount();
+ lock.writeLock().lock();
+ for (int i = 0; i < count; i++) {
+ SCMCommandProto.Type command = cmds.getCommand(i);
+ if (command == SCMCommandProto.Type.unknownScmCommand) {
+ LOG.warn("Unknown SCM Command received from {} in the "
+ + "heartbeat. SCM and the DN may not be at the same version.",
+ this);
+ continue;
+ }
+ int cmdCount = cmds.getCount(i);
+ if (cmdCount < 0) {
+ LOG.warn("Command count of {} from {} should be greater than zero. "
+
+ "Setting it to zero", cmdCount, this);
+ cmdCount = 0;
+ }
+ commandCounts.put(command, cmdCount);
+ }
+ } finally {
+ lock.writeLock().unlock();
+ }
+ }
+
+ /**
+ * Retrieve the number of queued commands of the given type, as reported by
+ * the datanode at the last heartbeat.
+ * @param cmd The command for which to receive the queued command count
Review Comment:
Git it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
