xichen01 commented on code in PR #10573:
URL: https://github.com/apache/ozone/pull/10573#discussion_r3471976135
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java:
##########
@@ -722,6 +724,49 @@ public void processNodeReport(DatanodeDetails
datanodeDetails,
}
}
+ private void updateSupportedStorageTier(DatanodeInfo datanodeInfo) {
+ if (scmContext.getScm() == null) {
+ LOG.debug("Skip the updating of Pipeline supported StorageTier for
Recon");
+ return;
+ }
+
+ PipelineManager pipelineManager = scmContext.getScm().getPipelineManager();
+ if (pipelineManager == null) {
+ LOG.debug("Skip the updating of Pipeline supported StorageTier for
Recon");
+ return;
+ }
+
+ Set<PipelineID> pipelines = nodeStateManager.getPipelineByDnID(
+ datanodeInfo.getID());
+ for (PipelineID pipelineId : pipelines) {
+ try {
+ Pipeline pipeline = pipelineManager.getPipeline(pipelineId);
+ Set<StorageTier> currentTiers =
+ pipeline.getSupportedStorageTier() != null
+ ? new HashSet<>(pipeline.getSupportedStorageTier())
+ : Collections.emptySet();
+ List<StorageTier> newSupportedTiers =
+ NodeUtils.getDatanodesStorageTypes(pipeline.getNodes(), this);
+ Set<StorageTier> newSupportedTierSet =
+ new HashSet<>(newSupportedTiers);
+ if (!currentTiers.equals(newSupportedTierSet)) {
+ pipeline.setSupportedStorageTier(newSupportedTiers);
+ LOG.info("Updated supported storage tiers for Pipeline ID {} from {}
"
+ + "to {} by Datanode {}",
+ pipeline.getId(), currentTiers, newSupportedTierSet,
+ datanodeInfo.getID());
+ }
Review Comment:
If the storage type of a DN's volume changes, the `NodeReportHandler` can
process `StorageTier` more quickly, and it is also more semantically
appropriate.
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java:
##########
@@ -722,6 +724,49 @@ public void processNodeReport(DatanodeDetails
datanodeDetails,
}
}
+ private void updateSupportedStorageTier(DatanodeInfo datanodeInfo) {
+ if (scmContext.getScm() == null) {
+ LOG.debug("Skip the updating of Pipeline supported StorageTier for
Recon");
+ return;
+ }
+
+ PipelineManager pipelineManager = scmContext.getScm().getPipelineManager();
+ if (pipelineManager == null) {
+ LOG.debug("Skip the updating of Pipeline supported StorageTier for
Recon");
+ return;
+ }
+
+ Set<PipelineID> pipelines = nodeStateManager.getPipelineByDnID(
+ datanodeInfo.getID());
+ for (PipelineID pipelineId : pipelines) {
+ try {
+ Pipeline pipeline = pipelineManager.getPipeline(pipelineId);
+ Set<StorageTier> currentTiers =
+ pipeline.getSupportedStorageTier() != null
+ ? new HashSet<>(pipeline.getSupportedStorageTier())
+ : Collections.emptySet();
+ List<StorageTier> newSupportedTiers =
+ NodeUtils.getDatanodesStorageTypes(pipeline.getNodes(), this);
+ Set<StorageTier> newSupportedTierSet =
+ new HashSet<>(newSupportedTiers);
+ if (!currentTiers.equals(newSupportedTierSet)) {
+ pipeline.setSupportedStorageTier(newSupportedTiers);
+ LOG.info("Updated supported storage tiers for Pipeline ID {} from {}
"
+ + "to {} by Datanode {}",
+ pipeline.getId(), currentTiers, newSupportedTierSet,
+ datanodeInfo.getID());
+ }
+ } catch (PipelineNotFoundException e) {
+ LOG.debug("Reported Datanode {} pipeline {} is not found",
Review Comment:
updated.
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java:
##########
@@ -722,6 +724,49 @@ public void processNodeReport(DatanodeDetails
datanodeDetails,
}
}
+ private void updateSupportedStorageTier(DatanodeInfo datanodeInfo) {
+ if (scmContext.getScm() == null) {
Review Comment:
`scmContext.getScm() == null` this results in one less level of indentation
and does not skip the `ReconStorageContainerManagerFacade`, so I think we can
keep it this way.
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/pipeline/Pipeline.java:
##########
@@ -170,6 +178,32 @@ public DatanodeID getSuggestedLeaderId() {
return suggestedLeaderId;
}
+ /**
+ * Return the Pipeline supported StorageTier.
+ *
+ * @return Supported StorageTier
+ */
+ public List<StorageTier> getSupportedStorageTier() {
+ lock.readLock().lock();
+ try {
+ return supportedStorageTier;
+ } finally {
+ lock.readLock().unlock();
+ }
+ }
+
+ /**
+ * Set the storageTier supported by the pipeline.
+ */
+ public void setSupportedStorageTier(List<StorageTier> supportedStorageTier) {
Review Comment:
Pipeline has not yet been marked as @Immutable, and there are some other
variables that are mutable. Furthermore, if you want to make
`supportedStorageTier` immutable, you'll need to implement the interface that
the `PipelineUpdate` method calls for in `PipelineManager`.
So I think we can temporarily update `supportedStorageTier` directly using
`set`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
