xichen01 commented on code in PR #10928:
URL: https://github.com/apache/ozone/pull/10928#discussion_r3744459441
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java:
##########
@@ -1078,22 +1078,36 @@ public DatanodeInfo getDatanodeInfo(DatanodeDetails dn)
{
*/
@Override
public boolean hasSpaceForNewContainerAllocation(DatanodeID datanodeID) {
+ return hasSpaceForNewContainerAllocation(datanodeID, null);
+ }
+
+ @Override
+ public boolean hasSpaceForNewContainerAllocation(
+ DatanodeID datanodeID, StorageType storageType) {
DatanodeInfo datanodeInfo = getNode(datanodeID);
if (datanodeInfo == null) {
LOG.warn("DatanodeInfo not found for node {}", datanodeID);
return false;
}
- return
pendingContainerTracker.hasEffectiveAllocatableSpaceForNewContainer(datanodeInfo);
+ return pendingContainerTracker.hasEffectiveAllocatableSpaceForNewContainer(
+ datanodeInfo, storageType);
}
@Override
public void recordPendingAllocationForDatanode(DatanodeID datanodeID,
ContainerID containerID) {
Review Comment:
This can be removed too, make that `recordPendingAllocationForDatanode` must
pass a `storageType`
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/NodeManager.java:
##########
@@ -189,6 +190,14 @@ default int getAllNodeCount() {
*/
boolean hasSpaceForNewContainerAllocation(DatanodeID datanodeID);
Review Comment:
To prevent callers from forgetting to pass the `storageType` parameter, we
should retain only the `hasSpaceForNewContainerAllocation` method that includes
`storageType`.
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerManagerImpl.java:
##########
@@ -277,6 +277,7 @@ private ContainerInfo allocateContainer(final Pipeline
pipeline,
}
containerStateManager.addContainer(containerInfoBuilder.build());
+ pipelineManager.recordPendingAllocation(pipeline, containerID);
Review Comment:
`allocateContainer` can be executed in parallel;
`pipelineManager.hasEnoughSpace` and
`pipelineManager.recordPendingAllocation` are not protected by locks. In some
cases, a DN may be allocated too many containers.
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/PendingContainerTracker.java:
##########
@@ -117,25 +119,37 @@ synchronized void rollIfNeeded() {
}
synchronized boolean contains(ContainerID containerID) {
- return currentWindow.contains(containerID) ||
previousWindow.contains(containerID);
+ return currentWindow.containsKey(containerID) ||
previousWindow.containsKey(containerID);
}
/**
* Add container to current window.
*/
synchronized boolean add(ContainerID containerID) {
Review Comment:
this old method can be removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
