JiangHua Zhu created HDFS-16439:
-----------------------------------
Summary: Makes calculating maxNodesPerRack simpler
Key: HDFS-16439
URL: https://issues.apache.org/jira/browse/HDFS-16439
Project: Hadoop HDFS
Issue Type: Improvement
Components: namenode
Affects Versions: 3.4.0
Reporter: JiangHua Zhu
When creating a new file, it is usually necessary to communicate with the
namenode first to obtain the location of some DataNodes as the target location
of Blockd. At this time, when BlockPlacementPolicyDefault#getMaxNodesPerRack()
is executed, if the number of replicas is very large, once it exceeds the
number of all nodes in the cluster. The following piece of code will be
executed:
int clusterSize = clusterMap.getNumOfLeaves();
int totalNumOfReplicas = numOfChosen + numOfReplicas;
if (totalNumOfReplicas > clusterSize) {
numOfReplicas -= (totalNumOfReplicas-clusterSize);
totalNumOfReplicas = clusterSize;
}
Here, the calculation for numOfReplicas gets a little more complicated. It can
be simplified like:
numOfReplicas = clusterSize - numOfChosen
It would be more helpful to understand it this way, while also freeing up a
little cpu (though not a lot).
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
