[
https://issues.apache.org/jira/browse/HDFS-17210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773079#comment-17773079
]
ASF GitHub Bot commented on HDFS-17210:
---------------------------------------
Hexiaoqiao commented on code in PR #6113:
URL: https://github.com/apache/hadoop/pull/6113#discussion_r1349824415
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java:
##########
@@ -1240,6 +1240,12 @@ public class DFSConfigKeys extends
CommonConfigurationKeys {
public static final int
DFS_NAMENODE_AVAILABLE_SPACE_BLOCK_PLACEMENT_POLICY_BALANCED_SPACE_TOLERANCE_DEFAULT
=
5;
+ public static final String
+
DFS_NAMENODE_AVAILABLE_SPACE_BLOCK_PLACEMENT_POLICY_BALANCED_SPACE_TOLERANCE_LIMIT_KEY
=
Review Comment:
Align as other configuration item.
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/AvailableSpaceBlockPlacementPolicy.java:
##########
@@ -71,6 +76,11 @@ public void initialize(Configuration conf, FSClusterStats
stats,
DFS_NAMENODE_AVAILABLE_SPACE_BLOCK_PLACEMENT_POLICY_BALANCED_SPACE_TOLERANCE_KEY,
DFS_NAMENODE_AVAILABLE_SPACE_BLOCK_PLACEMENT_POLICY_BALANCED_SPACE_TOLERANCE_DEFAULT);
+ balancedSpaceToleranceLimit =
Review Comment:
Suggest to check if this value is between 0 and 100. Otherwise, it could
work abnormally.
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestAvailableSpaceBlockPlacementPolicy.java:
##########
@@ -219,6 +222,77 @@ public void testChooseSimilarDataNode() {
tolerateDataNodes[0], false) == 1);
}
+
+ @Test
+ public void testCompareDataNode() {
+ DatanodeDescriptor[] tolerateDataNodes;
+ DatanodeStorageInfo[] tolerateStorages;
+ int capacity = 5;
+ Collection<Node> allTolerateNodes = new ArrayList<>(capacity);
+ String[] ownerRackOfTolerateNodes = new String[capacity];
+ for (int i = 0; i < capacity; i++) {
+ ownerRackOfTolerateNodes[i] = "rack"+i;
+ }
+ tolerateStorages =
DFSTestUtil.createDatanodeStorageInfos(ownerRackOfTolerateNodes);
+ tolerateDataNodes = DFSTestUtil.toDatanodeDescriptor(tolerateStorages);
+
+ Collections.addAll(allTolerateNodes, tolerateDataNodes);
+ final BlockManager bm = namenode.getNamesystem().getBlockManager();
+ AvailableSpaceBlockPlacementPolicy toleratePlacementPolicy =
+ (AvailableSpaceBlockPlacementPolicy)bm.getBlockPlacementPolicy();
Review Comment:
Suggest to keep the alignment as usual. You should add the 'checkstyle.xml'
to your IDE or follow this [guide
docs](https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute#HowToContribute-Contributingyourwork).
> Optimize AvailableSpaceBlockPlacementPolicy
> -------------------------------------------
>
> Key: HDFS-17210
> URL: https://issues.apache.org/jira/browse/HDFS-17210
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
> Affects Versions: 3.3.6
> Reporter: Fei Guo
> Priority: Minor
> Labels: pull-request-available
>
> for now ,we may have many nodes usage over 85%, some nodes'usage have been
> over 90%, if we choose three nodes as nodeA-97%,nodeB-98%,nodeC-99%, actually
> i don't want nodeC(99%) be choosen, for nodeC usage is really high , even
> random chance ,nodeC will reach 100% soon, so we can directly choose the less
> usage node(nodeA) if all nodes's usage are over 95%(just a example)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
