huangzhaobo99 commented on code in PR #6279:
URL: https://github.com/apache/hadoop/pull/6279#discussion_r1412732623
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/SlowPeerDisabledTracker.java:
##########
@@ -46,11 +46,6 @@ public class SlowPeerDisabledTracker extends SlowPeerTracker
{
public SlowPeerDisabledTracker(Configuration conf, Timer timer) {
super(conf, timer);
- final boolean dataNodePeerStatsEnabledVal =
- conf.getBoolean(DFSConfigKeys.DFS_DATANODE_PEER_STATS_ENABLED_KEY,
- DFSConfigKeys.DFS_DATANODE_PEER_STATS_ENABLED_DEFAULT);
- Preconditions.checkArgument(!dataNodePeerStatsEnabledVal,
- "SlowPeerDisabledTracker should only be used for disabled slow peer
stats.");
Review Comment:
1. The logic here may not be correct because the system is now changing the
value of the key from true to false, while the key of getConf is the old key,
throw IllegalArgumentException in Preconditions.checkArgument.
<img width="1100" alt="image"
src="https://github.com/apache/hadoop/assets/63718681/613e610f-3999-4960-9adb-0b9fed18dcdf";>
---<img width="944" alt="image"
src="https://github.com/apache/hadoop/assets/63718681/7495c5e2-c8d4-479f-9f3b-4f4eda5d7787";>
2. It does not affect reconfig DFS_ DATANODE_ PEER_ STATS_ ENABLED_ KEY's
logic
3. TestNameNodeReconfigure#testSlowPeerTrackerEnabled UT skipped the logic
of refreshing the configuration, so when true is changed to false, the original
value in the system remains false. Therefore, the following code passed the UT
<img width="1181" alt="image"
src="https://github.com/apache/hadoop/assets/63718681/8772ae17-6da0-49e9-8b83-de7da84b63e2";>
4. When to refresh the configuration, refer to the following
code(ReconfigurableBase#reconfigureProperty)
<img width="941" alt="image"
src="https://github.com/apache/hadoop/assets/63718681/e3dd7d4e-1649-472f-8d68-2f0fc436a8e3";>
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyExcludeSlowNodes.java:
##########
@@ -136,4 +138,41 @@ public void testChooseTargetExcludeSlowNodes() throws
Exception {
NameNode.LOG.info("Done working on it");
}
+ @Test
+ public void testSlowPeerTrackerEnabledClearSlowNodes() throws Exception {
+ namenode.getNamesystem().writeLock();
+ try {
+ // add nodes
+ for (DatanodeDescriptor dataNode : dataNodes) {
+ dnManager.addDatanode(dataNode);
+ }
+
+ // mock slow nodes
+ SlowPeerTracker tracker = dnManager.getSlowPeerTracker();
+ assert tracker != null;
+ OutlierMetrics outlierMetrics = new OutlierMetrics(0.0, 0.0, 0.0, 5.0);
+ tracker.addReport(dataNodes[0].getInfoAddr(), dataNodes[3].getInfoAddr(),
+ outlierMetrics);
+ tracker.addReport(dataNodes[1].getInfoAddr(), dataNodes[3].getInfoAddr(),
+ outlierMetrics);
+ tracker.addReport(dataNodes[2].getInfoAddr(), dataNodes[3].getInfoAddr(),
+ outlierMetrics);
+
+ // waiting for slow nodes collector run
+ Thread.sleep(3000);
Review Comment:
fix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
