Hadoop Version: 3.1.0-SNAPSHOT
Environment: ARMv8 AArch64, Ubuntu16.04
Description: Hi, When I ran the HDFS unit test and got a failure in
TestDirectoryScanner.java test case :
TestDirectoryScanner.testThrottling:624 Throttle is too permissive
detail:
Running
org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
Tests run: 7, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
227.046 sec <<< FAILURE! - in
org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
testThrottling(org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner)
Time elapsed: 198.014 sec <<< FAILURE!
java.lang.AssertionError: Throttle is too permissive
at
org.junit.Assert.fail(Assert.java:88)
at
org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner.testThrottling(TestDirectoryScanner.java:624)
And below is the failure part of source code TestDirectoryScanner.java:
...........
while ((retries > 0) && ((ratio < 7f) || (ratio > 10f))) {
scanner = new DirectoryScanner(dataNode, fds, conf);
ratio = runThrottleTest(blocks);
retries -= 1;
}
// Waiting should be about 9x running.
LOG.info("RATIO: " + ratio);
assertTrue("Throttle is too restrictive", ratio <= 10f);
assertTrue("Throttle is too permissive", ratio >= 7f);
............
private float runThrottleTest(int blocks) throws IOException {
scanner.setRetainDiffs(true);
scan(blocks, 0, 0, 0, 0, 0);
scanner.shutdown();
assertFalse(scanner.getRunStatus());
return (float)scanner.timeWaitingMs.get() / scanner.timeRunningMs.get();
}
.............
The ratio in my test is 6.0578866, which is smaller than 7f in the code. So the
code thrown out an assertTrue failure.
My questions are:
1. Why the ratio was set between 7f and 10f, is it a empirical value?
2. The ratio is smaller than 7f in AArch64 platform, is this value
within normal range?
Could anyone help? Thanks a lot.
Best Regards
Guangming Zhang
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient,
please notify the sender immediately and do not disclose the contents to any
other person, use it for any purpose, or store or copy the information in any
medium. Thank you.