[
https://issues.apache.org/jira/browse/HDFS-17272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17793132#comment-17793132
]
ASF GitHub Bot commented on HDFS-17272:
---------------------------------------
ayushtkn commented on code in PR #6319:
URL: https://github.com/apache/hadoop/pull/6319#discussion_r1414895365
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNNThroughputBenchmark.java:
##########
@@ -187,4 +187,60 @@ public void testNNThroughputForBlockReportOp() throws
Exception {
"blockReport", "-datanodes", "3", "-reports", "2"});
}
}
+
+ /**
+ * This test runs {@link NNThroughputBenchmark} against a mini DFS cluster
+ * with explicit -baseDirName option.
+ */
+ @Test(timeout = 120000)
+ public void testNNThroughputWithBaseDir() throws Exception {
+ final Configuration conf = new HdfsConfiguration();
+ conf.setInt(DFSConfigKeys.DFS_NAMENODE_MIN_BLOCK_SIZE_KEY, 16);
+ MiniDFSCluster cluster = null;
+ try {
+ cluster = new MiniDFSCluster.Builder(conf).numDataNodes(0).build();
+ cluster.waitActive();
+ final Configuration benchConf = new HdfsConfiguration();
+ benchConf.setInt(DFSConfigKeys.DFS_BLOCK_SIZE_KEY, 16);
+ FileSystem.setDefaultUri(benchConf, cluster.getURI());
+
+ NNThroughputBenchmark.runBenchmark(benchConf,
+ new String[] {"-op", "create", "-keepResults", "-files", "3",
"-baseDirName",
+ "/nnThroughputBenchmark1", "-close"});
+ FSNamesystem fsNamesystem = cluster.getNamesystem();
+ DirectoryListing listing = fsNamesystem.getListing("/",
HdfsFileStatus.EMPTY_NAME, false);
+ Boolean dirExist1 = false;
+ Boolean dirExist2 = false;
+ for (HdfsFileStatus f : listing.getPartialListing()) {
+ if (f.getFullName("/").equals("/nnThroughputBenchmark1")) {
+ dirExist1 = true;
+ }
+ if (f.getFullName("/").equals("/nnThroughputBenchmark")) {
+ dirExist2 = true;
+ }
+ }
+ Assert.assertEquals(dirExist1, true);
+ Assert.assertEquals(dirExist2, false);
+
+ NNThroughputBenchmark.runBenchmark(benchConf,
+ new String[] {"-op", "all", "-baseDirName",
"/nnThroughputBenchmark1"});
+ listing = fsNamesystem.getListing("/", HdfsFileStatus.EMPTY_NAME, false);
Review Comment:
can't we use fs.exists(path) for such validations rather than listing and
then checking?
> NNThroughputBenchmark should support specifying the base directory for
> multi-client test
> ----------------------------------------------------------------------------------------
>
> Key: HDFS-17272
> URL: https://issues.apache.org/jira/browse/HDFS-17272
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Affects Versions: 3.4.0
> Reporter: caozhiqiang
> Assignee: caozhiqiang
> Priority: Major
> Labels: pull-request-available
>
> Currently, NNThroughputBenchmark does not support specifying the base
> directory, therefore does not support multiple clients performing stress
> testing at the same time. However, for high-performance namenode machine,
> only one client submitting stress test can not make the namenode rpc access
> reach the bottleneck. Therefore, multiple clients are required for parallel
> testing to make the namenode pressure reach the level of the large-scale
> production cluster.
> So I specify the base directory through the -baseDirName parameter to support
> multiple clients submitting stress tests at the same time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
