[
https://issues.apache.org/jira/browse/HDFS-17272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792552#comment-17792552
]
ASF GitHub Bot commented on HDFS-17272:
---------------------------------------
ayushtkn commented on code in PR #6319:
URL: https://github.com/apache/hadoop/pull/6319#discussion_r1413157796
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java:
##########
@@ -1348,7 +1363,7 @@ class ReplicationStats extends OperationStatsBase {
static final String OP_REPLICATION_USAGE =
"-op replication [-datanodes T] [-nodesToDecommission D] " +
"[-nodeReplicationLimit C] [-totalBlocks B] [-blockSize S] "
- + "[-replication R]";
+ + "[-replication R] [-baseDirName N]";
Review Comment:
somewhere us baseDirName D and here it is baseDirName N
Shouldn't be inconsistent I believe
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java:
##########
@@ -736,7 +746,7 @@ class OpenFileStats extends CreateFileStats {
static final String OP_OPEN_NAME = "open";
static final String OP_USAGE_ARGS =
" [-threads T] [-files N] [-blockSize S] [-filesPerDir P]"
- + " [-useExisting]";
+ + " [-baseDirName D] [-useExisting]";
Review Comment:
add the new parameter towards the end
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNNThroughputBenchmark.java:
##########
@@ -60,6 +60,18 @@ public void testNNThroughput() throws Exception {
NNThroughputBenchmark.runBenchmark(conf, new String[] {"-op", "all"});
}
+ @Test
+ public void testNNThroughputWithBaseDir() throws Exception {
+ Configuration conf = new HdfsConfiguration();
+ conf.setInt(DFSConfigKeys.DFS_BLOCK_SIZE_KEY, 16);
+ File nameDir = new File(MiniDFSCluster.getBaseDirectory(), "name");
+ conf.set(DFSConfigKeys.DFS_NAMENODE_NAME_DIR_KEY,
+ nameDir.getAbsolutePath());
+ DFSTestUtil.formatNameNode(conf);
+ NNThroughputBenchmark.runBenchmark(conf,
+ new String[] {"-op", "all", "-baseDirName",
"/nnThroughputBenchmark1"});
Review Comment:
test should check ``nnThroughputBenchmark1 `` was used & the default
directory wasn't created nor used.
try with other operations as well apart from ``all``
> NNThroughputBenchmark should support specifying the base directory for
> multi-client test
> ----------------------------------------------------------------------------------------
>
> Key: HDFS-17272
> URL: https://issues.apache.org/jira/browse/HDFS-17272
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Affects Versions: 3.4.0
> Reporter: caozhiqiang
> Assignee: caozhiqiang
> Priority: Major
> Labels: pull-request-available
>
> Currently, NNThroughputBenchmark does not support specifying the base
> directory, therefore does not support multiple clients performing stress
> testing at the same time. However, for high-performance namenode machine,
> only one client submitting stress test can not make the namenode rpc access
> reach the bottleneck. Therefore, multiple clients are required for parallel
> testing to make the namenode pressure reach the level of the large-scale
> production cluster.
> So I specify the base directory through the -baseDirName parameter to support
> multiple clients submitting stress tests at the same time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
