[
https://issues.apache.org/jira/browse/HDFS-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HDFS-9421:
--------------------------------
Attachment: HDFS-9241.000.patch
Per discussion in [HDFS-9387], the {{replication}} test is not supporting
namenode in another process or on another host.
In the v0 patch:
- If arguments include {{-op replication -namenode URI}}, the test will be
ignored, along with a warning message for ignoring the {{replication}} test.
- If arguments include {{-op all -namenode URI}}, all tests other than
{{replication}} will run against the standalone name node URI, along with a
warning message.
> NNThroughputBenchmark replication test NPE with -namenode option
> ----------------------------------------------------------------
>
> Key: HDFS-9421
> URL: https://issues.apache.org/jira/browse/HDFS-9421
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: benchmarks
> Reporter: Xiaoyu Yao
> Assignee: Mingliang Liu
> Attachments: HDFS-9241.000.patch
>
>
> Hit the following NPE when reviewing fix for HDFS-9387 with manual tests as
> NNThroughputBenchmark currently does not have JUnit tests.
>
> {code}
> HW11217:centos6.4 xyao$ hadoop
> org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark -op replication
> -namenode hdfs://HW11217.local:9000
> 15/11/12 14:52:03 INFO namenode.NNThroughputBenchmark: Starting benchmark:
> replication
> 15/11/12 14:52:03 ERROR namenode.NNThroughputBenchmark:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark$ReplicationStats.generateInputs(NNThroughputBenchmark.java:1312)
> at
> org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark$OperationStatsBase.benchmark(NNThroughputBenchmark.java:280)
> at
> org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark.run(NNThroughputBenchmark.java:1509)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at
> org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark.main(NNThroughputBenchmark.java:1534)
> ...
> {code}
> However, the root cause is different from HDFS-9387.
> From ReplicationStats#generateInputs, *nameNode* is uninitialized before use,
> which causes the NPE.
> {code}
> final FSNamesystem namesystem = nameNode.getNamesystem();
> {code}
> From NNThroughputBenchmark#run, nameNode is only initialized when -namenode
> option is not specified. The fix is to initialize it properly in the else
> block when -namenode option is specified OR we should block this if it is not
> supported.
> {code}
> if (namenodeUri == null) {
> nameNode = NameNode.createNameNode(argv, config);
> NamenodeProtocols nnProtos = nameNode.getRpcServer();
> nameNodeProto = nnProtos;
> clientProto = nnProtos;
> dataNodeProto = nnProtos;
> refreshUserMappingsProto = nnProtos;
> bpid = nameNode.getNamesystem().getBlockPoolId();
> } else {
> FileSystem.setDefaultUri(getConf(), namenodeUri);
> DistributedFileSystem dfs = (DistributedFileSystem)
> FileSystem.get(getConf());
> final URI nnUri = new URI(namenodeUri);
> nameNodeProto = DFSTestUtil.getNamenodeProtocolProxy(config, nnUri,
> UserGroupInformation.getCurrentUser());
>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
