[
https://issues.apache.org/jira/browse/HADOOP-15020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16241962#comment-16241962
]
Brahma Reddy Battula commented on HADOOP-15020:
-----------------------------------------------
dupe of MAPREDUCE-6363 which is availble from {{2.7.2}}..?
> NNBench not support run more than one map task on the same host
> ---------------------------------------------------------------
>
> Key: HADOOP-15020
> URL: https://issues.apache.org/jira/browse/HADOOP-15020
> Project: Hadoop Common
> Issue Type: Improvement
> Components: benchmarks
> Affects Versions: 2.7.2
> Environment: Hadoop 2.7.2
> Reporter: zhoutai.zt
> Priority: Minor
>
> When benchmark NameNode performance with NNBench, I start with pseudo
> distributed deploy. Everything goes well with "-maps 1". BUT with -maps N
> (n>1) and -operation create_write, many exceptions meet during the benchmark.
> Hostname is part of the file path, which can differentiate hosts. With more
> than two map tasks run on the same host, more than two map tasks may operate
> on the same file, leading to exceptions.
> 17/11/07 15:22:32 INFO hdfs.NNBench: RAW DATA: AL Total #1:
> 84
> 17/11/07 15:22:32 INFO hdfs.NNBench: RAW DATA: AL Total #2:
> 43
> 17/11/07 15:22:32 INFO hdfs.NNBench: RAW DATA: TPS Total (ms):
> 2570
> 17/11/07 15:22:32 INFO hdfs.NNBench: RAW DATA: Longest Map Time (ms):
> 814.0
> 17/11/07 15:22:32 INFO hdfs.NNBench: RAW DATA: Late maps: 0
> 17/11/07 15:22:32 INFO hdfs.NNBench: {color:red}RAW DATA: # of
> exceptions: 3000{color}
> 2017-11-07 14:54:08,082 INFO org.apache.hadoop.hdfs.NNBench: Exception
> recorded in op: Create/Write/Close
> 2017-11-07 14:54:08,082 INFO org.apache.hadoop.hdfs.NNBench: Exception
> recorded in op: Create/Write/Close
> 2017-11-07 14:54:08,083 INFO org.apache.hadoop.hdfs.NNBench: Exception
> recorded in op: Create/Write/Close
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
