Hi Expert,
  I am using nnbench to test the namenode performance,I am using this command:
  hadoop jar 
/opt/hadoop-2.7.4/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.4.jar
 nnbench -operation create_write -maps 10 -reduces 5 -numberOfFiles 100 
-replicationFactorPerFile 2 -blockSize 128 -bytesToWrite 20  -readFileAfterOpen 
true
  and I got below output:
18/05/17 16:59:29 INFO hdfs.NNBench: -------------- NNBench -------------- :
18/05/17 16:59:29 INFO hdfs.NNBench:                                Version: 
NameNode Benchmark 0.4
18/05/17 16:59:29 INFO hdfs.NNBench:                            Date & time: 
2018-05-17 16:59:29,870
18/05/17 16:59:29 INFO hdfs.NNBench:
18/05/17 16:59:29 INFO hdfs.NNBench:                         Test Operation: 
create_write
18/05/17 16:59:29 INFO hdfs.NNBench:                             Start time: 
2018-05-17 16:59:22,127
18/05/17 16:59:29 INFO hdfs.NNBench:                            Maps to run: 10
18/05/17 16:59:29 INFO hdfs.NNBench:                         Reduces to run: 5
18/05/17 16:59:29 INFO hdfs.NNBench:                     Block Size (bytes): 128
18/05/17 16:59:29 INFO hdfs.NNBench:                         Bytes to write: 20
18/05/17 16:59:29 INFO hdfs.NNBench:                     Bytes per checksum: 1
18/05/17 16:59:29 INFO hdfs.NNBench:                        Number of files: 100
18/05/17 16:59:29 INFO hdfs.NNBench:                     Replication factor: 2
18/05/17 16:59:29 INFO hdfs.NNBench:             Successful file operations: 0
18/05/17 16:59:29 INFO hdfs.NNBench:
18/05/17 16:59:29 INFO hdfs.NNBench:         # maps that missed the barrier: 0
18/05/17 16:59:29 INFO hdfs.NNBench:                           # exceptions: 
10000
18/05/17 16:59:29 INFO hdfs.NNBench:
18/05/17 16:59:29 INFO hdfs.NNBench:                TPS: Create/Write/Close: 0
18/05/17 16:59:29 INFO hdfs.NNBench: Avg exec time (ms): Create/Write/Close: 
Infinity
18/05/17 16:59:29 INFO hdfs.NNBench:             Avg Lat (ms): Create/Write: NaN
18/05/17 16:59:29 INFO hdfs.NNBench:                    Avg Lat (ms): Close: NaN
18/05/17 16:59:29 INFO hdfs.NNBench:
18/05/17 16:59:29 INFO hdfs.NNBench:                  RAW DATA: AL Total #1: 0
18/05/17 16:59:29 INFO hdfs.NNBench:                  RAW DATA: AL Total #2: 0
18/05/17 16:59:29 INFO hdfs.NNBench:               RAW DATA: TPS Total (ms): 
14924
18/05/17 16:59:29 INFO hdfs.NNBench:        RAW DATA: Longest Map Time (ms): 
1838.0
18/05/17 16:59:29 INFO hdfs.NNBench:                    RAW DATA: Late maps: 0
18/05/17 16:59:29 INFO hdfs.NNBench:              RAW DATA: # of exceptions: 
10000

My question is that how to understand the output,how to measure the performance 
by the output.is there any official instruction about this?

Yet another performance test :
[apache@namenode01 sbin]$ hadoop 
org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark -fs 
hdfs://namenode01:8020 -op create -threads 100 -files 100000 -filesPerDir 4 
-close
18/05/18 11:55:48 INFO namenode.NNThroughputBenchmark: Starting benchmark: 
create
18/05/18 11:55:48 INFO namenode.NNThroughputBenchmark: Generate 100000 intputs 
for create
18/05/18 11:55:49 FATAL namenode.NNThroughputBenchmark: Log level = ERROR
18/05/18 11:55:49 INFO namenode.NNThroughputBenchmark: Starting 100000 
create(s).
18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark:
18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: --- create inputs ---
18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: nrFiles = 100000
18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: nrThreads = 100
18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: nrFilesPerDir = 4
18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: --- create stats  ---
18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: # operations: 100000  
###number of files has been created
18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: Elapsed Time: 14041   
###how much time has been used to create 100000 files
18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark:  Ops per sec: 
7121.999857560003  ### how many files has been created per second
18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: Average Time: 6   
######## what dose this number mean?how to calculate this?###########

Which test tool do you prefer?

Any help would be highly appreciated!



Best Regards,

倪项菲/ David Ni
中移德电网络科技有限公司
Virtue Intelligent Network Ltd, co.
Add: 2003,20F No.35 Luojia creative city,Luoyu Road,Wuhan,HuBei
Mob: +86 13797007811|Tel: + 86 27 5024 2516

Reply via email to