Hi Expert, I am using nnbench to test the namenode performance,I am using this command: hadoop jar /opt/hadoop-2.7.4/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.4.jar nnbench -operation create_write -maps 10 -reduces 5 -numberOfFiles 100 -replicationFactorPerFile 2 -blockSize 128 -bytesToWrite 20 -readFileAfterOpen true and I got below output: 18/05/17 16:59:29 INFO hdfs.NNBench: -------------- NNBench -------------- : 18/05/17 16:59:29 INFO hdfs.NNBench: Version: NameNode Benchmark 0.4 18/05/17 16:59:29 INFO hdfs.NNBench: Date & time: 2018-05-17 16:59:29,870 18/05/17 16:59:29 INFO hdfs.NNBench: 18/05/17 16:59:29 INFO hdfs.NNBench: Test Operation: create_write 18/05/17 16:59:29 INFO hdfs.NNBench: Start time: 2018-05-17 16:59:22,127 18/05/17 16:59:29 INFO hdfs.NNBench: Maps to run: 10 18/05/17 16:59:29 INFO hdfs.NNBench: Reduces to run: 5 18/05/17 16:59:29 INFO hdfs.NNBench: Block Size (bytes): 128 18/05/17 16:59:29 INFO hdfs.NNBench: Bytes to write: 20 18/05/17 16:59:29 INFO hdfs.NNBench: Bytes per checksum: 1 18/05/17 16:59:29 INFO hdfs.NNBench: Number of files: 100 18/05/17 16:59:29 INFO hdfs.NNBench: Replication factor: 2 18/05/17 16:59:29 INFO hdfs.NNBench: Successful file operations: 0 18/05/17 16:59:29 INFO hdfs.NNBench: 18/05/17 16:59:29 INFO hdfs.NNBench: # maps that missed the barrier: 0 18/05/17 16:59:29 INFO hdfs.NNBench: # exceptions: 10000 18/05/17 16:59:29 INFO hdfs.NNBench: 18/05/17 16:59:29 INFO hdfs.NNBench: TPS: Create/Write/Close: 0 18/05/17 16:59:29 INFO hdfs.NNBench: Avg exec time (ms): Create/Write/Close: Infinity 18/05/17 16:59:29 INFO hdfs.NNBench: Avg Lat (ms): Create/Write: NaN 18/05/17 16:59:29 INFO hdfs.NNBench: Avg Lat (ms): Close: NaN 18/05/17 16:59:29 INFO hdfs.NNBench: 18/05/17 16:59:29 INFO hdfs.NNBench: RAW DATA: AL Total #1: 0 18/05/17 16:59:29 INFO hdfs.NNBench: RAW DATA: AL Total #2: 0 18/05/17 16:59:29 INFO hdfs.NNBench: RAW DATA: TPS Total (ms): 14924 18/05/17 16:59:29 INFO hdfs.NNBench: RAW DATA: Longest Map Time (ms): 1838.0 18/05/17 16:59:29 INFO hdfs.NNBench: RAW DATA: Late maps: 0 18/05/17 16:59:29 INFO hdfs.NNBench: RAW DATA: # of exceptions: 10000
My question is that how to understand the output,how to measure the performance by the output.is there any official instruction about this? Yet another performance test : [apache@namenode01 sbin]$ hadoop org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark -fs hdfs://namenode01:8020 -op create -threads 100 -files 100000 -filesPerDir 4 -close 18/05/18 11:55:48 INFO namenode.NNThroughputBenchmark: Starting benchmark: create 18/05/18 11:55:48 INFO namenode.NNThroughputBenchmark: Generate 100000 intputs for create 18/05/18 11:55:49 FATAL namenode.NNThroughputBenchmark: Log level = ERROR 18/05/18 11:55:49 INFO namenode.NNThroughputBenchmark: Starting 100000 create(s). 18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: 18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: --- create inputs --- 18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: nrFiles = 100000 18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: nrThreads = 100 18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: nrFilesPerDir = 4 18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: --- create stats --- 18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: # operations: 100000 ###number of files has been created 18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: Elapsed Time: 14041 ###how much time has been used to create 100000 files 18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: Ops per sec: 7121.999857560003 ### how many files has been created per second 18/05/18 11:56:03 INFO namenode.NNThroughputBenchmark: Average Time: 6 ######## what dose this number mean?how to calculate this?########### Which test tool do you prefer? Any help would be highly appreciated! Best Regards, 倪项菲/ David Ni 中移德电网络科技有限公司 Virtue Intelligent Network Ltd, co. Add: 2003,20F No.35 Luojia creative city,Luoyu Road,Wuhan,HuBei Mob: +86 13797007811|Tel: + 86 27 5024 2516
