Hello, these are results from 3 and 6 nodes cluster. 3modes Throughput mb/sec: 16.32167574644889
6nodes Throughput mb/sec: 23.875848571907156 The insertion time of data is ~triple with 3nodes, but throughput s are near same. Whats the meaning of that throughput is? Thanks 3nodes 11/05/03 17:34:23 INFO mapred.FileInputFormat: ----- TestDFSIO ----- : write 11/05/03 17:34:23 INFO mapred.FileInputFormat: Date & time: Tue May 03 17:34:23 CEST 2011 11/05/03 17:34:23 INFO mapred.FileInputFormat: Number of files: 1000 11/05/03 17:34:23 INFO mapred.FileInputFormat: Total MBytes processed: 128000 11/05/03 17:34:23 INFO mapred.FileInputFormat: Throughput mb/sec: 16.32167574644889 11/05/03 17:34:23 INFO mapred.FileInputFormat: Average IO rate mb/sec: 21.17328643798828 11/05/03 17:34:23 INFO mapred.FileInputFormat: IO rate std deviation: 5.105609326092506 11/05/03 17:34:23 INFO mapred.FileInputFormat: Test exec time sec: 1912.383 6 nodes 11/05/03 16:21:46 INFO mapred.FileInputFormat: ----- TestDFSIO ----- : write 11/05/03 16:21:46 INFO mapred.FileInputFormat: Date & time: Tue May 03 16:21:46 CEST 2011 11/05/03 16:21:46 INFO mapred.FileInputFormat: Number of files: 1000 11/05/03 16:21:46 INFO mapred.FileInputFormat: Total MBytes processed: 128000 11/05/03 16:21:46 INFO mapred.FileInputFormat: Throughput mb/sec: 23.875848571907156 11/05/03 16:21:46 INFO mapred.FileInputFormat: Average IO rate mb/sec: 25.087316513061523 11/05/03 16:21:46 INFO mapred.FileInputFormat: IO rate std deviation: 5.885708523356224 11/05/03 16:21:46 INFO mapred.FileInputFormat: Test exec time sec: 734.372 11/05/03 16:21:46 INFO mapred.FileInputFormat: 2011/5/3 pob <peterob...@gmail.com> > Hello, > > I run hadoop jar hadoop-0.20.2-test.jar TestDFSIO -write -nrFiles 1000 > -fileSize 128, because wanted to know what is the througput of my cluster > (how many data per sec its able to write). > > Here are results, that doesnt have any "meaning" for my point of view. > > 11/05/03 16:21:46 INFO mapred.FileInputFormat: Throughput mb/sec: > 23.875848571907156 > 11/05/03 16:21:46 INFO mapred.FileInputFormat: Average IO rate mb/sec: > 25.087316513061523 > > The througput mb/s suppose to be: total MB written / total time. > Apparently not? Or its for one node or what does it mean? > > During the test i was monitoring all nodes disk and there was stale write > traffic of 120MB/s. > > > Thanks > > > Best > P >
