Hi. Can you provide more information about your workload and the > environment? eg are you running t.o.a.h.h.BenchmarkThroughput, > TestDFSIO, or timing hadoop fs -put/get to transfer data to hdfs from > another machine, looking at metrics, etc. What else is running on the > cluster? Have you profiled? etc. 77Mb/s (<10MB/s) seems low but w/o > context is not meaningful. >
I actually tested it with a simple Java test loader I quickly put together, which ran on each machine and continuously has written random data to DFS. I tuned the writing rate until I got ~77Mb/s - above it the iowait loads on each disk (measured by iostat) became above 50% - 60%, which is quite close to disks limits. If there is a more official / better way to do it, I'll be happy to get some pointers to it. You mentioned some TestDFSIO, any idea if it's present in 0.18.3? Regards.
