[
https://issues.apache.org/jira/browse/MAPREDUCE-4651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455895#comment-13455895
]
Ravi Prakash commented on MAPREDUCE-4651:
-----------------------------------------
Thanks Konstantin! I applied the patch and ran the random and backward read
tests on my single node dev box.
{noformat}
$HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -random
-fileSize 10MB
Average IO rate mb/sec: 134.43310546875
IO rate std deviation: 0.00896365222201456
$HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -backward
-fileSize 10MB
Average IO rate mb/sec: 134.49253845214844
IO rate std deviation: 0.026679629420752023
$HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -random
-fileSize 1GB
Average IO rate mb/sec: 249.47183227539062
IO rate std deviation: 0.014617091655162118
$HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -backward
-fileSize 1GB
Average IO rate mb/sec: 295.8538818359375
IO rate std deviation: 0.061419808441541615
$HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -random
-fileSize 10GB
Average IO rate mb/sec: 320.3417663574219
IO rate std deviation: 0.05935480659067817
$HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -backward
-fileSize 10GB
Average IO rate mb/sec: 323.28045654296875
IO rate std deviation: 0.0598550775330073
$HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -backward
-fileSize 30GB
Average IO rate mb/sec: 390.9880065917969
IO rate std deviation: 0.06083891027478396
$HADOOP_PREFIX/bin/hadoop org.apache.hadoop.fs.TestDFSIO -read -random
-fileSize 30GB
Average IO rate mb/sec: 369.2136535644531
IO rate std deviation: 0.056819116587427144
{noformat}
Could you please post recommended usage? And at what sizes do we expect to
achieve stable IO rates?
> Benchmarking random reads with DFSIO
> ------------------------------------
>
> Key: MAPREDUCE-4651
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4651
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: benchmarks, test
> Affects Versions: 1.0.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Attachments: randomDFSIO.patch, randomDFSIO.patch
>
>
> TestDFSIO measures throughput of HDFS write, read, and append operations. It
> will be useful to have an option to use it for benchmarking random reads.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
