[
https://issues.apache.org/jira/browse/YARN-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851507#comment-13851507
]
Luke Lu commented on YARN-938:
------------------------------
Thanks for the results Jeff!. It's interesting to note that the best terasort
throughput in your configuration is ~140MB/s (mrv1, 96MB/s for mrv2) per
physical host for a 8TB data set, compared with ~23MB/s (1.x, 21MB/s for 2.2)
per physical host in Mayank's results for a 1TB (?) data set. Obviously 10Gb
networking and 12 15K RPM SAS disks per host helped. OTOH, I'd expect Mayank's
results to be a lot faster as the data set fits into the 260 slave host cluster
memory (buffer cache).
It'll be interesting to show the Apache 1.2.1 results for Jeff's configuration
as well, so it's more comparable to Mayank's results, as I suspect that CDH
mrv1 have more optimizations than Apache.
> Hadoop 2 benchmarking
> ----------------------
>
> Key: YARN-938
> URL: https://issues.apache.org/jira/browse/YARN-938
> Project: Hadoop YARN
> Issue Type: Task
> Reporter: Mayank Bansal
> Assignee: Mayank Bansal
> Attachments: Hadoop-benchmarking-2.x-vs-1.x-1.xls,
> Hadoop-benchmarking-2.x-vs-1.x.xls, cdh500beta1_cpu_util.jpg,
> cdh500beta1_mr1_mr2.xlsx
>
>
> I am running the benchmarks on Hadoop 2 and will update the results soon.
> Thanks,
> Mayank
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
