[
https://issues.apache.org/jira/browse/HDFS-8198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14564106#comment-14564106
]
Takuya Fukudome commented on HDFS-8198:
---------------------------------------
I report the results I ran teragen and terasort on our test cluster. The number
of rows, teragen parameter was set 100m(It wrote 10G byte data).
Result
_elapsed time_
|| || non EC teragen || EC teragen || non EC terasort || EC terasort ||
|| 1 | 1m2.486s | 3m3.966s | 2m56.277s | 6m45.136s |
|| 2 | 1m2.609s | 2m55.928s | 3m4.428s | 6m11.019s |
|| 3 | 1m8.516s | 2m51.004s | 2m58.427s | 6m3.055s |
And I checked "Total time spent by all maps/reduces in occupied slots(ms)"
_Maps_
|| || non EC teragen || EC teragen || non EC terasort || EC terasort ||
|| 1 | 103591 | 335320 | 628538 | 701388 |
|| 2 | 102937 | 322062 | 640839 | 719531 |
|| 3 | 113472 | 313274 | 631408 | 654707 |
_Reduces_
|| || non EC teargen || EC teragen || non EC terasort || EC terasort ||
|| 1 | \- | \- | 155554 | 383402 |
|| 2 | \- | \- | 162759 | 348135 |
|| 3 | \- | \- | 156585 | 340584 |
About our test cluster
|| CPU | 2CPU(Xeon E5-2660v2 2.2GHz) |
|| RAM | 128GB |
The number of Data Nodes: 39
Network bandwidth: 10Gbps
> Erasure Coding: system test of TeraSort
> ---------------------------------------
>
> Key: HDFS-8198
> URL: https://issues.apache.org/jira/browse/HDFS-8198
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: HDFS-7285
> Reporter: Kai Sasaki
>
> Functional system test of TeraSort on EC files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
