[
https://issues.apache.org/jira/browse/HDFS-8425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takuya Fukudome updated HDFS-8425:
----------------------------------
Attachment: testdfsio-read-mbsec.png
testdfsio-write-mbsec.png
Hi. I have ran TestDFSIO on both normal directory and the
directory with EC policy set.
I attached two charts which respectively show write and
read throughput(mb/sec) of both replicaions files and EC files.
And the throughputs are calculated by dividing the total
bytes of TestDFSIO's data by the total elapsed time.
In summary, writing EC files is better than writing
replication files at throughput. And reading EC files is the
same performance as reading replication files.
Though DataNodes' average CPU usage of writing EC files raised 5.5% comparing
to writing replication files(from 9.8% to 15.3%).
The specification of our test cluster is bellow
|| Number of DataNodes | 20 |
server info:
|| CPU | Xeon E5-2630L 2.00GHz/2CPU |
|| RAM | 64GB |
|| Disk | SATA 300 |
Our test cluster was build with trunk codes. Its commit
revision id is r30e2f836a26490a24c7ddea754dd19f95b24bbc8.
Those are initial performance test result, we are still working
on further test. Please let me know if the initial test result
make sense to you. Any advise is welcome! Thank you.
> [umbrella] Performance tuning, investigation and optimization for erasure
> coding
> --------------------------------------------------------------------------------
>
> Key: HDFS-8425
> URL: https://issues.apache.org/jira/browse/HDFS-8425
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: HDFS-7285
> Reporter: GAO Rui
> Attachments: testClientWriteReadFile_v1.pdf,
> testdfsio-read-mbsec.png, testdfsio-write-mbsec.png
>
>
> This {{umbrella}} jira aims to track performance tuning, investigation and
> optimization for erasure coding.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
