[
https://issues.apache.org/jira/browse/HADOOP-14407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004854#comment-16004854
]
Yongjun Zhang commented on HADOOP-14407:
----------------------------------------
Hi [~omkarksa], thanks for the good finding. Do we want to have separate config
for the input and output buffer size? In your test, did you use the same buf
size for both?
Thanks.
> DistCp - Introduce a configurable copy buffer size
> --------------------------------------------------
>
> Key: HADOOP-14407
> URL: https://issues.apache.org/jira/browse/HADOOP-14407
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tools/distcp
> Affects Versions: 2.9.0
> Reporter: Omkar Aradhya K S
> Assignee: Yongjun Zhang
> Fix For: 2.9.0, 3.0.0-alpha3
>
>
> Currently, the RetriableFileCopyCommand has a fixed copy buffer size of just
> 8KB. We have noticed in our performance tests that with bigger buffer sizes
> we saw upto ~3x performance boost. Hence, making the copy buffer size a
> configurable setting via the new parameter <copybuffersize>.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
