[
https://issues.apache.org/jira/browse/HADOOP-15292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16391120#comment-16391120
]
Hudson commented on HADOOP-15292:
---------------------------------
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13796 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13796/])
HADOOP-15292. Distcp's use of pread is slowing it down. Contributed by (stevel:
rev 3bd6b1fd85c44354c777ef4fda6415231505b2a4)
* (edit)
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
* (edit)
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java
* (edit)
hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
> Distcp's use of pread is slowing it down.
> -----------------------------------------
>
> Key: HADOOP-15292
> URL: https://issues.apache.org/jira/browse/HADOOP-15292
> Project: Hadoop Common
> Issue Type: Bug
> Components: tools/distcp
> Affects Versions: 2.5.0
> Reporter: Virajith Jalaparti
> Assignee: Virajith Jalaparti
> Priority: Minor
> Fix For: 3.1.0
>
> Attachments: HADOOP-15292.000.patch, HADOOP-15292.001.patch,
> HADOOP-15292.002.patch
>
>
> Distcp currently uses positioned-reads (in
> RetriableFileCopyCommand#copyBytes) when the source offset is > 0. This
> results in unnecessary overheads (new BlockReader being created on the
> client-side, multiple readBlock() calls to the Datanodes, each of which
> requires the creation of a BlockSender and an inputstream to the ReplicaInfo).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
