[
https://issues.apache.org/jira/browse/HADOOP-15292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387190#comment-16387190
]
Íñigo Goiri commented on HADOOP-15292:
--------------------------------------
Thanks [~virajith] for the patch.
{{TestCopyMapper}} tests this behavior so we can check that this doesn't break.
Not sure if it's worth extending that unit test to track how many times we open
the stream.
Probably not worth adding metrics but maybe extend the stream in the unit test
and track how many times we open it.
[~jingzhao] you implemented MAPREDUCE-5899, do you mind double checking that
this approach is correct?
> Distcp's use of pread is slowing it down.
> -----------------------------------------
>
> Key: HADOOP-15292
> URL: https://issues.apache.org/jira/browse/HADOOP-15292
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Virajith Jalaparti
> Priority: Major
> Attachments: HADOOP-15292.000.patch
>
>
> Distcp currently uses positioned-reads (in
> RetriableFileCopyCommand#copyBytes) when the source offset is > 0. This
> results in unnecessary overheads (new BlockReader being created on the
> client-side, multiple readBlock() calls to the Datanodes, each of requires
> the creation of a BlockSender and an inputstream to the ReplicaInfo).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
