[
https://issues.apache.org/jira/browse/HDFS-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957343#comment-16957343
]
David Mollitor commented on HDFS-14872:
---------------------------------------
Might be able to create a new copy routine at a higher level with the existing
HDFS FS API. Need to check,
https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/fs/FileUtil.html#copy-java.io.File-org.apache.hadoop.fs.FileSystem-org.apache.hadoop.fs.Path-boolean-org.apache.hadoop.conf.Configuration-
> Read HDFS Blocks in Random Order
> --------------------------------
>
> Key: HDFS-14872
> URL: https://issues.apache.org/jira/browse/HDFS-14872
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: hdfs-client
> Affects Versions: 2.8.5, 3.2.1
> Reporter: David Mollitor
> Priority: Major
>
> When the HDFS client is downloading (copying) an entire file, allow the
> client to download the blocks in random order. If a lot of clients are
> reading the same file, in parallel, they will all download the first block,
> the second block, and so on, stampeding down the line.
> It would be interesting to spread the load across across all the available
> DataNodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
