[
https://issues.apache.org/jira/browse/HDFS-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942067#comment-16942067
]
Erik Krogen commented on HDFS-14872:
------------------------------------
+1 on this idea from me, we have considered building the same internally using
the same pre-allocation strategy that you've described. For a large file with
numerous blocks, it should be able to make a big difference.
> Read HDFS Blocks in Random Order
> --------------------------------
>
> Key: HDFS-14872
> URL: https://issues.apache.org/jira/browse/HDFS-14872
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: hdfs-client
> Affects Versions: 2.8.5, 3.2.1
> Reporter: David Mollitor
> Priority: Major
>
> When the HDFS client is downloading (copying) an entire file, allow the
> client to download the blocks in random order. If a lot of clients are
> reading the same file, in parallel, they will all download the first block,
> the second block, and so on, stampeding down the line.
> It would be interesting to spread the load across across all the available
> DataNodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
