[
https://issues.apache.org/jira/browse/HDFS-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069631#comment-14069631
]
Colin Patrick McCabe commented on HDFS-6482:
--------------------------------------------
bq. One thread per storage directory doesn't make sense here since this is the
number of threads to use for the hard link process for ONE storage directory.
The hard link processes for the storage directories are currently not run in
parallel.
Understood. It seems like we should be parallelizing the upgrade of different
storage directories, since clearly we'd like to keep all those disks busy if we
could. Anyway, this JIRA is big enough as-is, so let's not worry about it
right now.
James, given that the you've gotten the upgrade times in the single seconds
now, I am +1 on putting this change in 2.x. [~sureshms], [~atm], what are your
thoughts here?
> Use block ID-based block layout on datanodes
> --------------------------------------------
>
> Key: HDFS-6482
> URL: https://issues.apache.org/jira/browse/HDFS-6482
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 2.5.0
> Reporter: James Thomas
> Assignee: James Thomas
> Attachments: 6482-design.doc, HDFS-6482.1.patch, HDFS-6482.2.patch,
> HDFS-6482.3.patch, HDFS-6482.4.patch, HDFS-6482.5.patch, HDFS-6482.6.patch,
> HDFS-6482.7.patch, HDFS-6482.8.patch, HDFS-6482.9.patch, HDFS-6482.patch,
> hadoop-24-datanode-dir.tgz
>
>
> Right now blocks are placed into directories that are split into many
> subdirectories when capacity is reached. Instead we can use a block's ID to
> determine the path it should go in. This eliminates the need for the LDir
> data structure that facilitates the splitting of directories when they reach
> capacity as well as fields in ReplicaInfo that keep track of a replica's
> location.
> An extension of the work in HDFS-3290.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
