[
https://issues.apache.org/jira/browse/HDFS-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo (Nicholas), SZE updated HDFS-4990:
-----------------------------------------
Attachment: h4990_20130918.patch
Thanks Arpit for the comments.
> ... It does not check the usage on the Storage so it could keep picking a
> full storage. ...
Good catch. We should check usage. We should also move getBlocksScheduled()
to DatanodeStorageInfo. Let's do the move later.
> ... Also we should probably try to round robin the storages. ...
Let's just randomize the storages. Then, we won't need to add any new state.
h4990_20130918.patch: shuffles storage array, checks storage remaining size and
cleans up some code.
> Block placement for storage types
> ---------------------------------
>
> Key: HDFS-4990
> URL: https://issues.apache.org/jira/browse/HDFS-4990
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Reporter: Suresh Srinivas
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h4990_20130909.patch, h4990_20130916.patch,
> h4990_20130917b.patch, h4990_20130917c.patch, h4990_20130917.patch,
> h4990_20130918.patch
>
>
> Currently block location for writes are made based on:
> # Datanode load (number of transceivers)
> # Space left on datanode
> # Topology
> With storage abstraction, namenode must choose a storage instead of a
> datanode for block placement. It also needs to consider storage type, load on
> the storage etc.
> As an additional benefit, currently HDFS support heterogeneous nodes (nodes
> with different number of spindles etc.) poorly. This work should help solve
> that issue as well.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
