[
https://issues.apache.org/jira/browse/HDFS-9355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026254#comment-15026254
]
nijel commented on HDFS-9355:
-----------------------------
Idea looks good
But co-location is a very broad topic. So i suggest to focus on favored nodes
optimization as part of this JIRA as you mentioned.
One option is to give a client API to get the DNs based on storage policy.
> Support colocation in HDFS.
> ---------------------------
>
> Key: HDFS-9355
> URL: https://issues.apache.org/jira/browse/HDFS-9355
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: hdfs-client
> Reporter: Surendra Singh Lilhore
> Assignee: Surendra Singh Lilhore
>
> Through this feature client can give suggestion to HDFS to write his all the
> blocks on same set of datanodes. Currently this we can achieve through
> HDFS-2576. HDFS-2576 give option to hint namenode about favored nodes, but in
> heterogeneous cluster this will not work out. Support client wants to write
> his data in directory which have COLD policy, but he don't know which DN have
> ARCHIVE storage, So he will not able to give favoredNodes list.
> *Implementation*
> Colocation can enable by setting "dfs.colocation.enable" true in client
> configuration. If colocation is enable and favoredNodes list is empty then
> {{DataStreamer}} will set first set of datanodes as favoredNodes which is
> chosen for first block and subsequent block will use the same datanodes for
> write. Before closing file client can get the favoredNodes list and same he
> can use for writing new file.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
