[
https://issues.apache.org/jira/browse/HDFS-7068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14357806#comment-14357806
]
Zhe Zhang commented on HDFS-7068:
---------------------------------
Very good thoughts [~walter.k.su]! And thanks Kai for the helpful comments.
I think option #1 is the lightest in terms of dev effort. Assuming _all EC
files use a single placement policy_, that should work for us. Right now I
don't see a need for multiple EC placement policies. The basic logic is just to
spread across as many racks as possible based on m and k. So maybe we should
start with implementing option #1.
If we all agree with this option, then I imagine the change should look like
HDFS-3601.
> Support multiple block placement policies
> -----------------------------------------
>
> Key: HDFS-7068
> URL: https://issues.apache.org/jira/browse/HDFS-7068
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Affects Versions: 2.5.1
> Reporter: Zesheng Wu
> Assignee: Walter Su
> Attachments: HDFS-7068.patch
>
>
> According to the code, the current implement of HDFS only supports one
> specific type of block placement policy, which is BlockPlacementPolicyDefault
> by default.
> The default policy is enough for most of the circumstances, but under some
> special circumstances, it works not so well.
> For example, on a shared cluster, we want to erasure encode all the files
> under some specified directories. So the files under these directories need
> to use a new placement policy.
> But at the same time, other files still use the default placement policy.
> Here we need to support multiple placement policies for the HDFS.
> One plain thought is that, the default placement policy is still configured
> as the default. On the other hand, HDFS can let user specify customized
> placement policy through the extended attributes(xattr). When the HDFS choose
> the replica targets, it firstly check the customized placement policy, if not
> specified, it fallbacks to the default one.
> Any thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
