[
https://issues.apache.org/jira/browse/HDFS-9006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947903#comment-14947903
]
Lei (Eddy) Xu commented on HDFS-9006:
-------------------------------------
Thanks for working on this, [~mingma]. In general, it looks great to me.
A few small nitpicks:
* We can make class members in {{BlockPlacementPolicyWithUpgradeDomain}} and
{{BlockPlacementStatusWithUpgradeDomain}} to be {{private final}}.
* Since we are using JDK7 now, we can use type inference to make the code
consistent with new code base, e.g.,
{{Set<String> upgradeDomains = new HashSet<String>();}} to {{... = new
HashSet<>();}}
* Can you make some adjustments on the comments for
{{BlockPlacementPolicyWithUpgradeDomain#pickupReplicaSet}}, it is a little bit
hard to understand. I'd suggest to modify the comments in the following aspects:
1. Describes that it is for "picking the node for deleting the
over-replicated replica" and meet the rack and upgrade domain policy.
2. Make {{S1}} and {{S2}} terminology consistent to
{{moreThanOneDomainSet}} and {{shareRackAndUPgradeDomainSet}}?
3. Is the order of choosing replicas: (rack + upgrade domain(UD)) > (no
rack + UD) > (rack + no UD) > (no rack + no UD)? Can we explicitly describe
it in the comments?
4. should {{upgradeDomainFactor}} equal to {{replicaFactor}} in general ?
e.g., does it make sense to set the default value to {{dfs.replication}} ?
Regarding of #3, what is the typical relationship between {{rack}} and
{{upgrade domains}}. As we have priorities to choice block placement policy to
satisfy these two requirements. Do you know what kind of implication it will
have to the current data availability model? I'd love to hear more about your
thoughts on this.
Again, thanks a lot for this great work!
> Provide BlockPlacementPolicy that supports upgrade domain
> ---------------------------------------------------------
>
> Key: HDFS-9006
> URL: https://issues.apache.org/jira/browse/HDFS-9006
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Ming Ma
> Assignee: Ming Ma
> Attachments: HDFS-9006.patch
>
>
> As part of the upgrade domain feature, we need to provide the actual upgrade
> domain block placement.
> Namenode provides a mechanism to specify custom block placement policy. We
> can use that to implement BlockPlacementPolicy with upgrade domain support.
> {noformat}
> <property>
> <name>dfs.block.replicator.classname</name>
> <value>org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyWithUpgradeDomain</value>
> </property>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
