[
https://issues.apache.org/jira/browse/HDFS-7891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14497428#comment-14497428
]
Walter Su commented on HDFS-7891:
---------------------------------
bq. If we simply let getMaxNodesPerRack() return the ceiling of
totalNumOfReplicas/numOfRacks, like (totalNumOfReplicas - 1) / numOfRacks + 1,
then do not override chooseTargetInOrder(), can we get the right distribution
we want?
If totalNumOfReplicas < numOfRacks, it's much simpler, maxNodesPerRack==1. It's
usually the case in production cluster. The tricky thing is, when
totalNumOfReplicas > numOfRacks.
*Notation*
R = total racks in cluster
X = total expected replicas
Q=Math.floor(X/R)
T=X%R
if X<R, X racks have 1 replica.
if X>R && T==0, R racks have Q replicas.
if X>R && T!=0, X-T racks have Q replicas. T racks have Q+1 replicas.
The tricky thing is,
1. if X>R && T!=0, X-T racks have Q replicas. T racks have Q+1 replicas. *We
want local rack in the T racks group*. So we have to override
chooseTargetInOrder() anyway.
2. if X>R && T!=0, and if _"we simply let getMaxNodesPerRack() return the
ceiling of totalNumOfReplicas/numOfRacks"_, there is a chance that more than T
racks have Q+1 replicas while some racks have none.
> A block placement policy with best rack failure tolerance
> ---------------------------------------------------------
>
> Key: HDFS-7891
> URL: https://issues.apache.org/jira/browse/HDFS-7891
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: namenode
> Reporter: Walter Su
> Assignee: Walter Su
> Priority: Minor
> Attachments: HDFS-7891.005.dup.patch, HDFS-7891.005.patch
>
>
> a block placement policy tries its best to place replicas to most racks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
