[
https://issues.apache.org/jira/browse/HDFS-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317424#comment-14317424
]
Walter Su commented on HDFS-7613:
---------------------------------
The facebook version has two policies:
1st, BlockPlacementPolicyRaid.java , Parity block and source block both have 3
replicas. The 3 replicas all are chosen randomly. After the file is finished
writing, Client API call setReplication() function to reduce the number of
replicas to 2. The policy has a strategy on choosing which replica to delete.
The strategy is to delete replica from the host which has most replica. The
policy is fit for XOR.
2nd, FaltTolerantBlockPlacementPolicy.java, Every new added block is placed at
different racks and different hosts. It's fit for RS.
I think 1st policy place replicas on relative exclusive hosts. It's fast, but
has no guarantee that the one host has only one replica. Since the final
replica factor is 2( for XOR), I think it's ok.
2nd policy is slower, But it places replicas on strict exclusive hosts.
If we use RS, and only has one replica for each block, We should stick to 2nd
policy.
hi, Zhe Zhang, do you know will we support XOR? and how many replicas each
block has?
> Block placement policy for erasure coding groups
> ------------------------------------------------
>
> Key: HDFS-7613
> URL: https://issues.apache.org/jira/browse/HDFS-7613
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Zhe Zhang
> Assignee: Walter Su
>
> Blocks in an erasure coding group should be placed in different failure
> domains -- different DataNodes at the minimum, and different racks ideally.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
