Doris Gu created HDFS-10604:
-------------------------------
Summary: What about this?Group DNs and add DN groups--named region
to HDFS model , use this region to instead of single DN when saving files.
Key: HDFS-10604
URL: https://issues.apache.org/jira/browse/HDFS-10604
Project: Hadoop HDFS
Issue Type: Wish
Reporter: Doris Gu
The biggest difference this feature will bring is *strong* making blocks belong
to the same file to save in the same region(DN group).*strong*
So the process will be:
1.Config DN groups, for example
bq.Region1：dn1,dn2,dn3
bq.Region2：dn4,dn5,dn6
bq.Region3：dn7,dn8,dn9,dn10
2.Client uploads a file, first analyze whether this file has any existed blocks:
bq.i)Yes:assign new blocks to the DN group where the existed blocks belong to.
bq.ii)No:assign new blocks to a DN group which is chosen by some certain policy
to avoid imbalance.
3.Other related processes,including append,balancer etc. also need to modify as
well.
The benefit we wish is when some DNs are down at the same time, the number of
affected files(miss all replicas) is small.
But we are wondering if this is worth doing or not, or if there are problems we
haven't noticed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
