[
https://issues.apache.org/jira/browse/HDFS-17534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17854608#comment-17854608
]
ASF GitHub Bot commented on HDFS-17534:
---------------------------------------
yuanboliu commented on code in PR #6861:
URL: https://github.com/apache/hadoop/pull/6861#discussion_r1637443709
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/order/DestinationOrder.java:
##########
@@ -29,11 +29,14 @@ public enum DestinationOrder {
LOCAL, // Local first
RANDOM, // Random order
HASH_ALL, // Follow consistent hashing
- SPACE; // Available space based order
+ SPACE, // Available space based order
+ LEADER_FOLLOWER; // try leader sub-cluster first, if failed, try followers
Review Comment:
fixed
##########
hadoop-hdfs-project/hadoop-hdfs-rbf/src/site/markdown/HDFSRouterFederation.md:
##########
@@ -297,6 +298,9 @@ RANDOM can be used for reading and writing data from/into
different subclusters.
The common use for this approach is to have the same data in multiple
subclusters and balance the reads across subclusters.
For example, if thousands of containers need to read the same data (e.g., a
library), one can use RANDOM to read the data from any of the subclusters.
+LEADER_FOLLOWER can be used in cross-cluster disaster tolerance, it's not for
sharing overloads among sub-clusters. When using this mode like `-add /data
ns2,ns1 /data -order LEADER_FOLLOWER`,
+`ns2` is considered as a active sub-cluster and `ns1` is considered as a
follower sub-cluster. The order of namespaces is always
`leader,follower,follower...`
Review Comment:
fixed
> RBF: Support leader follower mode for multiple subclusters
> ----------------------------------------------------------
>
> Key: HDFS-17534
> URL: https://issues.apache.org/jira/browse/HDFS-17534
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: rbf
> Reporter: Yuanbo Liu
> Assignee: Yuanbo Liu
> Priority: Major
> Labels: pull-request-available
>
> Currently there are five modes in multiple subclusters like
> HASH, LOCAL, RANDOM, HASH_ALL,SPACE;
> Proposal a new mode called leader/follower mode. routers try to write to
> leader subcluster as many as possible. When routers read data, put leader
> subcluster into first rank.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
