[
https://issues.apache.org/jira/browse/HDFS-17223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17778231#comment-17778231
]
ASF GitHub Bot commented on HDFS-17223:
---------------------------------------
xinglin commented on code in PR #6183:
URL: https://github.com/apache/hadoop/pull/6183#discussion_r1367803434
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/client/QuorumJournalManager.java:
##########
@@ -146,6 +149,16 @@ public QuorumJournalManager(Configuration conf,
this.nameServiceId = nameServiceId;
this.loggers = new AsyncLoggerSet(createLoggers(loggerFactory));
+ // Check whether the number of jn maintenance lists is valid
+ String[] skipNodesHostPort = conf.getTrimmedStrings(
+ DFS_JOURNALNODE_MAINTENANCE_NODES_KEY,
DFS_JOURNALNODE_MAINTENANCE_NODES_DEFAULT);
+
+ int quorumThreshold = (this.loggers.size() / 2) + 1;
+ Preconditions.checkArgument(
+ (this.loggers.size() - skipNodesHostPort.length) >= quorumThreshold,
Review Comment:
Instead of creating a QJPFakedTranslatorPB.java, can we just remove these
maintenance JNs from `this.loggers `array? We can also make` quorumThreshold` a
class member to remember the minimum number of available JNs and the minimum
responses we need to commit a write to HDFS.
> Add journalnode maintenance node list
> -------------------------------------
>
> Key: HDFS-17223
> URL: https://issues.apache.org/jira/browse/HDFS-17223
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: qjm
> Affects Versions: 3.3.6
> Reporter: kuper
> Priority: Major
> Labels: pull-request-available
>
> * In the case of configuring 3 journal nodes in HDFS, if only 2 journal nodes
> are available and 1 journal node fails to start due to machine issues, it
> will result in a long initialization time for the namenode (around 30-40
> minutes, depending on the IPC timeout and retry policy configuration).
> * The failed journal node cannot recover immediately, but HDFS can still
> function in this situation. In our production environment, we encountered
> this issue and had to reduce the IPC timeout and adjust the retry policy to
> accelerate the namenode initialization and provide services.
> * I'm wondering if it would be possible to have a journal node maintenance
> list to speed up the namenode initialization knowing that one journal node
> cannot provide services in advance?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
