HDFS-14154. [SBN read] Document dfs.ha.tail-edits.period in user guide. Contributed by Chao Sun.
Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/c546b12e Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/c546b12e Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/c546b12e Branch: refs/heads/trunk Commit: c546b12e457f0a33d8d3023f220480edf7dd3810 Parents: c9d7737 Author: Chao Sun <sunc...@apache.org> Authored: Wed Dec 19 22:20:31 2018 -0800 Committer: Konstantin V Shvachko <s...@apache.org> Committed: Mon Dec 24 09:34:01 2018 -0800 ---------------------------------------------------------------------- .../site/markdown/HDFSHighAvailabilityWithQJM.md | 7 ++++++- .../src/site/markdown/ObserverNameNode.md | 18 ++++++++++++++++-- 2 files changed, 22 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/c546b12e/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithQJM.md ---------------------------------------------------------------------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithQJM.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithQJM.md index 76a9837..4f3df27 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithQJM.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithQJM.md @@ -27,7 +27,12 @@ This document assumes that the reader has a general understanding of general com Note: Using the Quorum Journal Manager or Conventional Shared Storage --------------------------------------------------------------------- -This guide discusses how to configure and use HDFS HA using the Quorum Journal Manager (QJM) to share edit logs between the Active and Standby NameNodes. For information on how to configure HDFS HA using NFS for shared storage instead of the QJM, please see [this alternative guide.](./HDFSHighAvailabilityWithNFS.html) +This guide discusses how to configure and use HDFS HA using the Quorum +Journal Manager (QJM) to share edit logs between the Active and Standby +NameNodes. For information on how to configure HDFS HA using NFS for +shared storage instead of the QJM, please see [this alternative +guide.](./HDFSHighAvailabilityWithNFS.html). For information on how to +configure HDFS HA with Observer NameNode, please see [this guide](./ObserverNameNode.html) Background ---------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/c546b12e/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ObserverNameNode.md ---------------------------------------------------------------------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ObserverNameNode.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ObserverNameNode.md index 2548315..d93256c 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ObserverNameNode.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ObserverNameNode.md @@ -23,13 +23,13 @@ Purpose This guide provides an overview of the HDFS Observer NameNode feature and how to configure/install it in a typical HA-enabled cluster. For a detailed technical design overview, please check the doc attached to -HDFS-12943. +[HDFS-12943](https://issues.apache.org/jira/browse/HDFS-12943). Background ----------- In a HA-enabled HDFS cluster (for more information, check -[HDFSHighAvailabilityWithQJM](./HDFSHighAvailabilityWithQJM.md)), there +[HDFSHighAvailabilityWithQJM](./HDFSHighAvailabilityWithQJM.html)), there is a single Active NameNode and one or more Standby NameNode(s). The Active NameNode is responsible for serving all client requests, while Standby NameNode just keep the up-to-date information regarding the @@ -104,6 +104,20 @@ few configurations to your **hdfs-site.xml**: <value>true</value> </property> +* **dfs.ha.tail-edits.period** - how often Standby/Observer NameNodes + should fetch edits from JournalNodes. + + This determines the staleness of Observer NameNode w.r.t the Active. + If too large, RPC time will increase as client requests will wait + longer in the RPC queue before Observer tails edit logs and catches + up the latest state of Active. The default value is 1min. It is + **highly recommend** to configure this to a much lower value. + + <property> + <name>dfs.ha.tail-edits.period</name> + <value>0ms</value> + </property> + * **dfs.journalnode.edit-cache-size.bytes** - the in-memory cache size, in bytes, on the JournalNodes. --------------------------------------------------------------------- To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org