This is an automated email from the ASF dual-hosted git repository.
xuanwo pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-opendal.git
The following commit(s) were added to refs/heads/main by this push:
new ed9028174 docs: add hdfs namenode High Availability related
troubleshoot (#2601)
ed9028174 is described below
commit ed90281742c16ec42fa7cb785f1527d359fa0fcd
Author: congyi wang <[email protected]>
AuthorDate: Thu Jul 6 16:12:41 2023 +0800
docs: add hdfs namenode High Availability related troubleshoot (#2601)
add troubleshoot doc for hdfs ha
---
core/src/services/hdfs/docs.md | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/core/src/services/hdfs/docs.md b/core/src/services/hdfs/docs.md
index 873c29715..7c30fd5c4 100644
--- a/core/src/services/hdfs/docs.md
+++ b/core/src/services/hdfs/docs.md
@@ -67,6 +67,21 @@ To set `CLASSPATH`:
export CLASSPATH=$(find $HADOOP_HOME -iname "*.jar" | xargs echo | tr ' '
':'):${CLASSPATH}
```
+- If HDFS has High Availability (HA) enabled with multiple available
NameNodes, some configuration is required:
+1. Obtain the entire HDFS config folder (usually located at
HADOOP_HOME/etc/hadoop).
+2. Set the environment variable HADOOP_CONF_DIR to the path of this folder.
+```shell
+export HADOOP_CONF_DIR=<path of the config folder>
+```
+3. Append the HADOOP_CONF_DIR to the `CLASSPATH`
+```shell
+export CLASSPATH=$CLASSPATH:$HADOOP_CONF_DIR
+```
+4. Use the `cluster_name` specified in the `core-site.xml` file (located in
the HADOOP_CONF_DIR folder) to replace namenode:port.
+
+```rust
+builder.name_node("hdfs://cluster_name");
+```
## Example
### Via Builder
