This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new f4e808afe6c9 [docs](multi-catalog) Add hdfs transfer protection error
FAQ. (#537)
f4e808afe6c9 is described below
commit f4e808afe6c9b482273cfe6250264730a0ea7517
Author: Qi Chen <[email protected]>
AuthorDate: Thu Apr 11 10:05:08 2024 +0800
[docs](multi-catalog) Add hdfs transfer protection error FAQ. (#537)
---
docs/lakehouse/faq.md | 15 +++++++++++++++
.../current/lakehouse/faq.md | 16 ++++++++++++++++
.../version-1.2/lakehouse/multi-catalog/faq.md | 16 ++++++++++++++++
.../version-2.1/lakehouse/faq.md | 16 ++++++++++++++++
.../version-1.2/lakehouse/multi-catalog/faq.md | 15 +++++++++++++++
versioned_docs/version-2.1/lakehouse/faq.md | 15 +++++++++++++++
6 files changed, 93 insertions(+)
diff --git a/docs/lakehouse/faq.md b/docs/lakehouse/faq.md
index 3b20fa878c20..855f0003e9a4 100644
--- a/docs/lakehouse/faq.md
+++ b/docs/lakehouse/faq.md
@@ -293,6 +293,21 @@ ln -s
/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt /etc/ssl/certs/ca-
You need to check whether `HADOOP_CONF_DIR` is configured correctly, or
unset this environment variable.
+4. `BlockMissingExcetpion: Could not obtain block: BP-XXXXXXXXX No live nodes
contain current block`
+
+ Possible solutions include:
+ - Use `hdfs fsck file -files -blocks -locations` to check if the file is
healthy.
+ - Use telnet to check connectivity with the DataNode.
+ - Check the DataNode logs.
+
+ If encountering the following error:
+ `org.apache.hadoop.hdfs.server.datanode.DataNode: Failed to read expected
SASL data transfer protection handshake from client at /XXX.XXX.XXX.XXX:XXXXX.
Perhaps the client is running an older version of Hadoop which does not support
SASL data transfer protection`.
+ It indicates that HDFS is configured for encrypted transmission while the
client is not, causing the error.
+
+ You can use any of the following solutions:
+ - Copying hdfs-site.xml and core-site.xml to the be/conf and fe/conf
directories. (Recommended)
+ - In hdfs-site.xml, find the corresponding configuration
`dfs.data.transfer.protection`, and set this parameter in the catalog.
+
## DLF Catalog
1. When using DLF Catalog, BE reads `Invalid address` when fetching JindoFS
data and needs to add the domain name to IP mapping that appears in the log in
`/ets/hosts`.
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/faq.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/faq.md
index 7c608eb61ae2..7e857b3cf90f 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/faq.md
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/faq.md
@@ -289,6 +289,22 @@ ln -s
/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt /etc/ssl/certs/ca-
需检查 `HADOOP_CONF_DIR` 是否配置正确，或将这个环境变量删除。
+4. `BlockMissingExcetpion: Could not obtain block: BP-XXXXXXXXX No live nodes
contain current block`
+
+ 可能的处理方式有：
+ - 通过 `hdfs fsck file -files -blocks -locations` 来查看具体该文件是否健康。
+ - 通过 `telnet` 来检查与 datanode 的连通性。
+ - 查看 datanode 日志。
+
+ 如果出现以下错误：
+
+ `org.apache.hadoop.hdfs.server.datanode.DataNode: Failed to read expected
SASL data transfer protection handshake from client at /XXX.XXX.XXX.XXX:XXXXX.
Perhaps the client is running an older version of Hadoop which does not support
SASL data transfer protection`
+ 则为当前 hdfs 开启了加密传输方式，而客户端未开启导致的错误。
+
+ 使用下面的任意一种解决方案即可:
+ - 拷贝 hdfs-site.xml 以及 core-site.xml 到 be/conf 和 fe/conf 目录。(推荐)
+ - 在 hdfs-site.xml 找到相应的配置 `dfs.data.transfer.protection`，并且在 catalog
里面设置该参数。
+
## DLF Catalog
1. 使用DLF Catalog时，BE读在取JindoFS数据出现`Invalid
address`，需要在`/ets/hosts`中添加日志中出现的域名到IP的映射。
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/lakehouse/multi-catalog/faq.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/lakehouse/multi-catalog/faq.md
index e45f37f05659..aabe59a50fee 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/lakehouse/multi-catalog/faq.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-1.2/lakehouse/multi-catalog/faq.md
@@ -185,3 +185,19 @@ under the License.
'hive.version' = '1.x.x'
);
```
+
+19. `BlockMissingExcetpion: Could not obtain block: BP-XXXXXXXXX No live nodes
contain current block`
+
+ 可能的处理方式有：
+ - 通过 `hdfs fsck file -files -blocks -locations` 来查看具体该文件是否健康。
+ - 通过 `telnet` 来检查与 datanode 的连通性。
+ - 查看 datanode 日志。
+
+ 如果出现以下错误：
+
+ `org.apache.hadoop.hdfs.server.datanode.DataNode: Failed to read expected
SASL data transfer protection handshake from client at /XXX.XXX.XXX.XXX:XXXXX.
Perhaps the client is running an older version of Hadoop which does not support
SASL data transfer protection`
+ 则为当前 hdfs 开启了加密传输方式，而客户端未开启导致的错误。
+
+ 使用下面的任意一种解决方案即可:
+ - 拷贝 hdfs-site.xml 以及 core-site.xml 到 be/conf 和 fe/conf 目录。(推荐)
+ - 在 hdfs-site.xml 找到相应的配置 `dfs.data.transfer.protection`，并且在 catalog
里面设置该参数。
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/faq.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/faq.md
index 7c608eb61ae2..7e857b3cf90f 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/faq.md
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/faq.md
@@ -289,6 +289,22 @@ ln -s
/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt /etc/ssl/certs/ca-
需检查 `HADOOP_CONF_DIR` 是否配置正确，或将这个环境变量删除。
+4. `BlockMissingExcetpion: Could not obtain block: BP-XXXXXXXXX No live nodes
contain current block`
+
+ 可能的处理方式有：
+ - 通过 `hdfs fsck file -files -blocks -locations` 来查看具体该文件是否健康。
+ - 通过 `telnet` 来检查与 datanode 的连通性。
+ - 查看 datanode 日志。
+
+ 如果出现以下错误：
+
+ `org.apache.hadoop.hdfs.server.datanode.DataNode: Failed to read expected
SASL data transfer protection handshake from client at /XXX.XXX.XXX.XXX:XXXXX.
Perhaps the client is running an older version of Hadoop which does not support
SASL data transfer protection`
+ 则为当前 hdfs 开启了加密传输方式，而客户端未开启导致的错误。
+
+ 使用下面的任意一种解决方案即可:
+ - 拷贝 hdfs-site.xml 以及 core-site.xml 到 be/conf 和 fe/conf 目录。(推荐)
+ - 在 hdfs-site.xml 找到相应的配置 `dfs.data.transfer.protection`，并且在 catalog
里面设置该参数。
+
## DLF Catalog
1. 使用DLF Catalog时，BE读在取JindoFS数据出现`Invalid
address`，需要在`/ets/hosts`中添加日志中出现的域名到IP的映射。
diff --git a/versioned_docs/version-1.2/lakehouse/multi-catalog/faq.md
b/versioned_docs/version-1.2/lakehouse/multi-catalog/faq.md
index e73a0a5bf407..bbaff1ca091b 100644
--- a/versioned_docs/version-1.2/lakehouse/multi-catalog/faq.md
+++ b/versioned_docs/version-1.2/lakehouse/multi-catalog/faq.md
@@ -189,3 +189,18 @@ under the License.
'hive.version' = '2.x.x'
);
```
+
+19. `BlockMissingExcetpion: Could not obtain block: BP-XXXXXXXXX No live nodes
contain current block`
+
+ Possible solutions include:
+ - Use `hdfs fsck file -files -blocks -locations` to check if the file is
healthy.
+ - Use telnet to check connectivity with the DataNode.
+ - Check the DataNode logs.
+
+ If encountering the following error:
+ `org.apache.hadoop.hdfs.server.datanode.DataNode: Failed to read expected
SASL data transfer protection handshake from client at /XXX.XXX.XXX.XXX:XXXXX.
Perhaps the client is running an older version of Hadoop which does not support
SASL data transfer protection`.
+ It indicates that HDFS is configured for encrypted transmission while the
client is not, causing the error.
+
+ You can use any of the following solutions:
+ - Copying hdfs-site.xml and core-site.xml to the be/conf and fe/conf
directories. (Recommended)
+ - In hdfs-site.xml, find the corresponding configuration
`dfs.data.transfer.protection`, and set this parameter in the catalog.
diff --git a/versioned_docs/version-2.1/lakehouse/faq.md
b/versioned_docs/version-2.1/lakehouse/faq.md
index 3b20fa878c20..855f0003e9a4 100644
--- a/versioned_docs/version-2.1/lakehouse/faq.md
+++ b/versioned_docs/version-2.1/lakehouse/faq.md
@@ -293,6 +293,21 @@ ln -s
/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt /etc/ssl/certs/ca-
You need to check whether `HADOOP_CONF_DIR` is configured correctly, or
unset this environment variable.
+4. `BlockMissingExcetpion: Could not obtain block: BP-XXXXXXXXX No live nodes
contain current block`
+
+ Possible solutions include:
+ - Use `hdfs fsck file -files -blocks -locations` to check if the file is
healthy.
+ - Use telnet to check connectivity with the DataNode.
+ - Check the DataNode logs.
+
+ If encountering the following error:
+ `org.apache.hadoop.hdfs.server.datanode.DataNode: Failed to read expected
SASL data transfer protection handshake from client at /XXX.XXX.XXX.XXX:XXXXX.
Perhaps the client is running an older version of Hadoop which does not support
SASL data transfer protection`.
+ It indicates that HDFS is configured for encrypted transmission while the
client is not, causing the error.
+
+ You can use any of the following solutions:
+ - Copying hdfs-site.xml and core-site.xml to the be/conf and fe/conf
directories. (Recommended)
+ - In hdfs-site.xml, find the corresponding configuration
`dfs.data.transfer.protection`, and set this parameter in the catalog.
+
## DLF Catalog
1. When using DLF Catalog, BE reads `Invalid address` when fetching JindoFS
data and needs to add the domain name to IP mapping that appears in the log in
`/ets/hosts`.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
