This is an automated email from the ASF dual-hosted git repository.
leesf pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/asf-site by this push:
new d9fe2d7 [HUDI-1124] Document the usage of Tencent COSN (#1891)
d9fe2d7 is described below
commit d9fe2d7a092d55a4c751fb10358221959253718b
Author: DeyinZhong <[email protected]>
AuthorDate: Mon Aug 3 20:19:01 2020 +0800
[HUDI-1124] Document the usage of Tencent COSN (#1891)
Co-authored-by: deyzhong <[email protected]>
---
docs/_docs/0_7_cos_filesystem.cn.md | 74 +++++++++++++++++++++++++++++++++++++
docs/_docs/0_7_cos_filesystem.md | 72 ++++++++++++++++++++++++++++++++++++
docs/_docs/2_7_cloud.cn.md | 2 +
docs/_docs/2_7_cloud.md | 2 +
4 files changed, 150 insertions(+)
diff --git a/docs/_docs/0_7_cos_filesystem.cn.md
b/docs/_docs/0_7_cos_filesystem.cn.md
new file mode 100644
index 0000000..d8244c8
--- /dev/null
+++ b/docs/_docs/0_7_cos_filesystem.cn.md
@@ -0,0 +1,74 @@
+---
+title: COS Filesystem
+keywords: hudi, hive, tencent, cos, spark, presto
+permalink: /cn/docs/cos_hoodie.html
+summary: In this page, we go over how to configure Hudi with COS filesystem.
+last_modified_at: 2020-04-21T12:50:50-10:00
+language: cn
+---
+这个页面描述了如何让你的Hudi spark任务使用Tencent Cloud COS存储。
+
+## Tencent Cloud COS 部署
+
+为了让Hudi使用COS，需要增加两部分的配置:
+
+- 为Hidi增加Tencent Cloud COS的相关配置
+- 增加Jar包的MVN依赖
+
+### Tencent Cloud COS 相关的配置
+
+新增下面的配置到你的Hudi能访问的core-site.xml文件。使用你的COS bucket
name替换掉`fs.defaultFS`，使用COS的key和secret分别替换`fs.cosn.userinfo.secretKey`和`fs.cosn.userinfo.secretId`。主要Hudi就能读写相应的bucket。
+
+
+```xml
+ <property>
+ <name>fs.defaultFS</name>
+ <value>cosn://bucketname</value>
+ <description>COS bucket name</description>
+ </property>
+
+ <property>
+ <name>fs.cosn.userinfo.secretId</name>
+ <value>cos-secretId</value>
+ <description>Tencent Cloud Secret Id</description>
+ </property>
+
+ <property>
+ <name>fs.cosn.userinfo.secretKey</name>
+ <value>cos-secretkey</value>
+ <description>Tencent Cloud Secret Key</description>
+ </property>
+
+ <property>
+ <name>fs.cosn.bucket.region</name>
+ <value>ap-region</value>
+ <description>The region where the bucket is located.</description>
+ </property>
+
+ <property>
+ <name>fs.cosn.bucket.endpoint_suffix</name>
+ <value>cos.endpoint.suffix</value>
+ <description>
+ COS endpoint to connect to.
+ For public cloud users, it is recommended not to set this option,
and only the correct area field is required.
+ </description>
+ </property>
+
+ <property>
+ <name>fs.cosn.impl</name>
+ <value>org.apache.hadoop.fs.CosFileSystem</value>
+ <description>The implementation class of the CosN
Filesystem.</description>
+ </property>
+
+ <property>
+ <name>fs.AbstractFileSystem.cosn.impl</name>
+ <value>org.apache.hadoop.fs.CosN</value>
+ <description>The implementation class of the CosN
AbstractFileSystem.</description>
+ </property>
+
+```
+
+### Tencent Cloud COS Libs
+添加COS依赖jar包到classpath
+
+- org.apache.hadoop:hadoop-cos:2.8.5
diff --git a/docs/_docs/0_7_cos_filesystem.md b/docs/_docs/0_7_cos_filesystem.md
new file mode 100644
index 0000000..7acc0a1
--- /dev/null
+++ b/docs/_docs/0_7_cos_filesystem.md
@@ -0,0 +1,72 @@
+---
+title: COS Filesystem
+keywords: hudi, hive, tencent, cos, spark, presto
+permalink: /docs/cos_hoodie.html
+summary: In this page, we go over how to configure Hudi with COS filesystem.
+last_modified_at: 2020-04-21T11:38:24-10:00
+---
+In this page, we explain how to get your Hudi spark job to store into Tencent
Cloud COS.
+
+## Tencent Cloud COS configs
+
+There are two configurations required for Hudi-COS compatibility:
+
+- Adding Tencent Cloud COS Credentials for Hudi
+- Adding required Jars to classpath
+
+### Tencent Cloud COS Credentials
+
+Add the required configs in your core-site.xml from where Hudi can fetch them.
Replace the `fs.defaultFS` with your COS bucket name, replace
`fs.cosn.userinfo.secretId` with your COS secret Id, replace
`fs.cosn.userinfo.secretKey` with your COS key. Hudi should be able to
read/write from the bucket.
+
+```xml
+ <property>
+ <name>fs.defaultFS</name>
+ <value>cosn://bucketname</value>
+ <description>COS bucket name</description>
+ </property>
+
+ <property>
+ <name>fs.cosn.userinfo.secretId</name>
+ <value>cos-secretId</value>
+ <description>Tencent Cloud Secret Id</description>
+ </property>
+
+ <property>
+ <name>fs.cosn.userinfo.secretKey</name>
+ <value>cos-secretkey</value>
+ <description>Tencent Cloud Secret Key</description>
+ </property>
+
+ <property>
+ <name>fs.cosn.bucket.region</name>
+ <value>ap-region</value>
+ <description>The region where the bucket is located.</description>
+ </property>
+
+ <property>
+ <name>fs.cosn.bucket.endpoint_suffix</name>
+ <value>cos.endpoint.suffix</value>
+ <description>
+ COS endpoint to connect to.
+ For public cloud users, it is recommended not to set this option,
and only the correct area field is required.
+ </description>
+ </property>
+
+ <property>
+ <name>fs.cosn.impl</name>
+ <value>org.apache.hadoop.fs.CosFileSystem</value>
+ <description>The implementation class of the CosN
Filesystem.</description>
+ </property>
+
+ <property>
+ <name>fs.AbstractFileSystem.cosn.impl</name>
+ <value>org.apache.hadoop.fs.CosN</value>
+ <description>The implementation class of the CosN
AbstractFileSystem.</description>
+ </property>
+
+```
+
+### Tencent Cloud COS Libs
+COS hadoop libraries to add to our classpath
+
+- org.apache.hadoop:hadoop-cos:2.8.5
diff --git a/docs/_docs/2_7_cloud.cn.md b/docs/_docs/2_7_cloud.cn.md
index 209db78..fbd18b6 100644
--- a/docs/_docs/2_7_cloud.cn.md
+++ b/docs/_docs/2_7_cloud.cn.md
@@ -20,3 +20,5 @@ language: cn
阿里云和Hudi协同工作所需的配置。
* [Microsoft Azure](/cn/docs/azure_hoodie.html) <br/>
Azure和Hudi协同工作所需的配置。
+ * [Tencent Cloud Object Storage](/cn/docs/cos_hoodie.html) <br/>
+ COS和Hudi协同工作所需的配置。
diff --git a/docs/_docs/2_7_cloud.md b/docs/_docs/2_7_cloud.md
index 56b75b7..226180c 100644
--- a/docs/_docs/2_7_cloud.md
+++ b/docs/_docs/2_7_cloud.md
@@ -20,3 +20,5 @@ to cloud stores.
Configurations required for OSS and Hudi co-operability.
* [Microsoft Azure](/docs/azure_hoodie.html) <br/>
Configurations required for Azure and Hudi co-operability.
+* [Tencent Cloud Object Storage](/docs/cos_hoodie.html) <br/>
+ Configurations required for COS and Hudi co-operability.
