This is an automated email from the ASF dual-hosted git repository.
gavinchou pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 93d0286ed1a [opt] Add more storage vault demo about different product
(#1788)
93d0286ed1a is described below
commit 93d0286ed1af4ab534776d373e55f0579b4f18ee
Author: Lei Zhang <[email protected]>
AuthorDate: Fri Jan 17 21:54:53 2025 +0800
[opt] Add more storage vault demo about different product (#1788)
---
.../managing-storage-vault.md | 29 +--
.../Create/CREATE-STORAGE-VAULT.md | 197 ++++++++++--------
.../managing-storage-vault.md | 53 ++---
.../Create/CREATE-STORAGE-VAULT.md | 202 ++++++++++--------
.../managing-storage-vault.md | 51 ++---
.../storage-management/CREATE-STORAGE-VAULT.md | 228 ++++++++++++---------
.../managing-storage-vault.md | 29 +--
.../storage-management/CREATE-STORAGE-VAULT.md | 220 +++++++++++---------
8 files changed, 581 insertions(+), 428 deletions(-)
diff --git a/docs/compute-storage-decoupled/managing-storage-vault.md
b/docs/compute-storage-decoupled/managing-storage-vault.md
index d3496192509..688f1617b60 100644
--- a/docs/compute-storage-decoupled/managing-storage-vault.md
+++ b/docs/compute-storage-decoupled/managing-storage-vault.md
@@ -44,7 +44,7 @@ PROPERTIES
To create an HDFS-based decoupled storage-compute Doris cluster, ensure that
all nodes (including FE/BE nodes, Meta Service) have privilege to access the
specified HDFS, including completing Kerberos authorization configuration and
connectivity checks in advance (which can be tested using Hadoop Client on each
corresponding node).
```sql
-CREATE STORAGE VAULT IF NOT EXISTS ssb_hdfs_vault
+CREATE STORAGE VAULT IF NOT EXISTS hdfs_vault_demo
PROPERTIES (
"type"="hdfs", -- required
"fs.defaultFS"="hdfs://127.0.0.1:8020", -- required
@@ -59,17 +59,18 @@ CREATE STORAGE VAULT IF NOT EXISTS ssb_hdfs_vault
### Create an S3 Storage Vault
```sql
-CREATE STORAGE VAULT IF NOT EXISTS ssb_s3_vault
- PROPERTIES (
- "type"="S3", -- required
- "s3.endpoint" = "oss-cn-beijing.aliyuncs.com", -- required
- "s3.region" = "bj", -- required
- "s3.bucket" = "bucket", -- required
- "s3.root.path" = "big/data/prefix", -- required
- "s3.access_key" = "ak", -- required
- "s3.secret_key" = "sk", -- required
- "provider" = "OSS" -- required
- );
+CREATE STORAGE VAULT IF NOT EXISTS s3_vault_demo
+PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "oss-cn-beijing.aliyuncs.com", -- required
+ "s3.region" = "cn-beijing", -- required
+ "s3.bucket" = "bucket", -- required
+ "s3.root.path" = "big/data/prefix", -- required
+ "s3.access_key" = "ak", -- required
+ "s3.secret_key" = "sk", -- required
+ "provider" = "OSS", -- required
+ "use_path_style" = "false" -- optional
+);
```
More parameter explanations and examples can be found in
[CREATE-STORAGE-VAULT](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-STORAGE-VAULT.md).
@@ -112,7 +113,7 @@ UNIQUE KEY (s_suppkey)
DISTRIBUTED BY HASH(s_suppkey) BUCKETS 1
PROPERTIES (
"replication_num" = "1",
-"storage_vault_name" = "ssb_hdfs_vault"
+"storage_vault_name" = "hdfs_demo_vault"
);
```
@@ -172,7 +173,7 @@ GRANT
Only Admin users have the authority to execute the `GRANT` statement, which is
used to grant specified Storage Vault privileges to User/Role. Users/Roles with
`USAGE_PRIV` privilege for a certain Storage Vault can perform the following
operations:
-- View the information of that Storage Vault through `SHOW STORAGE VAULT`;
+- View the information of that Storage Vault through `SHOW STORAGE VAULTS`;
- Specify the use of that Storage Vault in `PROPERTIES` when creating tables.
**Example**
diff --git
a/docs/sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-STORAGE-VAULT.md
b/docs/sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-STORAGE-VAULT.md
index f1f5214c74b..e0da5f83fbd 100644
---
a/docs/sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-STORAGE-VAULT.md
+++
b/docs/sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-STORAGE-VAULT.md
@@ -48,7 +48,7 @@ CREATE STORAGE VAULT [IF NOT EXISTS] vault
| param | is required | desc
|
|:----------------|:------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `s3.endpoint` | required | The endpoint used for object storage.
<br/>**Notice**, please don't provide the endpoint with any `http://` or
`https://`. And for Azure Blob Storage, the endpoint should be like
`${ak}.blob.core.windows.net/`. |
+| `s3.endpoint` | required | The endpoint used for object storage.
<br/>**Notice**, please don't provide the endpoint with any `http://` or
`https://`. And for Azure Blob Storage, the endpoint should be
`blob.core.windows.net`. |
| `s3.region` | required | The region of your bucket.(Not required
when you'r using GCP or AZURE).
|
| `s3.root.path` | required | The path where the data would be stored.
|
| `s3.bucket` | required | The bucket of your object storage account.
(StorageAccount if you're using Azure).
|
@@ -72,106 +72,143 @@ CREATE STORAGE VAULT [IF NOT EXISTS] vault
1. create a HDFS storage vault.
```sql
- CREATE STORAGE VAULT IF NOT EXISTS hdfs_vault
- PROPERTIES (
- "type"="hdfs",
- "fs.defaultFS"="hdfs://127.0.0.1:8020"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS hdfs_vault_demo
+ PROPERTIES (
+ "type" = "hdfs", -- required
+ "fs.defaultFS" = "hdfs://127.0.0.1:8020", -- required
+ "path_prefix" = "big/data", -- optional
+ "hadoop.username" = "user" -- optional
+ "hadoop.security.authentication" = "kerberos" -- optional
+ "hadoop.kerberos.principal" = "hadoop/127.0.0.1@XXX" -- optional
+ "hadoop.kerberos.keytab" = "/etc/emr.keytab" -- optional
+ );
```
-2. create a S3 storage vault using azure.
+2. create a S3 storage vault using OSS.
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="ak.blob.core.windows.net/",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "AZURE"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS oss_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "oss-cn-beijing.aliyuncs.com", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your OSS access key
+ "s3.secret_key" = "xxxxxx", -- required,
Your OSS secret key
+ "s3.region" = "cn-beijing", -- required
+ "s3.root.path" = "oss_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your OSS bucket name
+ "provider" = "OSS", -- required
+ "use_path_style" = "false" -- optional, OSS
suggest setting `false`
+ );
```
-3. create a S3 storage vault using OSS.
+3. create a S3 storage vault using COS.
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="oss.aliyuncs.com",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "cn-hangzhou",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "OSS",
- "use_path_style" = "false"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS cos_demo_vault
+ PROPERTIES (
+ "type" = "S3",
+ "s3.endpoint" = "cos.ap-guangzhou.myqcloud.com", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your COS access key
+ "s3.secret_key" = "xxxxxx", -- required,
Your COS secret key
+ "s3.region" = "ap-guangzhou", -- required
+ "s3.root.path" = "cos_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your COS bucket name
+ "provider" = "COS", -- required
+ "use_path_style" = "false" -- optional, COS
suggest setting `false`
+ );
```
-4. create a S3 storage vault using COS.
+4. create a S3 storage vault using OBS.
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "ap-guangzhou",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "COS",
- "use_path_style" = "false"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS obs_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "obs.cn-north-4.myhuaweicloud.com", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your OBS access key
+ "s3.secret_key" = "xxxxxx", -- required,
Your OBS secret key
+ "s3.region" = "cn-north-4", -- required
+ "s3.root.path" = "obs_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your COS bucket name
+ "provider" = "OBS", -- required
+ "use_path_style" = "false" -- optional, OBS
suggest setting `false`
+ );
```
-5. create a S3 storage vault using OBS.
+5. create a S3 storage vault using BOS.
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="obs.cn-north-4.myhuaweicloud.com",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "cn-north-4",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "OBS",
- "use_path_style" = "false"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS obs_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "s3.bj.bcebos.com", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your BOS access key
+ "s3.secret_key" = "xxxxxx", -- required,
Your BOS secret key
+ "s3.region" = "bj", -- required
+ "s3.root.path" = "bos_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your BOS bucket name
+ "provider" = "BOS", -- required
+ "use_path_style" = "false" -- optional, BOS
suggest setting `false`
+ );
```
6. create a S3 storage vault using AWS.
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="s3.us-east-1.amazonaws.com",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "us-east-1",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "S3",
- "use_path_style" = "false"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS s3_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "s3.us-east-1.amazonaws.com", -- required
+ "s3.access_key" = "xxxxxx", -- required, Your
S3 access key
+ "s3.secret_key" = "xxxxxx", -- required, Your
OBS secret key
+ "s3.region" = "us-east-1", -- required
+ "s3.root.path" = "s3_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required, Your
s3 bucket name
+ "provider" = "S3", -- required
+ "use_path_style" = "false" -- optional, S3
suggest setting `false`
+ );
```
7. create a S3 storage vault using MinIO.
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="127.0.0.1:9000",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "us-east-1",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "S3",
- "use_path_style" = "true"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS minio_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "127.0.0.1:9000", -- required
+ "s3.access_key" = "xxxxxx", -- required, Your
minio access key
+ "s3.secret_key" = "xxxxxx", -- required, Your
minio secret key
+ "s3.region" = "us-east-1", -- required
+ "s3.root.path" = "minio_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required, Your
minio bucket name
+ "provider" = "S3", -- required
+ "use_path_style" = "true" -- required, minio
suggest setting `true`
+ );
```
+8. create a S3 storage vault using AZURE.
+ ```sql
+ CREATE STORAGE VAULT IF NOT EXISTS azure_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "blob.core.windows.net", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your Azure AccountName
+ "s3.secret_key" = "xxxxxx", -- required,
Your Azure AccountKey
+ "s3.region" = "us-east-1", -- required
+ "s3.root.path" = "azure_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your Azure StorageAccount
+ "provider" = "AZURE" -- required
+ );
+ ```
+
+9. create a S3 storage vault using GCP.
+ ```sql
+ CREATE STORAGE VAULT IF NOT EXISTS gcp_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "storage.googleapis.com", -- required
+ "s3.access_key" = "xxxxxx", -- required
+ "s3.secret_key" = "xxxxxx", -- required
+ "s3.region" = "us-east-1", -- required
+ "s3.root.path" = "gcp_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required
+ "provider" = "GCP" -- required
+ );
+ ```
+
### Keywords
CREATE, STORAGE VAULT
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/compute-storage-decoupled/managing-storage-vault.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/compute-storage-decoupled/managing-storage-vault.md
index 915066bfe95..1b6a1369d42 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/compute-storage-decoupled/managing-storage-vault.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/compute-storage-decoupled/managing-storage-vault.md
@@ -40,38 +40,39 @@ PROPERTIES
### 创建 HDFS Storage Vault
-创建基于 HDFS 的存算分离模式 Doris 集群，需要确保所有的节点（包括 FE / BE 节点、Meta Service) 均有权限访问所指定的
HDFS，包括提前完成机器的 Kerberos 授权配置和连通性检查（可在对应的每个节点上使用 Hadoop Client 进行测试）等。
+创建基于 HDFS 的存算分离模式 Doris 集群，需要确保所有的节点 (包括 FE / BE 节点、Meta Service) 均有权限访问所指定的
HDFS，包括提前完成机器的 Kerberos 授权配置和连通性检查（可在对应的每个节点上使用 Hadoop Client 进行测试）等。
```sql
-CREATE STORAGE VAULT IF NOT EXISTS ssb_hdfs_vault
- PROPERTIES (
- "type"="hdfs", -- required
- "fs.defaultFS"="hdfs://127.0.0.1:8020", -- required
- "path_prefix"="big/data", -- optional,
一般按照业务名称填写
- "hadoop.username"="user" -- optional
- "hadoop.security.authentication"="kerberos" -- optional
- "hadoop.kerberos.principal"="hadoop/127.0.0.1@XXX" -- optional
- "hadoop.kerberos.keytab"="/etc/emr.keytab" -- optional
- );
+CREATE STORAGE VAULT IF NOT EXISTS hdfs_vault_demo
+PROPERTIES (
+ "type" = "hdfs", -- required
+ "fs.defaultFS" = "hdfs://127.0.0.1:8020", -- required
+ "path_prefix" = "big/data", -- optional,
一般按照业务名称填写
+ "hadoop.username" = "user" -- optional
+ "hadoop.security.authentication" = "kerberos" -- optional
+ "hadoop.kerberos.principal" = "hadoop/127.0.0.1@XXX" -- optional
+ "hadoop.kerberos.keytab" = "/etc/emr.keytab" -- optional
+);
```
### 创建 S3 Storage Vault
```SQL
-CREATE STORAGE VAULT IF NOT EXISTS ssb_s3_vault
- PROPERTIES (
- "type"="S3", -- required
- "s3.endpoint" = "oss-cn-beijing.aliyuncs.com", -- required
- "s3.region" = "bj", -- required
- "s3.bucket" = "bucket", -- required
- "s3.root.path" = "big/data/prefix", -- required
- "s3.access_key" = "ak", -- required
- "s3.secret_key" = "sk", -- required
- "provider" = "OSS" -- required
- );
+CREATE STORAGE VAULT IF NOT EXISTS s3_vault_demo
+PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "oss-cn-beijing.aliyuncs.com", -- required
+ "s3.region" = "cn-beijing", -- required
+ "s3.bucket" = "bucket", -- required
+ "s3.root.path" = "big/data/prefix", -- required
+ "s3.access_key" = "ak", -- required
+ "s3.secret_key" = "sk", -- required
+ "provider" = "OSS", -- required
+ "use_path_style" = "false" -- optional
+);
```
-更多参数说明及示例可见
[CREATE-STORAGE-VAULT](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-STORAGE-VAULT.md)。
+更多云厂商示例和参数说明可见
[CREATE-STORAGE-VAULT](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-STORAGE-VAULT.md)。
## 查看 Storage Vault
@@ -110,8 +111,8 @@ CREATE TABLE IF NOT EXISTS supplier (
UNIQUE KEY (s_suppkey)
DISTRIBUTED BY HASH(s_suppkey) BUCKETS 1
PROPERTIES (
-"replication_num" = "1",
-"storage_vault_name" = "ssb_hdfs_vault"
+ "replication_num" = "1",
+ "storage_vault_name" = "hdfs_demo_vault"
);
```
@@ -171,7 +172,7 @@ GRANT
仅 Admin 用户有权限执行 `GRANT` 语句，该语句用于向 User / Role 授予指定 Storage Vault 的权限。拥有某个
Storage Vault 的 `USAGE_PRIV` 权限的 User / Role 可进行以下操作：
-- 通过 `SHOW STORAGE VAULT` 查看该 Storage Vault 的信息；
+- 通过 `SHOW STORAGE VAULTS` 查看该 Storage Vault 的信息；
- 建表时在 `PROPERTIES` 中指定使用该 Storage Vault 。
### 撤销
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-STORAGE-VAULT.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-STORAGE-VAULT.md
index da6eaaabc0e..5c432060aed 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-STORAGE-VAULT.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-STORAGE-VAULT.md
@@ -47,7 +47,7 @@ CREATE STORAGE VAULT [IF NOT EXISTS] vault
| 参数 | 是否必需 | 描述
|
|:----------------|:-----|:--------------------------------------------------------------------------------------------------------|
-| `s3.endpoint` | 必需 | 用于对象存储的端点。<br/>注意，请不要提供带有 http:// 或 https://
开头的链接。对于 Azure Blob 存储，链接应该像 ${ak}.blob.core.windows.net/。 |
+| `s3.endpoint` | 必需 | 用于对象存储的端点。<br/>注意，请不要提供带有 http:// 或 https://
开头的链接。对于 Azure Blob 存储，endpoint是固定的blob.core.windows.net。 |
| `s3.region` | 必需 | 您的存储桶的区域。(如果您使用 GCP 或 AZURE,则不需要)。 |
| `s3.root.path` | 必需 | 存储数据的路径。 |
| `s3.bucket` | 必需 | 您的对象存储账户的存储桶。(如果您使用 Azure,则为 StorageAccount)。 |
@@ -65,112 +65,150 @@ CREATE STORAGE VAULT [IF NOT EXISTS] vault
| `hadoop.username` |可选| Hadoop 配置属性，指定访问文件系统的用户。如果没有指定则会使用启动
hadoop 进程的 user。 |
| `hadoop.security.authentication` |可选| 用于 hadoop 的认证方式。如果希望使用 kerberos
则可以填写`kerberos`。 |
| `hadoop.kerberos.principal` |可选| 您的 kerberos 主体的路径。 |
-| `hadoop.kerberos.keytab` |可选| 您的 kerberos keytab 的路径。 |
+| `hadoop.kerberos.keytab` |可选| 您的 kerberos keytab 的路径。 |
### 示例
1. 创建 HDFS storage vault。
```sql
- CREATE STORAGE VAULT IF NOT EXISTS hdfs_vault
- PROPERTIES (
- "type"="hdfs",
- "fs.defaultFS"="hdfs://127.0.0.1:8020"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS hdfs_vault_demo
+ PROPERTIES (
+ "type" = "hdfs", -- required
+ "fs.defaultFS" = "hdfs://127.0.0.1:8020", -- required
+ "path_prefix" = "big/data", -- optional,
一般按照业务名称填写
+ "hadoop.username" = "user" -- optional
+ "hadoop.security.authentication" = "kerberos" -- optional
+ "hadoop.kerberos.principal" = "hadoop/127.0.0.1@XXX" -- optional
+ "hadoop.kerberos.keytab" = "/etc/emr.keytab" -- optional
+ );
```
-2. 创建微软 azure S3 storage vault。
+2. 创建阿里云 OSS storage vault。
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="ak.blob.core.windows.net/",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "AZURE"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS oss_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "oss-cn-beijing.aliyuncs.com", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your OSS access key
+ "s3.secret_key" = "xxxxxx", -- required,
Your OSS secret key
+ "s3.region" = "cn-beijing", -- required
+ "s3.root.path" = "oss_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your OSS bucket name
+ "provider" = "OSS", -- required
+ "use_path_style" = "false" -- optional, OSS
建议设置false
+ );
```
-3. 创建阿里云 OSS S3 storage vault。
+3. 创建腾讯云 COS storage vault。
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="oss.aliyuncs.com",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "cn-hangzhou",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "OSS",
- "use_path_style" = "false"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS cos_demo_vault
+ PROPERTIES (
+ "type" = "S3",
+ "s3.endpoint" = "cos.ap-guangzhou.myqcloud.com", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your COS access key
+ "s3.secret_key" = "xxxxxx", -- required,
Your COS secret key
+ "s3.region" = "ap-guangzhou", -- required
+ "s3.root.path" = "cos_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your COS bucket name
+ "provider" = "COS", -- required
+ "use_path_style" = "false" -- optional, COS
建议设置false
+ );
```
-4. 创建腾讯云 COS S3 storage vault。
+4. 创建华为云 OBS storage vault。
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "ap-guangzhou",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "COS",
- "use_path_style" = "false"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS obs_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "obs.cn-north-4.myhuaweicloud.com", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your OBS access key
+ "s3.secret_key" = "xxxxxx", -- required,
Your OBS secret key
+ "s3.region" = "cn-north-4", -- required
+ "s3.root.path" = "obs_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your COS bucket name
+ "provider" = "OBS", -- required
+ "use_path_style" = "false" -- optional, OBS
建议设置false
+ );
```
-5. 创建华为云 OBS S3 storage vault。
+5. 创建百度云 BOS storage vault。
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="obs.cn-north-4.myhuaweicloud.com",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "cn-north-4",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "OBS",
- "use_path_style" = "false"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS obs_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "s3.bj.bcebos.com", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your BOS access key
+ "s3.secret_key" = "xxxxxx", -- required,
Your BOS secret key
+ "s3.region" = "bj", -- required
+ "s3.root.path" = "bos_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your BOS bucket name
+ "provider" = "BOS", -- required
+ "use_path_style" = "false" -- optional, BOS
建议设置false
+ );
```
6. 创建亚马逊云 S3 storage vault。
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="s3.us-east-1.amazonaws.com",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "us-east-1",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "S3",
- "use_path_style" = "false"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS s3_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "s3.us-east-1.amazonaws.com", -- required
+ "s3.access_key" = "xxxxxx", -- required, Your
S3 access key
+ "s3.secret_key" = "xxxxxx", -- required, Your
OBS secret key
+ "s3.region" = "us-east-1", -- required
+ "s3.root.path" = "s3_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required, Your
s3 bucket name
+ "provider" = "S3", -- required
+ "use_path_style" = "false" -- optional, S3
建议设置false
+ );
```
-7. 创建 MinIO S3 storage vault。
+
+7. 创建 MinIO storage vault。
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="127.0.0.1:9000",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "us-east-1",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "S3",
- "use_path_style" = "true"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS minio_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "127.0.0.1:9000", -- required
+ "s3.access_key" = "xxxxxx", -- required, Your
minio access key
+ "s3.secret_key" = "xxxxxx", -- required, Your
minio secret key
+ "s3.region" = "us-east-1", -- required
+ "s3.root.path" = "minio_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required, Your
minio bucket name
+ "provider" = "S3", -- required
+ "use_path_style" = "true" -- required, minio
建议设置true
+ );
```
+8. 创建微软 AZURE storage vault。
+ ```sql
+ CREATE STORAGE VAULT IF NOT EXISTS azure_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "blob.core.windows.net", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your Azure AccountName
+ "s3.secret_key" = "xxxxxx", -- required,
Your Azure AccountKey
+ "s3.region" = "us-east-1", -- required
+ "s3.root.path" = "azure_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your Azure StorageAccount
+ "provider" = "AZURE" -- required
+ );
+ ```
+
+9. 创建谷歌 GCP storage vault。
+ ```sql
+ CREATE STORAGE VAULT IF NOT EXISTS gcp_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "storage.googleapis.com", -- required
+ "s3.access_key" = "xxxxxx", -- required
+ "s3.secret_key" = "xxxxxx", -- required
+ "s3.region" = "us-east-1", -- required
+ "s3.root.path" = "gcp_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required
+ "provider" = "GCP" -- required
+ );
+ ```
+
### 关键词
CREATE, STORAGE VAULT
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/compute-storage-decoupled/managing-storage-vault.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/compute-storage-decoupled/managing-storage-vault.md
index 76b1e1e6947..b23a6bf68ef 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/compute-storage-decoupled/managing-storage-vault.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/compute-storage-decoupled/managing-storage-vault.md
@@ -40,35 +40,36 @@ PROPERTIES
### 创建 HDFS Storage Vault
-创建基于 HDFS 的存算分离模式 Doris 集群，需要确保所有的节点（包括 FE / BE 节点、Meta Service) 均有权限访问所指定的
HDFS，包括提前完成机器的 Kerberos 授权配置和连通性检查（可在对应的每个节点上使用 Hadoop Client 进行测试）等。
+创建基于 HDFS 的存算分离模式 Doris 集群，需要确保所有的节点 (包括 FE / BE 节点、Meta Service) 均有权限访问所指定的
HDFS，包括提前完成机器的 Kerberos 授权配置和连通性检查（可在对应的每个节点上使用 Hadoop Client 进行测试）等。
```sql
-CREATE STORAGE VAULT IF NOT EXISTS ssb_hdfs_vault
- PROPERTIES (
- "type"="hdfs", -- required
- "fs.defaultFS"="hdfs://127.0.0.1:8020", -- required
- "path_prefix"="big/data", -- optional,
一般按照业务名称填写
- "hadoop.username"="user" -- optional
- "hadoop.security.authentication"="kerberos" -- optional
- "hadoop.kerberos.principal"="hadoop/127.0.0.1@XXX" -- optional
- "hadoop.kerberos.keytab"="/etc/emr.keytab" -- optional
- );
+CREATE STORAGE VAULT IF NOT EXISTS hdfs_vault_demo
+PROPERTIES (
+ "type" = "hdfs", -- required
+ "fs.defaultFS" = "hdfs://127.0.0.1:8020", -- required
+ "path_prefix" = "big/data", -- optional,
一般按照业务名称填写
+ "hadoop.username" = "user" -- optional
+ "hadoop.security.authentication" = "kerberos" -- optional
+ "hadoop.kerberos.principal" = "hadoop/127.0.0.1@XXX" -- optional
+ "hadoop.kerberos.keytab" = "/etc/emr.keytab" -- optional
+);
```
### 创建 S3 Storage Vault
```SQL
-CREATE STORAGE VAULT IF NOT EXISTS ssb_s3_vault
- PROPERTIES (
- "type"="S3", -- required
- "s3.endpoint" = "oss-cn-beijing.aliyuncs.com", -- required
- "s3.region" = "bj", -- required
- "s3.bucket" = "bucket", -- required
- "s3.root.path" = "big/data/prefix", -- required
- "s3.access_key" = "ak", -- required
- "s3.secret_key" = "sk", -- required
- "provider" = "OSS" -- required
- );
+CREATE STORAGE VAULT IF NOT EXISTS s3_vault_demo
+PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "oss-cn-beijing.aliyuncs.com", -- required
+ "s3.region" = "cn-beijing", -- required
+ "s3.bucket" = "bucket", -- required
+ "s3.root.path" = "big/data/prefix", -- required
+ "s3.access_key" = "ak", -- required
+ "s3.secret_key" = "sk", -- required
+ "provider" = "OSS", -- required
+ "use_path_style" = "false" -- optional
+);
```
更多参数说明及示例可见
[CREATE-STORAGE-VAULT](../sql-manual/sql-statements/cluster-management/storage-management/CREATE-STORAGE-VAULT.md)。
@@ -110,8 +111,8 @@ CREATE TABLE IF NOT EXISTS supplier (
UNIQUE KEY (s_suppkey)
DISTRIBUTED BY HASH(s_suppkey) BUCKETS 1
PROPERTIES (
-"replication_num" = "1",
-"storage_vault_name" = "ssb_hdfs_vault"
+ "replication_num" = "1",
+ "storage_vault_name" = "hdfs_demo_vault"
);
```
@@ -171,7 +172,7 @@ GRANT
仅 Admin 用户有权限执行 `GRANT` 语句，该语句用于向 User / Role 授予指定 Storage Vault 的权限。拥有某个
Storage Vault 的 `USAGE_PRIV` 权限的 User / Role 可进行以下操作：
-- 通过 `SHOW STORAGE VAULT` 查看该 Storage Vault 的信息；
+- 通过 `SHOW STORAGE VAULTS` 查看该 Storage Vault 的信息；
- 建表时在 `PROPERTIES` 中指定使用该 Storage Vault 。
### 撤销
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/sql-manual/sql-statements/cluster-management/storage-management/CREATE-STORAGE-VAULT.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/sql-manual/sql-statements/cluster-management/storage-management/CREATE-STORAGE-VAULT.md
index 4e55975ccf0..5c432060aed 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/sql-manual/sql-statements/cluster-management/storage-management/CREATE-STORAGE-VAULT.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/sql-manual/sql-statements/cluster-management/storage-management/CREATE-STORAGE-VAULT.md
@@ -1,6 +1,6 @@
---
{
- "title": "CREATE STORAGE VAULT",
+ "title": "CREATE-STORAGE-VAULT",
"language": "zh-CN",
"toc_min_heading_level": 2,
"toc_max_heading_level": 4
@@ -26,7 +26,7 @@ specific language governing permissions and limitations
under the License.
-->
-
+## CREATE-STORAGE-VAULT
## 描述
@@ -47,130 +47,168 @@ CREATE STORAGE VAULT [IF NOT EXISTS] vault
| 参数 | 是否必需 | 描述
|
|:----------------|:-----|:--------------------------------------------------------------------------------------------------------|
-| `s3.endpoint` | 必需 | 用于对象存储的端点。<br/>注意，请不要提供带有 http:// 或 https://
开头的链接。对于 Azure Blob 存储，链接应该像 ${ak}.blob.core.windows.net/。 |
-| `s3.region` | 必需 | 您的存储桶的区域。(如果您使用 GCP 或 AZURE，则不需要)。 |
+| `s3.endpoint` | 必需 | 用于对象存储的端点。<br/>注意，请不要提供带有 http:// 或 https://
开头的链接。对于 Azure Blob 存储，endpoint是固定的blob.core.windows.net。 |
+| `s3.region` | 必需 | 您的存储桶的区域。(如果您使用 GCP 或 AZURE,则不需要)。 |
| `s3.root.path` | 必需 | 存储数据的路径。 |
-| `s3.bucket` | 必需 | 您的对象存储账户的存储桶。(如果您使用 Azure，则为 StorageAccount)。 |
-| `s3.access_key` | 必需 | 您的对象存储账户的访问密钥。(如果您使用 Azure，则为 AccountName)。 |
-| `s3.secret_key` | 必需 | 您的对象存储账户的秘密密钥。(如果您使用 Azure，则为 AccountKey)。 |
+| `s3.bucket` | 必需 | 您的对象存储账户的存储桶。(如果您使用 Azure,则为 StorageAccount)。 |
+| `s3.access_key` | 必需 | 您的对象存储账户的访问密钥。(如果您使用 Azure,则为 AccountName)。 |
+| `s3.secret_key` | 必需 | 您的对象存储账户的秘密密钥。(如果您使用 Azure,则为 AccountKey)。 |
| `provider` | 必需 |
提供对象存储服务的云供应商。支持的值有`COS`，`OSS`，`S3`，`OBS`，`BOS`，`AZURE`，`GCP` |
-| `use_path_style` | 可选 | 使用 `path-style
URL`(私有化部署环境)或者`virtual-hosted-style URL`(公有云环境建议), 默认值 `true` (`path-style`)
|
+| `use_path_style` | 可选 | 使用 `path-style
URL`(私有化部署环境)或者`virtual-hosted-style URL`(公有云环境建议), 默认值 `true` (path-style)
|
##### HDFS vault
| 参数 | 是否必需 | 描述
|
|:---------------------------------|:-----|:------------------------------------------------------|
-| `fs.defaultFS` |必需 | Hadoop 配置属性，指定要使用的默认文件系统。
|
-| `path_prefix` |可选 | 存储数据的路径前缀。如果没有指定则会使用 user 账户下的默认路径。
|
-| `hadoop.username` |可选 | Hadoop 配置属性，指定访问文件系统的用户。如果没有指定则会使用启动
hadoop 进程的 user。 |
-| `hadoop.security.authentication` |可选 | 用于 hadoop 的认证方式。如果希望使用 kerberos
则可以填写`kerberos`。 |
-| `hadoop.kerberos.principal` |可选 | 您的 kerberos 主体的路径。 |
-| `hadoop.kerberos.keytab` |可选 | 您的 kerberos keytab 的路径。 |
+| `fs.defaultFS` |必需| Hadoop 配置属性,指定要使用的默认文件系统。
|
+| `path_prefix` |可选| 存储数据的路径前缀。如果没有指定则会使用 user 账户下的默认路径。
|
+| `hadoop.username` |可选| Hadoop 配置属性，指定访问文件系统的用户。如果没有指定则会使用启动
hadoop 进程的 user。 |
+| `hadoop.security.authentication` |可选| 用于 hadoop 的认证方式。如果希望使用 kerberos
则可以填写`kerberos`。 |
+| `hadoop.kerberos.principal` |可选| 您的 kerberos 主体的路径。 |
+| `hadoop.kerberos.keytab` |可选| 您的 kerberos keytab 的路径。 |
### 示例
1. 创建 HDFS storage vault。
```sql
- CREATE STORAGE VAULT IF NOT EXISTS hdfs_vault
- PROPERTIES (
- "type"="hdfs",
- "fs.defaultFS"="hdfs://127.0.0.1:8020"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS hdfs_vault_demo
+ PROPERTIES (
+ "type" = "hdfs", -- required
+ "fs.defaultFS" = "hdfs://127.0.0.1:8020", -- required
+ "path_prefix" = "big/data", -- optional,
一般按照业务名称填写
+ "hadoop.username" = "user" -- optional
+ "hadoop.security.authentication" = "kerberos" -- optional
+ "hadoop.kerberos.principal" = "hadoop/127.0.0.1@XXX" -- optional
+ "hadoop.kerberos.keytab" = "/etc/emr.keytab" -- optional
+ );
```
-2. 创建微软 azure S3 storage vault。
+2. 创建阿里云 OSS storage vault。
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="ak.blob.core.windows.net/",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "AZURE"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS oss_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "oss-cn-beijing.aliyuncs.com", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your OSS access key
+ "s3.secret_key" = "xxxxxx", -- required,
Your OSS secret key
+ "s3.region" = "cn-beijing", -- required
+ "s3.root.path" = "oss_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your OSS bucket name
+ "provider" = "OSS", -- required
+ "use_path_style" = "false" -- optional, OSS
建议设置false
+ );
```
-3. 创建阿里云 OSS S3 storage vault。
+3. 创建腾讯云 COS storage vault。
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="oss.aliyuncs.com",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "cn-hangzhou",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "OSS",
- "use_path_style" = "false"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS cos_demo_vault
+ PROPERTIES (
+ "type" = "S3",
+ "s3.endpoint" = "cos.ap-guangzhou.myqcloud.com", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your COS access key
+ "s3.secret_key" = "xxxxxx", -- required,
Your COS secret key
+ "s3.region" = "ap-guangzhou", -- required
+ "s3.root.path" = "cos_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your COS bucket name
+ "provider" = "COS", -- required
+ "use_path_style" = "false" -- optional, COS
建议设置false
+ );
```
-4. 创建腾讯云 COS S3 storage vault。
+4. 创建华为云 OBS storage vault。
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "ap-guangzhou",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "COS",
- "use_path_style" = "false"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS obs_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "obs.cn-north-4.myhuaweicloud.com", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your OBS access key
+ "s3.secret_key" = "xxxxxx", -- required,
Your OBS secret key
+ "s3.region" = "cn-north-4", -- required
+ "s3.root.path" = "obs_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your COS bucket name
+ "provider" = "OBS", -- required
+ "use_path_style" = "false" -- optional, OBS
建议设置false
+ );
```
-5. 创建华为云 OBS S3 storage vault。
+5. 创建百度云 BOS storage vault。
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="obs.cn-north-4.myhuaweicloud.com",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "cn-north-4",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "OBS",
- "use_path_style" = "false"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS obs_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "s3.bj.bcebos.com", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your BOS access key
+ "s3.secret_key" = "xxxxxx", -- required,
Your BOS secret key
+ "s3.region" = "bj", -- required
+ "s3.root.path" = "bos_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your BOS bucket name
+ "provider" = "BOS", -- required
+ "use_path_style" = "false" -- optional, BOS
建议设置false
+ );
```
6. 创建亚马逊云 S3 storage vault。
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="s3.us-east-1.amazonaws.com",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "us-east-1",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "S3",
- "use_path_style" = "false"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS s3_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "s3.us-east-1.amazonaws.com", -- required
+ "s3.access_key" = "xxxxxx", -- required, Your
S3 access key
+ "s3.secret_key" = "xxxxxx", -- required, Your
OBS secret key
+ "s3.region" = "us-east-1", -- required
+ "s3.root.path" = "s3_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required, Your
s3 bucket name
+ "provider" = "S3", -- required
+ "use_path_style" = "false" -- optional, S3
建议设置false
+ );
```
-7. 创建 MinIO S3 storage vault。
+
+7. 创建 MinIO storage vault。
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="127.0.0.1:9000",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "us-east-1",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "S3",
- "use_path_style" = "true"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS minio_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "127.0.0.1:9000", -- required
+ "s3.access_key" = "xxxxxx", -- required, Your
minio access key
+ "s3.secret_key" = "xxxxxx", -- required, Your
minio secret key
+ "s3.region" = "us-east-1", -- required
+ "s3.root.path" = "minio_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required, Your
minio bucket name
+ "provider" = "S3", -- required
+ "use_path_style" = "true" -- required, minio
建议设置true
+ );
```
-## 关键词
+8. 创建微软 AZURE storage vault。
+ ```sql
+ CREATE STORAGE VAULT IF NOT EXISTS azure_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "blob.core.windows.net", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your Azure AccountName
+ "s3.secret_key" = "xxxxxx", -- required,
Your Azure AccountKey
+ "s3.region" = "us-east-1", -- required
+ "s3.root.path" = "azure_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your Azure StorageAccount
+ "provider" = "AZURE" -- required
+ );
+ ```
+
+9. 创建谷歌 GCP storage vault。
+ ```sql
+ CREATE STORAGE VAULT IF NOT EXISTS gcp_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "storage.googleapis.com", -- required
+ "s3.access_key" = "xxxxxx", -- required
+ "s3.secret_key" = "xxxxxx", -- required
+ "s3.region" = "us-east-1", -- required
+ "s3.root.path" = "gcp_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required
+ "provider" = "GCP" -- required
+ );
+ ```
+
+### 关键词
CREATE, STORAGE VAULT
diff --git
a/versioned_docs/version-3.0/compute-storage-decoupled/managing-storage-vault.md
b/versioned_docs/version-3.0/compute-storage-decoupled/managing-storage-vault.md
index 56170c579ce..24216e8077f 100644
---
a/versioned_docs/version-3.0/compute-storage-decoupled/managing-storage-vault.md
+++
b/versioned_docs/version-3.0/compute-storage-decoupled/managing-storage-vault.md
@@ -44,7 +44,7 @@ PROPERTIES
To create an HDFS-based decoupled storage-compute Doris cluster, ensure that
all nodes (including FE/BE nodes, Meta Service) have privilege to access the
specified HDFS, including completing Kerberos authorization configuration and
connectivity checks in advance (which can be tested using Hadoop Client on each
corresponding node).
```sql
-CREATE STORAGE VAULT IF NOT EXISTS ssb_hdfs_vault
+CREATE STORAGE VAULT IF NOT EXISTS hdfs_vault_demo
PROPERTIES (
"type"="hdfs", -- required
"fs.defaultFS"="hdfs://127.0.0.1:8020", -- required
@@ -59,17 +59,18 @@ CREATE STORAGE VAULT IF NOT EXISTS ssb_hdfs_vault
### Create an S3 Storage Vault
```sql
-CREATE STORAGE VAULT IF NOT EXISTS ssb_s3_vault
- PROPERTIES (
- "type"="S3", -- required
- "s3.endpoint" = "oss-cn-beijing.aliyuncs.com", -- required
- "s3.region" = "bj", -- required
- "s3.bucket" = "bucket", -- required
- "s3.root.path" = "big/data/prefix", -- required
- "s3.access_key" = "ak", -- required
- "s3.secret_key" = "sk", -- required
- "provider" = "OSS" -- required
- );
+CREATE STORAGE VAULT IF NOT EXISTS s3_vault_demo
+PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "oss-cn-beijing.aliyuncs.com", -- required
+ "s3.region" = "cn-beijing", -- required
+ "s3.bucket" = "bucket", -- required
+ "s3.root.path" = "big/data/prefix", -- required
+ "s3.access_key" = "ak", -- required
+ "s3.secret_key" = "sk", -- required
+ "provider" = "OSS", -- required
+ "use_path_style" = "false" -- optional
+);
```
More parameter explanations and examples can be found in
[CREATE-STORAGE-VAULT](../sql-manual/sql-statements/cluster-management/storage-management/CREATE-STORAGE-VAULT.md).
@@ -112,7 +113,7 @@ UNIQUE KEY (s_suppkey)
DISTRIBUTED BY HASH(s_suppkey) BUCKETS 1
PROPERTIES (
"replication_num" = "1",
-"storage_vault_name" = "ssb_hdfs_vault"
+"storage_vault_name" = "hdfs_demo_vault"
);
```
@@ -171,7 +172,7 @@ GRANT
Only Admin users have the authority to execute the `GRANT` statement, which is
used to grant specified Storage Vault privileges to User/Role. Users/Roles with
`USAGE_PRIV` privilege for a certain Storage Vault can perform the following
operations:
-- View the information of that Storage Vault through `SHOW STORAGE VAULT`;
+- View the information of that Storage Vault through `SHOW STORAGE VAULTS`;
- Specify the use of that Storage Vault in `PROPERTIES` when creating tables.
**Example**
diff --git
a/versioned_docs/version-3.0/sql-manual/sql-statements/cluster-management/storage-management/CREATE-STORAGE-VAULT.md
b/versioned_docs/version-3.0/sql-manual/sql-statements/cluster-management/storage-management/CREATE-STORAGE-VAULT.md
index 6fc0e219a3a..e0da5f83fbd 100644
---
a/versioned_docs/version-3.0/sql-manual/sql-statements/cluster-management/storage-management/CREATE-STORAGE-VAULT.md
+++
b/versioned_docs/version-3.0/sql-manual/sql-statements/cluster-management/storage-management/CREATE-STORAGE-VAULT.md
@@ -1,6 +1,6 @@
---
{
- "title": "CREATE STORAGE VAULT",
+ "title": "CREATE-STORAGE-VAULT",
"language": "en",
"toc_min_heading_level": 2,
"toc_max_heading_level": 4
@@ -26,9 +26,9 @@ specific language governing permissions and limitations
under the License.
-->
+## CREATE-STORAGE-VAULT
-
-## Description
+### Description
This command is used to create a storage vault. The subject of this document
describes the syntax for creating Doris self-maintained storage vault.
@@ -48,13 +48,13 @@ CREATE STORAGE VAULT [IF NOT EXISTS] vault
| param | is required | desc
|
|:----------------|:------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `s3.endpoint` | required | The endpoint used for object storage.
<br/>**Notice**, please don't provide the endpoint with any `http://` or
`https://`. And for Azure Blob Storage, the endpoint should be like
`${ak}.blob.core.windows.net/`. |
-| `s3.region` | required | The region of your bucket.(Not required when
you'r using GCP or AZURE).
|
-| `s3.root.path` | required | The path where the data would be stored.
|
-| `s3.bucket` | required | The bucket of your object storage account.
(StorageAccount if you're using Azure).
|
-| `s3.access_key` | required | The access key of your object storage
account. (AccountName if you're using Azure).
|
-| `s3.secret_key` | required | The secret key of your object storage
account. (AccountKey if you're using Azure).
|
-| `provider` | required | The cloud vendor which provides the object
storage service. The supported values include `COS`, `OSS`, `S3`, `OBS`, `BOS`,
`AZURE`, `GCP`
|
+| `s3.endpoint` | required | The endpoint used for object storage.
<br/>**Notice**, please don't provide the endpoint with any `http://` or
`https://`. And for Azure Blob Storage, the endpoint should be
`blob.core.windows.net`. |
+| `s3.region` | required | The region of your bucket.(Not required
when you'r using GCP or AZURE).
|
+| `s3.root.path` | required | The path where the data would be stored.
|
+| `s3.bucket` | required | The bucket of your object storage account.
(StorageAccount if you're using Azure).
|
+| `s3.access_key` | required | The access key of your object storage
account. (AccountName if you're using Azure).
|
+| `s3.secret_key` | required | The secret key of your object storage
account. (AccountKey if you're using Azure).
|
+| `provider` | required | The cloud vendor which provides the object
storage service. The supported values include `COS`, `OSS`, `S3`, `OBS`, `BOS`,
`AZURE`, `GCP`
|
| `use_path_style` | optional | Indicate using `path-style URL`(private
environment recommended) or `virtual-hosted-style URL`(public cloud
recommended), default `true` (`path-style`)
|
##### HDFS Vault
@@ -68,111 +68,147 @@ CREATE STORAGE VAULT [IF NOT EXISTS] vault
| `hadoop.kerberos.principal` | optional | The path to your kerberos
principal. |
| `hadoop.kerberos.keytab` | optional | The path to your kerberos
keytab. |
-
-## Example
+### Example
1. create a HDFS storage vault.
```sql
- CREATE STORAGE VAULT IF NOT EXISTS hdfs_vault
- PROPERTIES (
- "type"="hdfs",
- "fs.defaultFS"="hdfs://127.0.0.1:8020"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS hdfs_vault_demo
+ PROPERTIES (
+ "type" = "hdfs", -- required
+ "fs.defaultFS" = "hdfs://127.0.0.1:8020", -- required
+ "path_prefix" = "big/data", -- optional
+ "hadoop.username" = "user" -- optional
+ "hadoop.security.authentication" = "kerberos" -- optional
+ "hadoop.kerberos.principal" = "hadoop/127.0.0.1@XXX" -- optional
+ "hadoop.kerberos.keytab" = "/etc/emr.keytab" -- optional
+ );
```
-2. create a S3 storage vault using azure.
+2. create a S3 storage vault using OSS.
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="ak.blob.core.windows.net/",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "AZURE"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS oss_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "oss-cn-beijing.aliyuncs.com", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your OSS access key
+ "s3.secret_key" = "xxxxxx", -- required,
Your OSS secret key
+ "s3.region" = "cn-beijing", -- required
+ "s3.root.path" = "oss_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your OSS bucket name
+ "provider" = "OSS", -- required
+ "use_path_style" = "false" -- optional, OSS
suggest setting `false`
+ );
```
-3. create a S3 storage vault using OSS.
+3. create a S3 storage vault using COS.
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="oss.aliyuncs.com",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "cn-hangzhou",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "OSS",
- "use_path_style" = "false"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS cos_demo_vault
+ PROPERTIES (
+ "type" = "S3",
+ "s3.endpoint" = "cos.ap-guangzhou.myqcloud.com", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your COS access key
+ "s3.secret_key" = "xxxxxx", -- required,
Your COS secret key
+ "s3.region" = "ap-guangzhou", -- required
+ "s3.root.path" = "cos_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your COS bucket name
+ "provider" = "COS", -- required
+ "use_path_style" = "false" -- optional, COS
suggest setting `false`
+ );
```
-4. create a S3 storage vault using COS.
+4. create a S3 storage vault using OBS.
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="cos.ap-guangzhou.myqcloud.com",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "ap-guangzhou",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "COS",
- "use_path_style" = "false"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS obs_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "obs.cn-north-4.myhuaweicloud.com", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your OBS access key
+ "s3.secret_key" = "xxxxxx", -- required,
Your OBS secret key
+ "s3.region" = "cn-north-4", -- required
+ "s3.root.path" = "obs_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your COS bucket name
+ "provider" = "OBS", -- required
+ "use_path_style" = "false" -- optional, OBS
suggest setting `false`
+ );
```
-5. create a S3 storage vault using OBS.
+5. create a S3 storage vault using BOS.
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="obs.cn-north-4.myhuaweicloud.com",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "cn-north-4",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "OBS",
- "use_path_style" = "false"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS obs_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "s3.bj.bcebos.com", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your BOS access key
+ "s3.secret_key" = "xxxxxx", -- required,
Your BOS secret key
+ "s3.region" = "bj", -- required
+ "s3.root.path" = "bos_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your BOS bucket name
+ "provider" = "BOS", -- required
+ "use_path_style" = "false" -- optional, BOS
suggest setting `false`
+ );
```
6. create a S3 storage vault using AWS.
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="s3.us-east-1.amazonaws.com",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "us-east-1",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "S3",
- "use_path_style" = "false"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS s3_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "s3.us-east-1.amazonaws.com", -- required
+ "s3.access_key" = "xxxxxx", -- required, Your
S3 access key
+ "s3.secret_key" = "xxxxxx", -- required, Your
OBS secret key
+ "s3.region" = "us-east-1", -- required
+ "s3.root.path" = "s3_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required, Your
s3 bucket name
+ "provider" = "S3", -- required
+ "use_path_style" = "false" -- optional, S3
suggest setting `false`
+ );
```
7. create a S3 storage vault using MinIO.
```sql
- CREATE STORAGE VAULT IF NOT EXISTS s3_vault
- PROPERTIES (
- "type"="S3",
- "s3.endpoint"="127.0.0.1:9000",
- "s3.access_key" = "ak",
- "s3.secret_key" = "sk",
- "s3.region" = "us-east-1",
- "s3.root.path" = "ssb_sf1_p2_s3",
- "s3.bucket" = "doris-build-1308700295",
- "provider" = "S3",
- "use_path_style" = "true"
- );
+ CREATE STORAGE VAULT IF NOT EXISTS minio_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "127.0.0.1:9000", -- required
+ "s3.access_key" = "xxxxxx", -- required, Your
minio access key
+ "s3.secret_key" = "xxxxxx", -- required, Your
minio secret key
+ "s3.region" = "us-east-1", -- required
+ "s3.root.path" = "minio_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required, Your
minio bucket name
+ "provider" = "S3", -- required
+ "use_path_style" = "true" -- required, minio
suggest setting `true`
+ );
```
-## Keywords
+8. create a S3 storage vault using AZURE.
+ ```sql
+ CREATE STORAGE VAULT IF NOT EXISTS azure_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "blob.core.windows.net", -- required
+ "s3.access_key" = "xxxxxx", -- required,
Your Azure AccountName
+ "s3.secret_key" = "xxxxxx", -- required,
Your Azure AccountKey
+ "s3.region" = "us-east-1", -- required
+ "s3.root.path" = "azure_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required,
Your Azure StorageAccount
+ "provider" = "AZURE" -- required
+ );
+ ```
+
+9. create a S3 storage vault using GCP.
+ ```sql
+ CREATE STORAGE VAULT IF NOT EXISTS gcp_demo_vault
+ PROPERTIES (
+ "type" = "S3", -- required
+ "s3.endpoint" = "storage.googleapis.com", -- required
+ "s3.access_key" = "xxxxxx", -- required
+ "s3.secret_key" = "xxxxxx", -- required
+ "s3.region" = "us-east-1", -- required
+ "s3.root.path" = "gcp_demo_vault_prefix", -- required
+ "s3.bucket" = "xxxxxx", -- required
+ "provider" = "GCP" -- required
+ );
+ ```
+
+### Keywords
CREATE, STORAGE VAULT
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
