This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 80cc6460111 [opt](paimon) add paimon related properties (#3030)
80cc6460111 is described below
commit 80cc6460111d23873cfa5224d8b7e50e10c1ede1
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Thu Oct 30 22:21:23 2025 +0800
[opt](paimon) add paimon related properties (#3030)
## Versions
- [x] dev
- [x] 3.x
- [x] 2.1
- [ ] 2.0
## Languages
- [x] Chinese
- [x] English
## Docs Checklist
- [ ] Checked by AI
- [ ] Test Cases Built
---
docs/lakehouse/catalogs/paimon-catalog.mdx | 22 ++++++++++++++++++
.../current/lakehouse/catalogs/paimon-catalog.mdx | 24 +++++++++++++++++++-
.../lakehouse/catalogs/paimon-catalog.mdx | 26 ++++++++++++++++++++--
.../lakehouse/catalogs/paimon-catalog.mdx | 26 ++++++++++++++++++++--
.../lakehouse/catalogs/paimon-catalog.mdx | 22 ++++++++++++++++++
.../lakehouse/catalogs/paimon-catalog.mdx | 22 ++++++++++++++++++
6 files changed, 137 insertions(+), 5 deletions(-)
diff --git a/docs/lakehouse/catalogs/paimon-catalog.mdx
b/docs/lakehouse/catalogs/paimon-catalog.mdx
index 20acb8862dd..5d1053f0fe8 100644
--- a/docs/lakehouse/catalogs/paimon-catalog.mdx
+++ b/docs/lakehouse/catalogs/paimon-catalog.mdx
@@ -33,6 +33,7 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
'warehouse' = '<paimon_warehouse>'
{MetaStoreProperties},
{StorageProperties},
+ {PaimonProperties},
{CommonProperties}
);
```
@@ -61,6 +62,27 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
The StorageProperties section is used to fill in connection and
authentication information related to the storage system. Refer to the section
on [Supported Storage Systems] for details.
+* `{PaimonProperties}`
+
+ The PaimonProperties section is used to fill Paimon-related properties.
+
+ When Doris accesses Paimon metadata, it partially uses the Paimon Java SDK,
so some custom parameters may need to be passed through to the Paimon Java SDK.
You can use the following prefixes:
+
+ ```
+ paimon.s3.xxx
+ paimon.s3a.xxx
+ paimon.fs.xxx
+ ```
+
+ These parameters will be automatically converted and passed through. Here
are some examples:
+
+ | Input Parameter | Converted Parameter for Paimon Java SDK |
+ | --- | --- |
+ | "paimon.s3.list.version" = "1" | "fs.s3a.list.version" = "1" |
+ | "paimon.s3.paging.maximum" = "100" | "fs.s3a.paging.maximum" = "100" |
+ | "paimon.fs.s3.read.ahead.buffer.size" = "1" |
"fs.s3a.read.ahead.buffer.size" = "1" |
+ | "paimon.s3a.replication.factor" = "3" | "fs.s3a.replication.factor" = "3" |
+
* `{CommonProperties}`
The CommonProperties section is used to fill in common properties. Please
refer to the [Catalog Overview](../catalog-overview.md) section on [Common
Properties].
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/catalogs/paimon-catalog.mdx
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/catalogs/paimon-catalog.mdx
index 56fd157f58c..12947f61c3d 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/catalogs/paimon-catalog.mdx
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/lakehouse/catalogs/paimon-catalog.mdx
@@ -33,6 +33,7 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
'warehouse' = '<paimon_warehouse>'
{MetaStoreProperties},
{StorageProperties},
+ {PaimonProperties},
{CommonProperties}
);
```
@@ -61,6 +62,27 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
StorageProperties 部分用于填写存储系统相关的连接和认证信息。具体可参阅【支持的存储系统】部分。
+* `{PaimonProperties}`
+
+ PaimonProperties 部分由于填写 Paimon 相关的属性。
+
+ Doris 在访问 Paimon 元数据时，会部分使用 Paimon Java SDK，因此可能有一些自定义参数需要透传给 Paimon Java
SDK。可以使用以下前缀：
+
+ ```
+ paimon.s3.xxx
+ paimon.s3a.xxx
+ paimon.fs.xxx
+ ```
+
+ 这些参数会进行自动转换并透传，下面列举一些示例：
+
+ | 输入参数 | 转换后适用与 Paimon Java SDK 的参数 |
+ | --- | --- |
+ | "paimon.s3.list.version" = "1" | "fs.s3a.list.version" = "1" |
+ | "paimon.s3.paging.maximum" = "100" | "fs.s3a.paging.maximum" = "100" |
+ | "paimon.fs.s3.read.ahead.buffer.size" = "1" |
"fs.s3a.read.ahead.buffer.size" = "1" |
+ | "paimon.s3a.replication.factor" = "3" | "fs.s3a.replication.factor" = "3" |
+
* `{CommonProperties}`
CommonProperties 部分用于填写通用属性。请参阅[ 数据目录概述 ](../catalog-overview.md)中【通用属性】部分。
@@ -1086,7 +1108,7 @@ SELECT `bucket` , COUNT(*) as file_count,
SUM(file_size_in_bytes)/1024/1024 as t
3.1 之前的版本不支持 `s3a` 协议，如果必须使用，可以临时显示添加 `s3a` 协议相关参数临时解决。
- 3.1 之后的版本无需此操作。同时，3.1 之后的版本如果需要对 Paimon 存储参数进行覆盖，也可以显示的添加 paimon.fs.
前缀的参数。 Paimon底层使用 HDFS 的API，因此所有 fs.s3a 的参数都可以使用。
+ 3.1 之后的版本无需此操作。同时，3.1 之后的版本如果需要对 Paimon 存储参数进行覆盖，也可以显示的添加 paimon.fs.
前缀的参数。Paimon 底层使用 HDFS 的 API，因此所有 fs.s3a 的参数都可以使用。
如：
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/catalogs/paimon-catalog.mdx
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/catalogs/paimon-catalog.mdx
index f8d81cae0f8..12947f61c3d 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/catalogs/paimon-catalog.mdx
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/lakehouse/catalogs/paimon-catalog.mdx
@@ -33,6 +33,7 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
'warehouse' = '<paimon_warehouse>'
{MetaStoreProperties},
{StorageProperties},
+ {PaimonProperties},
{CommonProperties}
);
```
@@ -61,6 +62,27 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
StorageProperties 部分用于填写存储系统相关的连接和认证信息。具体可参阅【支持的存储系统】部分。
+* `{PaimonProperties}`
+
+ PaimonProperties 部分由于填写 Paimon 相关的属性。
+
+ Doris 在访问 Paimon 元数据时，会部分使用 Paimon Java SDK，因此可能有一些自定义参数需要透传给 Paimon Java
SDK。可以使用以下前缀：
+
+ ```
+ paimon.s3.xxx
+ paimon.s3a.xxx
+ paimon.fs.xxx
+ ```
+
+ 这些参数会进行自动转换并透传，下面列举一些示例：
+
+ | 输入参数 | 转换后适用与 Paimon Java SDK 的参数 |
+ | --- | --- |
+ | "paimon.s3.list.version" = "1" | "fs.s3a.list.version" = "1" |
+ | "paimon.s3.paging.maximum" = "100" | "fs.s3a.paging.maximum" = "100" |
+ | "paimon.fs.s3.read.ahead.buffer.size" = "1" |
"fs.s3a.read.ahead.buffer.size" = "1" |
+ | "paimon.s3a.replication.factor" = "3" | "fs.s3a.replication.factor" = "3" |
+
* `{CommonProperties}`
CommonProperties 部分用于填写通用属性。请参阅[ 数据目录概述 ](../catalog-overview.md)中【通用属性】部分。
@@ -1085,8 +1107,8 @@ SELECT `bucket` , COUNT(*) as file_count,
SUM(file_size_in_bytes)/1024/1024 as t
```
3.1 之前的版本不支持 `s3a` 协议，如果必须使用，可以临时显示添加 `s3a` 协议相关参数临时解决。
-
- 3.1 之后的版本无需此操作。同时，3.1 之后的版本如果需要对 Paimon 存储参数进行覆盖，也可以显示的添加 paimon.fs.
前缀的参数。 Paimon底层使用 HDFS 的API，因此所有 fs.s3a 的参数都可以使用。
+
+ 3.1 之后的版本无需此操作。同时，3.1 之后的版本如果需要对 Paimon 存储参数进行覆盖，也可以显示的添加 paimon.fs.
前缀的参数。Paimon 底层使用 HDFS 的 API，因此所有 fs.s3a 的参数都可以使用。
如：
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/lakehouse/catalogs/paimon-catalog.mdx
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/lakehouse/catalogs/paimon-catalog.mdx
index f8d81cae0f8..12947f61c3d 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/lakehouse/catalogs/paimon-catalog.mdx
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/lakehouse/catalogs/paimon-catalog.mdx
@@ -33,6 +33,7 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
'warehouse' = '<paimon_warehouse>'
{MetaStoreProperties},
{StorageProperties},
+ {PaimonProperties},
{CommonProperties}
);
```
@@ -61,6 +62,27 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
StorageProperties 部分用于填写存储系统相关的连接和认证信息。具体可参阅【支持的存储系统】部分。
+* `{PaimonProperties}`
+
+ PaimonProperties 部分由于填写 Paimon 相关的属性。
+
+ Doris 在访问 Paimon 元数据时，会部分使用 Paimon Java SDK，因此可能有一些自定义参数需要透传给 Paimon Java
SDK。可以使用以下前缀：
+
+ ```
+ paimon.s3.xxx
+ paimon.s3a.xxx
+ paimon.fs.xxx
+ ```
+
+ 这些参数会进行自动转换并透传，下面列举一些示例：
+
+ | 输入参数 | 转换后适用与 Paimon Java SDK 的参数 |
+ | --- | --- |
+ | "paimon.s3.list.version" = "1" | "fs.s3a.list.version" = "1" |
+ | "paimon.s3.paging.maximum" = "100" | "fs.s3a.paging.maximum" = "100" |
+ | "paimon.fs.s3.read.ahead.buffer.size" = "1" |
"fs.s3a.read.ahead.buffer.size" = "1" |
+ | "paimon.s3a.replication.factor" = "3" | "fs.s3a.replication.factor" = "3" |
+
* `{CommonProperties}`
CommonProperties 部分用于填写通用属性。请参阅[ 数据目录概述 ](../catalog-overview.md)中【通用属性】部分。
@@ -1085,8 +1107,8 @@ SELECT `bucket` , COUNT(*) as file_count,
SUM(file_size_in_bytes)/1024/1024 as t
```
3.1 之前的版本不支持 `s3a` 协议，如果必须使用，可以临时显示添加 `s3a` 协议相关参数临时解决。
-
- 3.1 之后的版本无需此操作。同时，3.1 之后的版本如果需要对 Paimon 存储参数进行覆盖，也可以显示的添加 paimon.fs.
前缀的参数。 Paimon底层使用 HDFS 的API，因此所有 fs.s3a 的参数都可以使用。
+
+ 3.1 之后的版本无需此操作。同时，3.1 之后的版本如果需要对 Paimon 存储参数进行覆盖，也可以显示的添加 paimon.fs.
前缀的参数。Paimon 底层使用 HDFS 的 API，因此所有 fs.s3a 的参数都可以使用。
如：
diff --git a/versioned_docs/version-2.1/lakehouse/catalogs/paimon-catalog.mdx
b/versioned_docs/version-2.1/lakehouse/catalogs/paimon-catalog.mdx
index 20acb8862dd..5d1053f0fe8 100644
--- a/versioned_docs/version-2.1/lakehouse/catalogs/paimon-catalog.mdx
+++ b/versioned_docs/version-2.1/lakehouse/catalogs/paimon-catalog.mdx
@@ -33,6 +33,7 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
'warehouse' = '<paimon_warehouse>'
{MetaStoreProperties},
{StorageProperties},
+ {PaimonProperties},
{CommonProperties}
);
```
@@ -61,6 +62,27 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
The StorageProperties section is used to fill in connection and
authentication information related to the storage system. Refer to the section
on [Supported Storage Systems] for details.
+* `{PaimonProperties}`
+
+ The PaimonProperties section is used to fill Paimon-related properties.
+
+ When Doris accesses Paimon metadata, it partially uses the Paimon Java SDK,
so some custom parameters may need to be passed through to the Paimon Java SDK.
You can use the following prefixes:
+
+ ```
+ paimon.s3.xxx
+ paimon.s3a.xxx
+ paimon.fs.xxx
+ ```
+
+ These parameters will be automatically converted and passed through. Here
are some examples:
+
+ | Input Parameter | Converted Parameter for Paimon Java SDK |
+ | --- | --- |
+ | "paimon.s3.list.version" = "1" | "fs.s3a.list.version" = "1" |
+ | "paimon.s3.paging.maximum" = "100" | "fs.s3a.paging.maximum" = "100" |
+ | "paimon.fs.s3.read.ahead.buffer.size" = "1" |
"fs.s3a.read.ahead.buffer.size" = "1" |
+ | "paimon.s3a.replication.factor" = "3" | "fs.s3a.replication.factor" = "3" |
+
* `{CommonProperties}`
The CommonProperties section is used to fill in common properties. Please
refer to the [Catalog Overview](../catalog-overview.md) section on [Common
Properties].
diff --git a/versioned_docs/version-3.x/lakehouse/catalogs/paimon-catalog.mdx
b/versioned_docs/version-3.x/lakehouse/catalogs/paimon-catalog.mdx
index 20acb8862dd..5d1053f0fe8 100644
--- a/versioned_docs/version-3.x/lakehouse/catalogs/paimon-catalog.mdx
+++ b/versioned_docs/version-3.x/lakehouse/catalogs/paimon-catalog.mdx
@@ -33,6 +33,7 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
'warehouse' = '<paimon_warehouse>'
{MetaStoreProperties},
{StorageProperties},
+ {PaimonProperties},
{CommonProperties}
);
```
@@ -61,6 +62,27 @@ CREATE CATALOG [IF NOT EXISTS] catalog_name PROPERTIES (
The StorageProperties section is used to fill in connection and
authentication information related to the storage system. Refer to the section
on [Supported Storage Systems] for details.
+* `{PaimonProperties}`
+
+ The PaimonProperties section is used to fill Paimon-related properties.
+
+ When Doris accesses Paimon metadata, it partially uses the Paimon Java SDK,
so some custom parameters may need to be passed through to the Paimon Java SDK.
You can use the following prefixes:
+
+ ```
+ paimon.s3.xxx
+ paimon.s3a.xxx
+ paimon.fs.xxx
+ ```
+
+ These parameters will be automatically converted and passed through. Here
are some examples:
+
+ | Input Parameter | Converted Parameter for Paimon Java SDK |
+ | --- | --- |
+ | "paimon.s3.list.version" = "1" | "fs.s3a.list.version" = "1" |
+ | "paimon.s3.paging.maximum" = "100" | "fs.s3a.paging.maximum" = "100" |
+ | "paimon.fs.s3.read.ahead.buffer.size" = "1" |
"fs.s3a.read.ahead.buffer.size" = "1" |
+ | "paimon.s3a.replication.factor" = "3" | "fs.s3a.replication.factor" = "3" |
+
* `{CommonProperties}`
The CommonProperties section is used to fill in common properties. Please
refer to the [Catalog Overview](../catalog-overview.md) section on [Common
Properties].
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
