This is an automated email from the ASF dual-hosted git repository.
liaoxin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 96066013ef9 [docs](load) fix incorrect placement of format properties
in broker load (#2553)
96066013ef9 is described below
commit 96066013ef9efab85ea8d3094ccc906a21ca660f
Author: Kaijie Chen <[email protected]>
AuthorDate: Fri Jun 27 15:41:09 2025 +0800
[docs](load) fix incorrect placement of format properties in broker load
(#2553)
---
.../import/import-way/broker-load-manual.md | 32 ++++++++++++++++++--
.../import/import-way/broker-load-manual.md | 35 ++++++++++++++++++++--
.../import/import-way/broker-load-manual.md | 35 ++++++++++++++++++++--
.../import/import-way/broker-load-manual.md | 35 ++++++++++++++++++++--
.../import/import-way/broker-load-manual.md | 32 ++++++++++++++++++--
.../import/import-way/broker-load-manual.md | 32 ++++++++++++++++++--
6 files changed, 186 insertions(+), 15 deletions(-)
diff --git a/docs/data-operate/import/import-way/broker-load-manual.md
b/docs/data-operate/import/import-way/broker-load-manual.md
index 82c7caac533..28a44989dd7 100644
--- a/docs/data-operate/import/import-way/broker-load-manual.md
+++ b/docs/data-operate/import/import-way/broker-load-manual.md
@@ -198,6 +198,7 @@ if not specified in the user properties, use the Compute
Group named ```default`
LOAD LABEL load_label
(
data_desc1[, data_desc2, ...]
+[format_properties]
)
WITH [S3|HDFS|BROKER broker_name]
[broker_properties]
@@ -225,10 +226,37 @@ The WITH clause specifies how to access the storage
system, and `broker_properti
| "load_parallelism" | Integer | 8 | Limits the maximum parallel instances on
each backend. |
| "send_batch_parallelism" | Integer | 1 | The parallelism for sink node to
send data, when memtable_on_sink_node is disabled. |
| "load_to_single_tablet" | Boolean | "false" | Used to specify whether to
load data only to a single tablet corresponding to the partition. This
parameter is only available when loading to an OLAP table with random
bucketing. |
-| "skip_lines" | Integer | "0" | It will skip some lines in the head of a csv
file. It will be ignored when the format is csv_with_names or
csv_with_names_and_types. |
-| "trim_double_quotes" | Boolean | "false" | Used to specify whether to trim
the outermost double quotes of each field in the source files. |
| "priority" | oneof "HIGH", "NORMAL", "LOW" | "NORMAL" | The priority of the
task. |
+**Format Properties**
+
+| Property Name | Type | Default Value | Description |
+|---------------------|----------|----------------|-------------|
+| `skip_lines` | Integer | `0` | Number of lines to skip at
the start of a CSV file. Ignored if using `csv_with_names` or
`csv_with_names_and_types`. |
+| `trim_double_quotes`| Boolean | `false` | If `true`, trims outermost
double quotes from each field. |
+| `enclose` | String | `""` | Enclosure character for
fields containing delimiters or newlines. E.g., if delimiter is `,` and
encloser is `'`, then `'b,c'` is parsed as one field. |
+| `escape` | String | `""` | Escape character to
include enclosure characters in field content. E.g., `'b,\'c'` keeps `'b,'c'`
as one field when `'` is the enclosure and `\` is the escape. |
+
+Note: Format properties define how to parse the source file (e.g., delimiters,
quote handling) and must be set inside the LOAD clause. Load properties control
the execution behavior (e.g., timeout, retries) and must be set outside, in the
outer PROPERTIES block.
+
+```sql
+LOAD LABEL s3_load_example (
+ DATA INFILE("s3://bucket/path/file.csv")
+ INTO TABLE users
+ COLUMNS TERMINATED BY ","
+ FORMAT AS "CSV"
+ (user_id, name, age)
+ PROPERTIES (
+ "trim_double_quotes" = "true" -- format property
+ )
+)
+WITH S3 (
+ ...
+)
+PROPERTIES (
+ "timeout" = "3600" -- load property
+);
+```
**fe.conf**
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-way/broker-load-manual.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-way/broker-load-manual.md
index b05c8c86b3c..4342837390a 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-way/broker-load-manual.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data-operate/import/import-way/broker-load-manual.md
@@ -194,6 +194,7 @@ CANCEL LOAD FROM demo WHERE LABEL =
"broker_load_2022_04_01";
LOAD LABEL load_label
(
data_desc1[, data_desc2, ...]
+[format_properties]
)
WITH [S3|HDFS|BROKER broker_name]
[broker_properties]
@@ -209,7 +210,7 @@ WITH [S3|HDFS|BROKER broker_name]
### 导入配置参数
-**load properties**
+**导入参数（Load Properties）**
| Property 名称 | 类型 | 默认值 | 说明 |
| --- | --- | --- | --- |
@@ -221,10 +222,38 @@ WITH [S3|HDFS|BROKER broker_name]
| "load_parallelism" | Integer | 8 | 每个 BE 上并发 instance 数量的上限。 |
| "send_batch_parallelism" | Integer | 1 | sink 节点发送数据的并发度，仅在关闭 memtable
前移时生效。 |
| "load_to_single_tablet" | Boolean | "false" | 是否每个分区只导入一个 tablet，默认值为
false。该参数只允许在对带有 random 分桶的 OLAP 表导数的时候设置。 |
-| "skip_lines" | Integer | "0" | 跳过 CSV 文件的前几行。当设置 format 设置为 csv_with_names 或
csv_with_names_and_types 时，该参数会失效。 |
-| "trim_double_quotes" | Boolean | "false" | 是否裁剪掉导入文件每个字段最外层的双引号。 |
| "priority" | "HIGH" 或 "NORMAL" 或 "LOW" | "NORMAL" | 导入任务的优先级。 |
+**格式参数（Format Properties）**
+
+| 参数名 | 类型 | 默认值 | 描述 |
+|---------------------|----------|----------------|-------------|
+| `skip_lines` | Integer | `0` | 跳过 CSV 文件开头的若干行。当格式为 `csv_with_names` 或
`csv_with_names_and_types` 时，此参数无效。 |
+| `trim_double_quotes` | Boolean | `false` | 是否去除字段外层的双引号。 |
+| `enclose` | String | `""` | 字段包含换行符或分隔符时的包裹字符。例如，分隔符为 `,`，包裹字符为 `'`
时，`'b,c'` 会被解析为一个字段。 |
+| `escape` | String | `""` | 用于转义包裹字符的转义字符。例如转义字符为 `\`，包裹字符为 `'`，字段 `'b,\'c'`
会被正确解析为 `'b,'c'`。 |
+
+**注意：**格式参数用于定义如何解析源文件（如分隔符、引号处理），应在 `LOAD` 语句内部的 `PROPERTIES`
中设置。导入参数用于控制导入行为（如超时、重试），应在 `LOAD` 语句外部的最外层 `PROPERTIES` 块中设置。
+
+```sql
+LOAD LABEL s3_load_example (
+ DATA INFILE("s3://bucket/path/file.csv")
+ INTO TABLE users
+ COLUMNS TERMINATED BY ","
+ FORMAT AS "CSV"
+ (user_id, name, age)
+ PROPERTIES (
+ "trim_double_quotes" = "true" -- 格式参数
+ )
+)
+WITH S3 (
+ ...
+)
+PROPERTIES (
+ "timeout" = "3600" -- 导入参数
+);
+```
+
**fe.conf**
下面几个配置属于 Broker load 的系统级别配置，也就是作用于所有 Broker load 导入任务的配置。主要通过修改
`fe.conf`来调整配置值。
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/data-operate/import/import-way/broker-load-manual.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/data-operate/import/import-way/broker-load-manual.md
index a15d1438a5e..a89ae5c7fec 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/data-operate/import/import-way/broker-load-manual.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/data-operate/import/import-way/broker-load-manual.md
@@ -186,6 +186,7 @@ CANCEL LOAD FROM demo WHERE LABEL =
"broker_load_2022_04_01";
LOAD LABEL load_label
(
data_desc1[, data_desc2, ...]
+[format_properties]
)
WITH [S3|HDFS|BROKER broker_name]
[broker_properties]
@@ -201,7 +202,7 @@ WITH [S3|HDFS|BROKER broker_name]
### 导入配置参数
-**load properties**
+**导入参数（Load Properties）**
| Property 名称 | 类型 | 默认值 | 说明 |
| --- | --- | --- | --- |
@@ -213,10 +214,38 @@ WITH [S3|HDFS|BROKER broker_name]
| "load_parallelism" | Integer | 8 | 每个 BE 上并发 instance 数量的上限。 |
| "send_batch_parallelism" | Integer | 1 | sink 节点发送数据的并发度，仅在关闭 memtable
前移时生效。 |
| "load_to_single_tablet" | Boolean | "false" | 是否每个分区只导入一个 tablet，默认值为
false。该参数只允许在对带有 random 分桶的 OLAP 表导数的时候设置。 |
-| "skip_lines" | Integer | "0" | 跳过 CSV 文件的前几行。当设置 format 设置为 csv_with_names 或
csv_with_names_and_types 时，该参数会失效。 |
-| "trim_double_quotes" | Boolean | "false" | 是否裁剪掉导入文件每个字段最外层的双引号。 |
| "priority" | "HIGH" 或 "NORMAL" 或 "LOW" | "NORMAL" | 导入任务的优先级。 |
+**格式参数（Format Properties）**
+
+| 参数名 | 类型 | 默认值 | 描述 |
+|---------------------|----------|----------------|-------------|
+| `skip_lines` | Integer | `0` | 跳过 CSV 文件开头的若干行。当格式为 `csv_with_names` 或
`csv_with_names_and_types` 时，此参数无效。 |
+| `trim_double_quotes` | Boolean | `false` | 是否去除字段外层的双引号。 |
+| `enclose` | String | `""` | 字段包含换行符或分隔符时的包裹字符。例如，分隔符为 `,`，包裹字符为 `'`
时，`'b,c'` 会被解析为一个字段。 |
+| `escape` | String | `""` | 用于转义包裹字符的转义字符。例如转义字符为 `\`，包裹字符为 `'`，字段 `'b,\'c'`
会被正确解析为 `'b,'c'`。 |
+
+**注意：**格式参数用于定义如何解析源文件（如分隔符、引号处理），应在 `LOAD` 语句内部的 `PROPERTIES`
中设置。导入参数用于控制导入行为（如超时、重试），应在 `LOAD` 语句外部的最外层 `PROPERTIES` 块中设置。
+
+```sql
+LOAD LABEL s3_load_example (
+ DATA INFILE("s3://bucket/path/file.csv")
+ INTO TABLE users
+ COLUMNS TERMINATED BY ","
+ FORMAT AS "CSV"
+ (user_id, name, age)
+ PROPERTIES (
+ "trim_double_quotes" = "true" -- 格式参数
+ )
+)
+WITH S3 (
+ ...
+)
+PROPERTIES (
+ "timeout" = "3600" -- 导入参数
+);
+```
+
**fe.conf**
下面几个配置属于 Broker load 的系统级别配置，也就是作用于所有 Broker load 导入任务的配置。主要通过修改
`fe.conf`来调整配置值。
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/data-operate/import/import-way/broker-load-manual.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/data-operate/import/import-way/broker-load-manual.md
index a15d1438a5e..a89ae5c7fec 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/data-operate/import/import-way/broker-load-manual.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/data-operate/import/import-way/broker-load-manual.md
@@ -186,6 +186,7 @@ CANCEL LOAD FROM demo WHERE LABEL =
"broker_load_2022_04_01";
LOAD LABEL load_label
(
data_desc1[, data_desc2, ...]
+[format_properties]
)
WITH [S3|HDFS|BROKER broker_name]
[broker_properties]
@@ -201,7 +202,7 @@ WITH [S3|HDFS|BROKER broker_name]
### 导入配置参数
-**load properties**
+**导入参数（Load Properties）**
| Property 名称 | 类型 | 默认值 | 说明 |
| --- | --- | --- | --- |
@@ -213,10 +214,38 @@ WITH [S3|HDFS|BROKER broker_name]
| "load_parallelism" | Integer | 8 | 每个 BE 上并发 instance 数量的上限。 |
| "send_batch_parallelism" | Integer | 1 | sink 节点发送数据的并发度，仅在关闭 memtable
前移时生效。 |
| "load_to_single_tablet" | Boolean | "false" | 是否每个分区只导入一个 tablet，默认值为
false。该参数只允许在对带有 random 分桶的 OLAP 表导数的时候设置。 |
-| "skip_lines" | Integer | "0" | 跳过 CSV 文件的前几行。当设置 format 设置为 csv_with_names 或
csv_with_names_and_types 时，该参数会失效。 |
-| "trim_double_quotes" | Boolean | "false" | 是否裁剪掉导入文件每个字段最外层的双引号。 |
| "priority" | "HIGH" 或 "NORMAL" 或 "LOW" | "NORMAL" | 导入任务的优先级。 |
+**格式参数（Format Properties）**
+
+| 参数名 | 类型 | 默认值 | 描述 |
+|---------------------|----------|----------------|-------------|
+| `skip_lines` | Integer | `0` | 跳过 CSV 文件开头的若干行。当格式为 `csv_with_names` 或
`csv_with_names_and_types` 时，此参数无效。 |
+| `trim_double_quotes` | Boolean | `false` | 是否去除字段外层的双引号。 |
+| `enclose` | String | `""` | 字段包含换行符或分隔符时的包裹字符。例如，分隔符为 `,`，包裹字符为 `'`
时，`'b,c'` 会被解析为一个字段。 |
+| `escape` | String | `""` | 用于转义包裹字符的转义字符。例如转义字符为 `\`，包裹字符为 `'`，字段 `'b,\'c'`
会被正确解析为 `'b,'c'`。 |
+
+**注意：**格式参数用于定义如何解析源文件（如分隔符、引号处理），应在 `LOAD` 语句内部的 `PROPERTIES`
中设置。导入参数用于控制导入行为（如超时、重试），应在 `LOAD` 语句外部的最外层 `PROPERTIES` 块中设置。
+
+```sql
+LOAD LABEL s3_load_example (
+ DATA INFILE("s3://bucket/path/file.csv")
+ INTO TABLE users
+ COLUMNS TERMINATED BY ","
+ FORMAT AS "CSV"
+ (user_id, name, age)
+ PROPERTIES (
+ "trim_double_quotes" = "true" -- 格式参数
+ )
+)
+WITH S3 (
+ ...
+)
+PROPERTIES (
+ "timeout" = "3600" -- 导入参数
+);
+```
+
**fe.conf**
下面几个配置属于 Broker load 的系统级别配置，也就是作用于所有 Broker load 导入任务的配置。主要通过修改
`fe.conf`来调整配置值。
diff --git
a/versioned_docs/version-2.1/data-operate/import/import-way/broker-load-manual.md
b/versioned_docs/version-2.1/data-operate/import/import-way/broker-load-manual.md
index feb2b2bcd31..f79af970efd 100644
---
a/versioned_docs/version-2.1/data-operate/import/import-way/broker-load-manual.md
+++
b/versioned_docs/version-2.1/data-operate/import/import-way/broker-load-manual.md
@@ -189,6 +189,7 @@ CANCEL LOAD FROM demo WHERE LABEL =
"broker_load_2022_04_01";
LOAD LABEL load_label
(
data_desc1[, data_desc2, ...]
+[format_properties]
)
WITH [S3|HDFS|BROKER broker_name]
[broker_properties]
@@ -216,10 +217,37 @@ The WITH clause specifies how to access the storage
system, and `broker_properti
| "load_parallelism" | Integer | 8 | Limits the maximum parallel instances on
each backend. |
| "send_batch_parallelism" | Integer | 1 | The parallelism for sink node to
send data, when memtable_on_sink_node is disabled. |
| "load_to_single_tablet" | Boolean | "false" | Used to specify whether to
load data only to a single tablet corresponding to the partition. This
parameter is only available when loading to an OLAP table with random
bucketing. |
-| "skip_lines" | Integer | "0" | It will skip some lines in the head of a csv
file. It will be ignored when the format is csv_with_names or
csv_with_names_and_types. |
-| "trim_double_quotes" | Boolean | "false" | Used to specify whether to trim
the outermost double quotes of each field in the source files. |
| "priority" | oneof "HIGH", "NORMAL", "LOW" | "NORMAL" | The priority of the
task. |
+**Format Properties**
+
+| Property Name | Type | Default Value | Description |
+|---------------------|----------|----------------|-------------|
+| `skip_lines` | Integer | `0` | Number of lines to skip at
the start of a CSV file. Ignored if using `csv_with_names` or
`csv_with_names_and_types`. |
+| `trim_double_quotes`| Boolean | `false` | If `true`, trims outermost
double quotes from each field. |
+| `enclose` | String | `""` | Enclosure character for
fields containing delimiters or newlines. E.g., if delimiter is `,` and
encloser is `'`, then `'b,c'` is parsed as one field. |
+| `escape` | String | `""` | Escape character to
include enclosure characters in field content. E.g., `'b,\'c'` keeps `'b,'c'`
as one field when `'` is the enclosure and `\` is the escape. |
+
+Note: Format properties define how to parse the source file (e.g., delimiters,
quote handling) and must be set inside the LOAD clause. Load properties control
the execution behavior (e.g., timeout, retries) and must be set outside, in the
outer PROPERTIES block.
+
+```sql
+LOAD LABEL s3_load_example (
+ DATA INFILE("s3://bucket/path/file.csv")
+ INTO TABLE users
+ COLUMNS TERMINATED BY ","
+ FORMAT AS "CSV"
+ (user_id, name, age)
+ PROPERTIES (
+ "trim_double_quotes" = "true" -- format property
+ )
+)
+WITH S3 (
+ ...
+)
+PROPERTIES (
+ "timeout" = "3600" -- load property
+);
+```
**fe.conf**
diff --git
a/versioned_docs/version-3.0/data-operate/import/import-way/broker-load-manual.md
b/versioned_docs/version-3.0/data-operate/import/import-way/broker-load-manual.md
index feb2b2bcd31..f79af970efd 100644
---
a/versioned_docs/version-3.0/data-operate/import/import-way/broker-load-manual.md
+++
b/versioned_docs/version-3.0/data-operate/import/import-way/broker-load-manual.md
@@ -189,6 +189,7 @@ CANCEL LOAD FROM demo WHERE LABEL =
"broker_load_2022_04_01";
LOAD LABEL load_label
(
data_desc1[, data_desc2, ...]
+[format_properties]
)
WITH [S3|HDFS|BROKER broker_name]
[broker_properties]
@@ -216,10 +217,37 @@ The WITH clause specifies how to access the storage
system, and `broker_properti
| "load_parallelism" | Integer | 8 | Limits the maximum parallel instances on
each backend. |
| "send_batch_parallelism" | Integer | 1 | The parallelism for sink node to
send data, when memtable_on_sink_node is disabled. |
| "load_to_single_tablet" | Boolean | "false" | Used to specify whether to
load data only to a single tablet corresponding to the partition. This
parameter is only available when loading to an OLAP table with random
bucketing. |
-| "skip_lines" | Integer | "0" | It will skip some lines in the head of a csv
file. It will be ignored when the format is csv_with_names or
csv_with_names_and_types. |
-| "trim_double_quotes" | Boolean | "false" | Used to specify whether to trim
the outermost double quotes of each field in the source files. |
| "priority" | oneof "HIGH", "NORMAL", "LOW" | "NORMAL" | The priority of the
task. |
+**Format Properties**
+
+| Property Name | Type | Default Value | Description |
+|---------------------|----------|----------------|-------------|
+| `skip_lines` | Integer | `0` | Number of lines to skip at
the start of a CSV file. Ignored if using `csv_with_names` or
`csv_with_names_and_types`. |
+| `trim_double_quotes`| Boolean | `false` | If `true`, trims outermost
double quotes from each field. |
+| `enclose` | String | `""` | Enclosure character for
fields containing delimiters or newlines. E.g., if delimiter is `,` and
encloser is `'`, then `'b,c'` is parsed as one field. |
+| `escape` | String | `""` | Escape character to
include enclosure characters in field content. E.g., `'b,\'c'` keeps `'b,'c'`
as one field when `'` is the enclosure and `\` is the escape. |
+
+Note: Format properties define how to parse the source file (e.g., delimiters,
quote handling) and must be set inside the LOAD clause. Load properties control
the execution behavior (e.g., timeout, retries) and must be set outside, in the
outer PROPERTIES block.
+
+```sql
+LOAD LABEL s3_load_example (
+ DATA INFILE("s3://bucket/path/file.csv")
+ INTO TABLE users
+ COLUMNS TERMINATED BY ","
+ FORMAT AS "CSV"
+ (user_id, name, age)
+ PROPERTIES (
+ "trim_double_quotes" = "true" -- format property
+ )
+)
+WITH S3 (
+ ...
+)
+PROPERTIES (
+ "timeout" = "3600" -- load property
+);
+```
**fe.conf**
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
