This is an automated email from the ASF dual-hosted git repository.
zhangshenghang pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/seatunnel.git
The following commit(s) were added to refs/heads/dev by this push:
new 9106488a36 [Docs][Connector-V2] Improve Kudu RocketMQ and Http docs
(#11317)
9106488a36 is described below
commit 9106488a3627abd1463b55d08b863214c5a551dc
Author: Daniel Carter <[email protected]>
AuthorDate: Sat Jul 4 09:51:31 2026 +0800
[Docs][Connector-V2] Improve Kudu RocketMQ and Http docs (#11317)
Co-authored-by: DanielCarter-stack
<[email protected]>
---
docs/en/connectors/sink/Http.md | 8 +++++---
docs/en/connectors/sink/Kudu.md | 18 ++++++++++++------
docs/en/connectors/sink/RocketMQ.md | 2 ++
docs/en/connectors/source/Http.md | 21 +++++++++------------
docs/en/connectors/source/Kudu.md | 13 +++++++++----
docs/en/connectors/source/RocketMQ.md | 2 ++
docs/zh/connectors/sink/Http.md | 15 +++++++++------
docs/zh/connectors/sink/Kudu.md | 30 ++++++++++++++++++------------
docs/zh/connectors/sink/RocketMQ.md | 2 ++
docs/zh/connectors/source/Http.md | 12 +++++++++---
docs/zh/connectors/source/Kudu.md | 13 +++++++++----
docs/zh/connectors/source/RocketMQ.md | 2 ++
12 files changed, 88 insertions(+), 50 deletions(-)
diff --git a/docs/en/connectors/sink/Http.md b/docs/en/connectors/sink/Http.md
index b1c5e42ce1..b06b2be3d0 100644
--- a/docs/en/connectors/sink/Http.md
+++ b/docs/en/connectors/sink/Http.md
@@ -40,18 +40,20 @@ They can be downloaded via install-plugin.sh or from the
Maven central repositor
|-----------------------------|--------|----------|---------|-------------------------------------------------------------------------------------------------------------|
| url | String | Yes | - | Http request url
|
| headers | Map | No | - | Http headers
|
+| params | Map | No | - | Accepted by the
option rule. For the current sink writer, put query parameters directly in
`url`; rows are posted to the final URL as the request body. |
| retry | Int | No | - | The max retry
times if request http return to `IOException`
|
| retry_backoff_multiplier_ms | Int | No | 100 | The
retry-backoff times(millis) multiplier if request http failed
|
| retry_backoff_max_ms | Int | No | 10000 | The maximum
retry-backoff times(millis) if request http failed
|
-| connect_timeout_ms | Int | No | 12000 | Connection
timeout setting, default 12s.
|
-| socket_timeout_ms | Int | No | 60000 | Socket timeout
setting, default 60s.
|
| array_mode | Boolean| No | false | Send data as a
JSON array when true, or as a single JSON object when false (default)
|
| batch_size | Int | No | 1 | The batch size
of records to send in one HTTP request. Only works when array_mode is true.
|
| request_interval_ms | Int | No | 0 | The interval
milliseconds between two HTTP requests, to avoid sending requests too
frequently. |
+| multi_table_sink_replica | Int | No | - | Number of sink
replicas used for multi-table write. See [Sink Common
Options](../common-options/sink-common-options.md). |
| common-options | | No | - | Sink plugin
common parameters, please refer to [Sink Common
Options](../common-options/sink-common-options.md) for details |
## Example
+The Http sink always sends `POST` requests. Each upstream row is converted to
JSON and used as the request body. When `array_mode = true`, rows are
accumulated into a JSON array before sending; `batch_size` controls the maximum
number of rows in one request.
+
simple:
```hocon
@@ -149,4 +151,4 @@ sink {
## Changelog
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
diff --git a/docs/en/connectors/sink/Kudu.md b/docs/en/connectors/sink/Kudu.md
index 8879dbc326..b1f0121462 100644
--- a/docs/en/connectors/sink/Kudu.md
+++ b/docs/en/connectors/sink/Kudu.md
@@ -40,7 +40,7 @@ import ChangeLog from '../changelog/connector-kudu.md';
| Name | Type | Required |
Default |
Description
|
|-------------------------------------------|--------|----------|------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|
| kudu_masters | String | Yes | -
| Kudu master address. Separated by ',',such
as '192.168.88.110:7051'.
|
-| table_name | String | Yes | -
| The name of kudu table.
|
+| table_name | String | No | Upstream
table name | The name of the Kudu table. If this
option is omitted, SeaTunnel uses the upstream table name. In multi-table jobs,
placeholders such as `${database_name}`, `${schema_name}`, and `${table_name}`
can be used. |
| client_worker_count | Int | No | 2 *
Runtime.getRuntime().availableProcessors() | Kudu worker count. Default value
is twice the current number of cpu cores.
|
| client_default_operation_timeout_ms | Long | No | 30000
| Kudu normal operation time out.
|
| client_default_admin_operation_timeout_ms | Long | No | 30000
| Kudu admin operation time out.
|
@@ -48,14 +48,20 @@ import ChangeLog from '../changelog/connector-kudu.md';
| kerberos_principal | String | No | -
| Kerberos principal. Note that all zeta
nodes require have this file.
|
| kerberos_keytab | String | No | -
| Kerberos keytab. Note that all zeta nodes
require have this file.
|
| kerberos_krb5conf | String | No | -
| Kerberos krb5 conf. Note that all zeta
nodes require have this file.
|
-| save_mode | String | No | APPEND
| Storage mode, support `overwrite` and
`append`.
|
-| session_flush_mode | String | No |
AUTO_FLUSH_SYNC | Kudu flush mode. Default
AUTO_FLUSH_SYNC.
|
-| batch_size | Int | No | 1024
| The flush max size (includes all append,
upsert and delete records), over this number of records, will flush data. The
default value is 1024 |
-| buffer_flush_interval | Int | No | 10000
| The flush interval mills, over this time,
asynchronous threads will flush data.
|
+| save_mode | String | No | APPEND
| Storage mode. Supported values are `append`
and `overwrite`. In `overwrite` mode, insert rows are written as Kudu upserts.
|
+| session_flush_mode | String | No |
AUTO_FLUSH_SYNC | Kudu flush mode. Supported
values are `AUTO_FLUSH_SYNC`, `AUTO_FLUSH_BACKGROUND`, and `MANUAL_FLUSH`.
|
+| batch_size | Int | No | 1024
| Required only when `session_flush_mode` is
`AUTO_FLUSH_BACKGROUND` or `MANUAL_FLUSH`. The writer flushes after this many
append, upsert, or delete records. |
+| buffer_flush_interval | Int | No | 10000
| Required only when `session_flush_mode =
AUTO_FLUSH_BACKGROUND`. The asynchronous writer flush interval, in
milliseconds. |
| ignore_not_found | Bool | No | false
| If true, ignore all not found rows.
|
-| ignore_not_duplicate | Bool | No | false
| If true, ignore all dulicate rows.
|
+| ignore_not_duplicate | Bool | No | false
| If true, ignore all duplicate rows.
|
| common-options | | No | -
| Sink plugin common parameters, please refer
to [Sink Common Options](../common-options/sink-common-options.md) for details.
|
+## Option Notes
+
+- `table_name` is optional. If it is not set, the sink writes to the table
name carried by the upstream row.
+- For multi-table jobs, use placeholders in `table_name` to route rows to
different Kudu tables.
+- CDC rows are supported: insert records are appended, update records are
written as upserts, and delete records are deleted by key.
+
## Task Example
### Simple
diff --git a/docs/en/connectors/sink/RocketMQ.md
b/docs/en/connectors/sink/RocketMQ.md
index f9a22f0a7f..b11c69378a 100644
--- a/docs/en/connectors/sink/RocketMQ.md
+++ b/docs/en/connectors/sink/RocketMQ.md
@@ -59,6 +59,8 @@ partition.key.fields = ["c_int"]
The sink supports exactly-once writes through RocketMQ transactional messages.
This behavior is disabled by default. Set `exactly.once = true` when the
RocketMQ cluster and the job checkpoint settings are ready for transactional
writes.
+When `format = text`, SeaTunnel serializes fields in the upstream schema order
and joins them with `field.delimiter`. When `format = json`, each row is
written as a JSON object.
+
## Task Examples
### Write JSON Messages
diff --git a/docs/en/connectors/source/Http.md
b/docs/en/connectors/source/Http.md
index 3ea5f9d13f..caedd1058e 100644
--- a/docs/en/connectors/source/Http.md
+++ b/docs/en/connectors/source/Http.md
@@ -23,15 +23,6 @@ import ChangeLog from '../changelog/connector-http.md';
Used to read data from Http.
-## Key features
-
-- [x] [batch](../../introduction/concepts/connector-v2-features.md)
-- [x] [stream](../../introduction/concepts/connector-v2-features.md)
-- [ ] [exactly-once](../../introduction/concepts/connector-v2-features.md)
-- [ ] [column projection](../../introduction/concepts/connector-v2-features.md)
-- [ ] [parallelism](../../introduction/concepts/connector-v2-features.md)
-- [ ] [support user-defined
split](../../introduction/concepts/connector-v2-features.md)
-
Supported DataSource Info
-------------------------
@@ -51,10 +42,10 @@ They can be downloaded via install-plugin.sh or from the
Maven central repositor
| schema.fields | Config | No | - | The
schema fields of upstream data
|
| json_field | Config | No | - | This
parameter helps you configure the schema,so this parameter must be used with
schema.
|
| pageing | Config | No | - | This
parameter is used for paging queries
|
-| pageing.page_field | String | No | - | This
parameter is used to specify the page field name in the request. It can be used
in headers, params, or body with placeholders like ${page_field}.
|
+| pageing.page_field | String | No | page | This
parameter is used to specify the page field name in the request. It can be used
in headers, params, or body with placeholders like `${page}`.
|
| pageing.use_placeholder_replacement | Boolean | No | false | If true, use
placeholder replacement (${field}) for headers, parameters and body values,
otherwise use key-based replacement.
|
-| pageing.total_page_size | Int | No | - | This
parameter is used to control the total number of pages
|
-| pageing.batch_size | Int | No | - | The batch
size returned per request is used to determine whether to continue when the
total number of pages is unknown
|
+| pageing.total_page_size | Long | No | 0 | This
parameter is used to control the total number of pages. `0` means the connector
continues until the returned row count is less than `pageing.batch_size`.
|
+| pageing.batch_size | Int | No | 100 | The batch
size returned per request is used to determine whether to continue when the
total number of pages is unknown.
|
| pageing.start_page_number | Int | No | 1 | Specify
the page number from which synchronization starts
|
| pageing.page_type | String | No | PageNumber | this
parameter is used to specify the page type ,or PageNumber if not set, only
support `PageNumber` and `Cursor`. |
| pageing.cursor_field | String | No | - | this
parameter is used to specify the Cursor field name in the request parameter.
|
@@ -78,6 +69,12 @@ They can be downloaded via install-plugin.sh or from the
Maven central repositor
| keep_page_param_as_http_param | Boolean | No | false |
Whether to set the paging parameters to params. For compatibility with legacy
behaviors.|
| json_filed_missed_return_null | Boolean | No | false
| When the json field is missing, set true return null else error.|
+## Option Notes
+
+- `pageing` is intentionally spelled this way in the connector option name.
Keep this spelling in job configs.
+- `format = json` normally needs `schema`. `json_field` is used with `schema`
when fields are extracted from nested JSON paths. `content_field` can extract a
JSON array or object fragment directly.
+- `format = binary` outputs fixed fields `(data: bytes, relativePath: string,
partIndex: long)`, only works in batch mode, and cannot be used with `pageing`.
+
## How to Create a Http Data Synchronization Jobs
diff --git a/docs/en/connectors/source/Kudu.md
b/docs/en/connectors/source/Kudu.md
index b6257e99fc..810baabebf 100644
--- a/docs/en/connectors/source/Kudu.md
+++ b/docs/en/connectors/source/Kudu.md
@@ -47,7 +47,7 @@ The tested kudu version is 1.11.1.
| Name | Type | Required | Default
| Description
|
|-------------------------------------------|--------|----------|------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| kudu_masters | String | Yes | -
| Kudu master address. Separated by ',',such
as '192.168.88.110:7051'.
|
-| table_name | String | Yes | -
| The name of kudu table.
|
+| table_name | String | Yes, if `table_list` is
not configured | - | The name of
the Kudu table. This option is mutually exclusive with `table_list`.
|
| client_worker_count | Int | No | 2 *
Runtime.getRuntime().availableProcessors() | Kudu worker count. Default value
is twice the current number of cpu cores.
|
| client_default_operation_timeout_ms | Long | No | 30000
| Kudu normal operation time out.
|
| client_default_admin_operation_timeout_ms | Long | No | 30000
| Kudu admin operation time out.
|
@@ -59,10 +59,16 @@ The tested kudu version is 1.11.1.
| scan_token_batch_size_bytes | Int | No | 1024 * 1024
| Kudu scan bytes. The maximum number of
bytes read at a time, the default is 1MB.
|
| use_regex | Bool | No | false
| Control regular expression matching for
`table_name`. When set to `true`, the `table_name` will be treated as a regular
expression pattern and can match multiple tables. When set to `false` or not
specified, the `table_name` will be treated as an exact table name (no regex
matching). |
| filter | String | No | -
| Kudu scan filter expressions,example id >
100 AND id < 200.
|
-| schema | Map | No | 1024 * 1024
| SeaTunnel Schema. For more details, please
refer to [Schema Feature](../../introduction/concepts/schema-feature.md).
|
-| table_list | Array | No | -
| The list of tables to be read. you can use
this configuration instead of `table_name`, for example: ```table_list = [{
table_name = "kudu_source_table_1"},{ table_name = "kudu_source_table_2"}] ```.
You can also configure `use_regex = true` inside each entry to enable regex
matching for `table_name`. |
+| schema | Map | No | -
| SeaTunnel Schema. For more details, please
refer to [Schema Feature](../../introduction/concepts/schema-feature.md).
|
+| table_list | Array | No | -
| The list of tables to read. Use this option
instead of `table_name`, for example: ```table_list = [{ table_name =
"kudu_source_table_1"},{ table_name = "kudu_source_table_2"}] ```. You can also
configure `use_regex = true` inside each entry to enable regex matching for
that `table_name`. |
| common-options | | No | -
| Source plugin common parameters, please
refer to [Source Common Options](../common-options/source-common-options.md)
for details.
|
+## Option Notes
+
+- Configure exactly one of `table_name` and `table_list`.
+- `filter` is pushed down to Kudu scans and can use Kudu predicate expressions
such as `id >= 1 AND id <= 2`.
+- `use_regex = true` treats `table_name` as a Java regular expression. This
can be used either at the top level or inside each `table_list` item.
+
## Task Example
### Simple
@@ -217,4 +223,3 @@ source {
## Changelog
<ChangeLog />
-
diff --git a/docs/en/connectors/source/RocketMQ.md
b/docs/en/connectors/source/RocketMQ.md
index 27b55ad39f..3dce2b4420 100644
--- a/docs/en/connectors/source/RocketMQ.md
+++ b/docs/en/connectors/source/RocketMQ.md
@@ -76,6 +76,8 @@ start.mode.offsets = {
Use `tables_configs` when different topics have different schemas. Each item
can define its own `topics`, `schema`, `format`, `tags`, and startup position.
If `schema.table` is not set, the output table name defaults to the topic name.
+`topics`, `tables_configs`, and the deprecated `table_list` are mutually
exclusive. In `tables_configs`, options that are not set on an item inherit the
top-level defaults, so each item only needs to override the topic-specific
schema, tags, or startup position.
+
## Task Examples
### Read JSON Messages
diff --git a/docs/zh/connectors/sink/Http.md b/docs/zh/connectors/sink/Http.md
index f2e5dda5ef..afb3be4d1c 100644
--- a/docs/zh/connectors/sink/Http.md
+++ b/docs/zh/connectors/sink/Http.md
@@ -2,7 +2,7 @@ import ChangeLog from '../changelog/connector-http.md';
# Http
-> Http 数据接收器
+> Http Sink 连接器
## 支持引擎
@@ -13,8 +13,9 @@ import ChangeLog from '../changelog/connector-http.md';
## 主要特性
- [ ] [精确一次](../../introduction/concepts/connector-v2-features.md)
-- [ ] [cdc](../../introduction/concepts/connector-v2-features.md)
-- [ ] [timer flush](../../introduction/concepts/connector-v2-features.md)
+- [ ] [变更数据捕获](../../introduction/concepts/connector-v2-features.md)
+- [x] [支持多表写入](../../introduction/concepts/connector-v2-features.md)
+- [ ] [定时刷新](../../introduction/concepts/connector-v2-features.md)
## 描述
@@ -37,19 +38,21 @@ import ChangeLog from '../changelog/connector-http.md';
| 名称 | 类型 | 是否必须 | 默认值 |
描述 |
|-----------------------------|--------|------|-------|------------------------------------------------------------|
| url | String | 是 | - | Http 请求链接
|
-| headers | Map | 否 | - | Http 标头
|
+| headers | Map | 否 | - | Http 请求头
|
+| params | Map | 否 | - | 该参数会通过参数校验。当前 Sink
写入器会把数据作为请求体 POST 到最终 URL,如需查询参数,建议直接写在 `url` 中。 |
| retry | Int | 否 | - |
如果请求http返回`IOException`的最大重试次数 |
| retry_backoff_multiplier_ms | Int | 否 | 100 | http请求失败,重试回退次数(毫秒)乘数
|
| retry_backoff_max_ms | Int | 否 | 10000 | http请求失败,最大重试回退时间(毫秒)
|
-| connect_timeout_ms | Int | 否 | 12000 | 连接超时设置,默认12s
|
-| socket_timeout_ms | Int | 否 | 60000 | 套接字超时设置,默认为60s
|
| array_mode | Boolean| 否 | false |
为true时将数据作为JSON数组发送,为false时作为单个JSON对象发送(默认) |
| batch_size | Int | 否 | 1 |
在一个HTTP请求中发送的记录批量大小。仅在array_mode为true时有效 |
| request_interval_ms | Int | 否 | 0 |
两次HTTP请求之间的间隔毫秒数,以避免请求过于频繁 |
+| multi_table_sink_replica | Int | 否 | - | 多表写入时使用的 Sink
副本数,详情请参考 [Sink 常用选项](../common-options/sink-common-options.md)。 |
| common-options | | 否 | - | Sink插件常用参数,请参考
[Sink常用选项 ](../common-options/sink-common-options.md) 了解详情 |
## 示例
+Http Sink 固定发送 `POST` 请求。每条上游数据会被转换成 JSON 作为请求体;当 `array_mode = true`
时,会先把多条数据攒成 JSON 数组再发送,`batch_size` 控制单次请求最多包含多少条数据。
+
简单示例:
```hocon
diff --git a/docs/zh/connectors/sink/Kudu.md b/docs/zh/connectors/sink/Kudu.md
index 5db7ffb262..18b6a0cb1d 100644
--- a/docs/zh/connectors/sink/Kudu.md
+++ b/docs/zh/connectors/sink/Kudu.md
@@ -2,9 +2,9 @@ import ChangeLog from '../changelog/connector-kudu.md';
# Kudu
-> Kudu数据接收器
+> Kudu Sink 连接器
-## 支持Kudu版本
+## 支持 Kudu 版本
- 1.11.1/1.12.0/1.13.0/1.14.0/1.15.0
@@ -16,10 +16,10 @@ import ChangeLog from '../changelog/connector-kudu.md';
## 主要特性
-- [ ] [exactly-once](../../introduction/concepts/connector-v2-features.md)
-- [x] [cdc](../../introduction/concepts/connector-v2-features.md)
-- [x] [support multiple table
write](../../introduction/concepts/connector-v2-features.md)
-- [ ] [timer flush](../../introduction/concepts/connector-v2-features.md)
+- [ ] [精确一次](../../introduction/concepts/connector-v2-features.md)
+- [x] [变更数据捕获](../../introduction/concepts/connector-v2-features.md)
+- [x] [支持多表写入](../../introduction/concepts/connector-v2-features.md)
+- [ ] [定时刷新](../../introduction/concepts/connector-v2-features.md)
## 数据类型映射
@@ -40,7 +40,7 @@ import ChangeLog from '../changelog/connector-kudu.md';
| 名称 | 类型 | 是否必填 |
默认值 |
描述 |
|-------------------------------------------|--------|----------|------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|
| kudu_masters | String | 是 | -
| Kudu主地址。用“,”分隔,例如“192.168.88.110:7051”。
|
-| table_name | String | 是 | -
| Kudu表的名字。
|
+| table_name | String | 否 | 上游表名
| Kudu 表名。不配置时,SeaTunnel 使用上游数据携带的表名。多表写入时可使用
`${database_name}`、`${schema_name}`、`${table_name}` 等占位符。 |
| client_worker_count | Int | 否 | 2 *
Runtime.getRuntime().availableProcessors() | Kudu工人数。默认值是当前cpu核数的两倍。
|
| client_default_operation_timeout_ms | Long | 否 | 30000
| Kudu正常运行超时。
|
| client_default_admin_operation_timeout_ms | Long | 否 | 30000
| Kudu管理员操作超时。
|
@@ -48,14 +48,20 @@ import ChangeLog from '../changelog/connector-kudu.md';
| kerberos_principal | String | 否 | -
| Kerberos主体。请注意,所有zeta节点都需要此文件。
|
| kerberos_keytab | String | 否 | -
| Kerberos密钥表。请注意,所有zeta节点都需要此文件。
|
| kerberos_krb5conf | String | 否 | -
| Kerberos krb5 conf.请注意,所有zeta节点都需要此文件。
|
-| save_mode | String | 否 | APPEND
| 存储模式,支持 `overwrite` 和 `append`.
|
-| session_flush_mode | String | 否 |
AUTO_FLUSH_SYNC | Kudu刷新模式。默认AUTO_FLUSH_SYNC。
|
-| batch_size | Int | 否 | 1024
| 超过此记录数的刷新最大大小(包括所有追加、追加和删除记录)将刷新数据。默认值为1024 |
-| buffer_flush_interval | Int | 否 | 10000
| 刷新间隔期间,异步线程将刷新数据。
|
+| save_mode | String | 否 | APPEND
| 存储模式。支持 `append` 和 `overwrite`。在 `overwrite`
模式下,插入记录会按 Kudu upsert 写入。
|
+| session_flush_mode | String | 否 |
AUTO_FLUSH_SYNC | Kudu 刷新模式。支持
`AUTO_FLUSH_SYNC`、`AUTO_FLUSH_BACKGROUND` 和 `MANUAL_FLUSH`。
|
+| batch_size | Int | 否 | 1024
| 仅当 `session_flush_mode` 为
`AUTO_FLUSH_BACKGROUND` 或 `MANUAL_FLUSH` 时需要关注。写入的 append、upsert、delete
记录达到该数量后会刷新。 |
+| buffer_flush_interval | Int | 否 | 10000
| 仅当 `session_flush_mode =
AUTO_FLUSH_BACKGROUND` 时生效,表示异步写入的刷新间隔,单位毫秒。
|
| ignore_not_found | Bool | 否 | false
| 如果为true,则忽略所有未找到的行。
|
-| ignore_not_duplicate | Bool | 否 | false
| 如果为true,则忽略所有dulicate行。
|
+| ignore_not_duplicate | Bool | 否 | false
| 如果为 true,则忽略所有重复行。
|
| common-options | | 否 | -
| Sink插件常用参数,详见[Sink common
Options](../common-options/sink-common-options.md)。 |
+## 参数说明
+
+- `table_name` 是可选参数。不配置时,Sink 会写入上游数据行携带的表名。
+- 多表写入时,可以在 `table_name` 中使用占位符,把不同来源表的数据写入不同 Kudu 表。
+- 支持 CDC 数据:插入记录会追加写入,更新记录会按 upsert 写入,删除记录会按主键删除。
+
## 任务示例
### 简单示例
diff --git a/docs/zh/connectors/sink/RocketMQ.md
b/docs/zh/connectors/sink/RocketMQ.md
index dce3ed440a..1e1abe912c 100644
--- a/docs/zh/connectors/sink/RocketMQ.md
+++ b/docs/zh/connectors/sink/RocketMQ.md
@@ -59,6 +59,8 @@ partition.key.fields = ["c_int"]
Sink 支持通过 RocketMQ 事务消息实现精确一次写入。该能力默认关闭。确认 RocketMQ 集群和作业 checkpoint
配置满足事务写入要求后,可设置 `exactly.once = true`。
+当 `format = text` 时,SeaTunnel 会按上游 schema 的字段顺序序列化,并使用 `field.delimiter`
拼接字段。当 `format = json` 时,每行数据会写成一个 JSON 对象。
+
## 任务示例
### 写入 JSON 消息
diff --git a/docs/zh/connectors/source/Http.md
b/docs/zh/connectors/source/Http.md
index 3ff91465b4..1d00d693d3 100644
--- a/docs/zh/connectors/source/Http.md
+++ b/docs/zh/connectors/source/Http.md
@@ -41,10 +41,10 @@ import ChangeLog from '../changelog/connector-http.md';
| schema.fields | Config | 否 | - | 上游数据的
schema 字段
|
| json_field | Config | 否 | - | 此参数帮助您配置
schema,因此此参数必须与 schema 一起使用。
|
| pageing | Config | 否 | - | 此参数用于分页查询
|
-| pageing.page_field | String | 否 | - |
此参数用于指定请求中的页面字段名称。它可以在 headers、params 或 body 中使用占位符,如 ${page_field}。
|
+| pageing.page_field | String | 否 | page |
此参数用于指定请求中的分页字段名。它可以在 headers、params 或 body 中使用占位符,例如 `${page}`。
|
| pageing.use_placeholder_replacement | Boolean | 否 | false | 如果为
true,则使用占位符替换(${field})用于 headers、parameters 和 body 值,否则使用基于键的替换。
|
-| pageing.total_page_size | Int | 否 | - | 此参数用于控制总页数
|
-| pageing.batch_size | Int | 否 | - |
每个请求返回的批量大小,用于在总页数未知时确定是否继续
|
+| pageing.total_page_size | Long | 否 | 0 |
用于控制总页数。`0` 表示总页数未知,连接器会在返回行数小于 `pageing.batch_size` 时停止。
|
+| pageing.batch_size | Int | 否 | 100 |
每个请求返回的批量大小,用于在总页数未知时判断是否继续。
|
| pageing.start_page_number | Int | 否 | 1 | 指定同步开始的页码
|
| pageing.page_type | String | 否 | PageNumber |
此参数用于指定页面类型,如果未设置则为 PageNumber,仅支持 `PageNumber` 和 `Cursor`。
|
| pageing.cursor_field | String | 否 | - |
此参数用于指定请求参数中的游标字段名称。
|
@@ -68,6 +68,12 @@ import ChangeLog from '../changelog/connector-http.md';
| keep_page_param_as_http_param | Boolean | 否 | false | 是否将分页参数设置为
params。用于兼容旧行为。
|
| json_filed_missed_return_null | Boolean | 否 | false | 当 JSON
字段缺失时,设置为 true 并返回 null,否则返回错误。|
+## 参数说明
+
+- `pageing` 是连接器现有参数名,作业配置中需要保持这个拼写。
+- `format = json` 通常需要配置 `schema`。当字段来自嵌套 JSON 路径时,可以配合 `json_field`
使用;如果想直接抽取某个 JSON 数组或对象片段,可以使用 `content_field`。
+- `format = binary` 会输出固定字段 `(data: bytes, relativePath: string, partIndex:
long)`,只支持批模式,且不能和 `pageing` 一起使用。
+
## 如何创建 Http 数据同步作业
```hocon
diff --git a/docs/zh/connectors/source/Kudu.md
b/docs/zh/connectors/source/Kudu.md
index 957609f634..8f25b9f191 100644
--- a/docs/zh/connectors/source/Kudu.md
+++ b/docs/zh/connectors/source/Kudu.md
@@ -47,7 +47,7 @@ import ChangeLog from '../changelog/connector-kudu.md';
| 参数名 | 类型 | 必须 | 默认值
| 描述
|
|-------------------------------------------|--------|----|------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| kudu_masters | String | 是 | -
| Kudu master 地址。用 ',' 分隔,例如 '192.168.88.110:7051'。
|
-| table_name | String | 是 | -
| Kudu 表的名称。
|
+| table_name | String | 当未配置 `table_list` 时必填 |
- | Kudu 表名。该参数与 `table_list` 二选一。
|
| client_worker_count | Int | 否 | 2 *
Runtime.getRuntime().availableProcessors() | Kudu worker 数量。默认值是当前 CPU 核心数的两倍。
|
| client_default_operation_timeout_ms | Long | 否 | 30000
| Kudu 普通操作超时时间。
|
| client_default_admin_operation_timeout_ms | Long | 否 | 30000
| Kudu 管理操作超时时间。
|
@@ -59,10 +59,16 @@ import ChangeLog from '../changelog/connector-kudu.md';
| scan_token_batch_size_bytes | Int | 否 | 1024 * 1024
| Kudu 扫描字节数。一次读取的最大字节数,默认为 1MB。
|
| use_regex | Bool | 否 | false
| 控制 `table_name` 的正则匹配。当设置为 `true` 时,`table_name`
将被视为正则表达式模式,可以匹配多张表。当设置为 `false` 或未指定时,`table_name` 将被视为精确表名(不进行正则匹配)。
|
| filter | String | 否 | -
| Kudu 扫描过滤表达式,例如 id > 100 AND id < 200。
|
-| schema | Map | 否 | 1024 * 1024
| SeaTunnel Schema。更多详情请参考 [Schema
特性](../../introduction/concepts/schema-feature.md)。
|
-| table_list | Array | 否 | -
| 要读取的表列表。您可以使用此配置代替 `table_name`,例如:```table_list
= [{ table_name = "kudu_source_table_1"},{ table_name = "kudu_source_table_2"}]
```。也可以在每个 entry 中配置 `use_regex = true` 来对 `table_name` 启用正则匹配。 |
+| schema | Map | 否 | -
| SeaTunnel Schema。更多详情请参考 [Schema
特性](../../introduction/concepts/schema-feature.md)。
|
+| table_list | Array | 否 | -
| 要读取的表列表。可以使用该参数代替 `table_name`,例如:```table_list =
[{ table_name = "kudu_source_table_1"},{ table_name = "kudu_source_table_2"}]
```。也可以在每个条目中配置 `use_regex = true`,让该条目的 `table_name` 按正则匹配。 |
| common-options | | 否 | -
| 源插件通用参数,请参考
[源通用选项](../common-options/source-common-options.md) 详见。
|
+## 参数说明
+
+- `table_name` 和 `table_list` 只能配置其中一个。
+- `filter` 会下推到 Kudu 扫描中,可以使用类似 `id >= 1 AND id <= 2` 的 Kudu 过滤表达式。
+- `use_regex = true` 会把 `table_name` 当作 Java 正则表达式处理,可以配置在顶层,也可以配置在
`table_list` 的每个条目中。
+
## 任务示例
### 简单
@@ -217,4 +223,3 @@ source {
## 变更日志
<ChangeLog />
-
diff --git a/docs/zh/connectors/source/RocketMQ.md
b/docs/zh/connectors/source/RocketMQ.md
index 097ff56266..b21f7565e8 100644
--- a/docs/zh/connectors/source/RocketMQ.md
+++ b/docs/zh/connectors/source/RocketMQ.md
@@ -76,6 +76,8 @@ start.mode.offsets = {
当不同 topic 的字段结构不一样时,使用 `tables_configs`。每一项都可以单独配置
`topics`、`schema`、`format`、`tags` 和启动消费位置。如果没有配置 `schema.table`,输出表名默认使用 topic
名称。
+`topics`、`tables_configs` 和已废弃的 `table_list` 互斥,只能配置其中一个。在 `tables_configs`
中,单个条目未配置的参数会沿用顶层默认值,因此每个条目只需要覆盖该 topic 特有的 schema、tag 或启动位置。
+
## 任务示例
### 读取 JSON 消息