This is an automated email from the ASF dual-hosted git repository.
davidzollo pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/seatunnel.git
The following commit(s) were added to refs/heads/dev by this push:
new 72f316d7bb [Docs][Connector-V2] Improve Aerospike DynamoDB and SQS
docs (#11336)
72f316d7bb is described below
commit 72f316d7bbcc16a8d5a19ad572d6795397b7f9ea
Author: Daniel Carter <[email protected]>
AuthorDate: Sat Jul 4 21:55:43 2026 +0800
[Docs][Connector-V2] Improve Aerospike DynamoDB and SQS docs (#11336)
Co-authored-by: DanielCarter-stack
<[email protected]>
---
docs/en/connectors/sink/Aerospike.md | 10 +++++-----
docs/en/connectors/sink/AmazonDynamoDB.md | 27 ++++++++++++++-------------
docs/en/connectors/sink/AmazonSqs.md | 2 ++
docs/en/connectors/source/AmazonDynamoDB.md | 24 +++++++++++++-----------
docs/en/connectors/source/AmazonSqs.md | 2 ++
docs/zh/connectors/sink/Aerospike.md | 10 +++++-----
docs/zh/connectors/sink/AmazonDynamoDB.md | 27 ++++++++++++++-------------
docs/zh/connectors/sink/AmazonSqs.md | 2 ++
docs/zh/connectors/source/AmazonDynamoDB.md | 24 +++++++++++++-----------
docs/zh/connectors/source/AmazonSqs.md | 2 ++
10 files changed, 72 insertions(+), 58 deletions(-)
diff --git a/docs/en/connectors/sink/Aerospike.md
b/docs/en/connectors/sink/Aerospike.md
index 4ee54df2e2..5b43eed5f4 100644
--- a/docs/en/connectors/sink/Aerospike.md
+++ b/docs/en/connectors/sink/Aerospike.md
@@ -44,14 +44,13 @@ The connector writes to one fixed target set. It does not
route records to diffe
| DOUBLE | DOUBLE | 64-bit floating point
|
| BOOLEAN | BOOLEAN | Stored as true/false values
|
| ARRAY | BYTEARRAY | Only support byte array type
|
-| LIST | LIST | Support generic list types
|
| DATE | LONG | Converted to epoch milliseconds
|
| TIMESTAMP | LONG | Converted to epoch milliseconds
|
Note:
-- When using ARRAY type, SeaTunnel's array elements must be byte type
-- LIST type supports any element types that can be serialized
-- DATE/TIMESTAMP conversion uses system default time zone
+- When using `ARRAY`, SeaTunnel's array elements must be byte type.
+- `LIST` is not inferred automatically from a SeaTunnel field type. Use
`schema.field` to explicitly map a field to `LIST` when the incoming value is
iterable.
+- `DATE` and `TIMESTAMP` conversion uses the system default time zone.
## Options
@@ -76,10 +75,11 @@ Note:
- **kv**: Store each non-key field as a separate bin. `bin_name` is not used
When `data_format` is `map` or `string`, configure `bin_name`; otherwise the
writer does not know which Aerospike bin should receive the packed row data.
+The `key` field must be present in the input schema because it is used as the
Aerospike record key for every write.
### schema.field
-`schema.field` is optional for `map` and `string` formats. Configure it when
you need to control the Aerospike type used for each field.
+`schema.field` is optional for `map` and `string` formats. If it is omitted,
the connector writes all input fields and maps the SeaTunnel field types
automatically. Configure it when you need to control the Aerospike type used
for each field.
For `kv` format, configure `schema.field` for the fields you want to write as
independent Aerospike bins. The writer iterates over `schema.field`, so fields
not listed there are not written in `kv` mode.
diff --git a/docs/en/connectors/sink/AmazonDynamoDB.md
b/docs/en/connectors/sink/AmazonDynamoDB.md
index d376c64419..ad1e626932 100644
--- a/docs/en/connectors/sink/AmazonDynamoDB.md
+++ b/docs/en/connectors/sink/AmazonDynamoDB.md
@@ -18,19 +18,19 @@ The target table must already exist. The connector writes
each row as a DynamoDB
## Options
-| name | type | required | default value |
-|---------------------|--------|----------|---------------|
-| url | string | yes | - |
-| region | string | yes | - |
-| access_key_id | string | yes | - |
-| secret_access_key | string | yes | - |
-| table | string | yes | - |
-| batch_size | int | no | 25 |
-| multi_table_sink_replica | int | no | - |
-| max_retries | int | no | 10 |
-| retry_base_delay_ms | long | no | 100 |
-| retry_max_delay_ms | long | no | 5000 |
-| common-options | | no | - |
+| name | type | required | default value | description
|
+|---------------------|--------|----------|---------------|--------------------------------------------------|
+| url | string | yes | - | DynamoDB endpoint
URL. |
+| region | string | yes | - | AWS region of the
DynamoDB service. |
+| access_key_id | string | yes | - | AWS access key ID.
|
+| secret_access_key | string | yes | - | AWS secret access
key. |
+| table | string | yes | - | DynamoDB table
name to write to. |
+| batch_size | int | no | 25 | Records buffered
for one batch write request. |
+| multi_table_sink_replica | int | no | - | Sink writer
replicas for each table. |
+| max_retries | int | no | 10 | Retries for
unprocessed items. |
+| retry_base_delay_ms | long | no | 100 | Initial retry
backoff delay in milliseconds. |
+| retry_max_delay_ms | long | no | 5000 | Maximum retry
backoff delay in milliseconds. |
+| common-options | object | no | - | Sink plugin common
parameters. |
### url [string]
@@ -61,6 +61,7 @@ For a normal single-table job, set this to the target table
name. In a multi-tab
The number of records buffered for one DynamoDB batch write request.
DynamoDB batch write supports up to 25 write requests per call, so the default
is `25`.
+Do not set this higher than `25`; larger values do not match the DynamoDB
batch write API limit.
### multi_table_sink_replica [int]
diff --git a/docs/en/connectors/sink/AmazonSqs.md
b/docs/en/connectors/sink/AmazonSqs.md
index 6e7af6bbba..e8b4de683b 100644
--- a/docs/en/connectors/sink/AmazonSqs.md
+++ b/docs/en/connectors/sink/AmazonSqs.md
@@ -36,6 +36,8 @@ is serialized by the configured `format`, and the serialized
value is sent as th
| field_delimiter | String | No | , | Field delimiter used when
`format = text`.
|
| common-options | | No | - | Sink plugin common
parameters. For details, see [Sink Common
Options](../common-options/sink-common-options.md).
|
+`url` can point to AWS SQS or to an SQS-compatible local service, for example
`http://sqs-host:4566/000000000000/sink_queue`.
+
## Format Notes
- `json` writes each row as a JSON object.
diff --git a/docs/en/connectors/source/AmazonDynamoDB.md
b/docs/en/connectors/source/AmazonDynamoDB.md
index 7edc464f8c..835ac9b52e 100644
--- a/docs/en/connectors/source/AmazonDynamoDB.md
+++ b/docs/en/connectors/source/AmazonDynamoDB.md
@@ -23,17 +23,17 @@ This source reads the current table data with scan
requests. It does not read Dy
## Options
-| name | type | required | default value |
-|-----------------------|--------|----------|---------------|
-| url | string | yes | - |
-| region | string | yes | - |
-| access_key_id | string | yes | - |
-| secret_access_key | string | yes | - |
-| table | string | yes | - |
-| schema | config | yes | - |
-| scan_item_limit | int | no | 1 |
-| parallel_scan_threads | int | no | 2 |
-| common-options | | no | - |
+| name | type | required | default value | description
|
+|-----------------------|--------|----------|---------------|--------------------------------------------------|
+| url | string | yes | - | DynamoDB
endpoint URL. |
+| region | string | yes | - | AWS region of
the DynamoDB service. |
+| access_key_id | string | yes | - | AWS access key
ID. |
+| secret_access_key | string | yes | - | AWS secret
access key. |
+| table | string | yes | - | DynamoDB table
name to scan. |
+| schema | config | yes | - | SeaTunnel fields
to read from DynamoDB items. |
+| scan_item_limit | int | no | 1 | Maximum items
returned by each scan request. |
+| parallel_scan_threads | int | no | 2 | Number of
logical segments for parallel scan. |
+| common-options | object | no | - | Source plugin
common parameters. |
### url [string]
@@ -89,6 +89,8 @@ For more schema syntax, see [Schema
Feature](../../introduction/concepts/schema-
The maximum number of items returned by each DynamoDB scan request.
+Larger values reduce the number of requests but may increase the memory used
by each read batch.
+
### parallel_scan_threads [int]
The number of logical scan segments used for DynamoDB parallel scan.
diff --git a/docs/en/connectors/source/AmazonSqs.md
b/docs/en/connectors/source/AmazonSqs.md
index 45dd32fe1d..6c6648acbf 100644
--- a/docs/en/connectors/source/AmazonSqs.md
+++ b/docs/en/connectors/source/AmazonSqs.md
@@ -45,6 +45,8 @@ Each receive request asks SQS for up to 10 messages. If more
messages are waitin
| debezium_record_include_schema | Boolean | No | true | Whether
Debezium JSON messages include a schema. This option is used only when `format
= debezium_json`. |
| common-options | | No | - | Source
plugin common parameters. For details, see [Source Common
Options](../common-options/source-common-options.md).
|
+`url` can point to AWS SQS or to an SQS-compatible local service, for example
`http://sqs-host:4566/000000000000/source_queue`.
+
## Format Notes
- `json` reads each message body as a JSON object that matches `schema`.
diff --git a/docs/zh/connectors/sink/Aerospike.md
b/docs/zh/connectors/sink/Aerospike.md
index 2b8088656d..e2aaf5fd2e 100644
--- a/docs/zh/connectors/sink/Aerospike.md
+++ b/docs/zh/connectors/sink/Aerospike.md
@@ -44,14 +44,13 @@ import ChangeLog from '../changelog/connector-aerospike.md';
| DOUBLE | DOUBLE | 64位浮点数
|
| BOOLEAN | BOOLEAN | 存储为 true/false 值
|
| ARRAY | BYTEARRAY | 仅支持字节数组类型
|
-| LIST | LIST | 支持泛型列表类型
|
| DATE | LONG | 转换为纪元时间毫秒数
|
| TIMESTAMP | LONG | 转换为纪元时间毫秒数
|
注意事项：
-- 使用ARRAY类型时，SeaTunnel数组元素必须是byte类型
-- LIST类型支持可序列化的任意元素类型
-- DATE/TIMESTAMP转换使用系统默认时区
+- 使用 `ARRAY` 类型时，SeaTunnel 数组元素必须是 byte 类型。
+- `LIST` 不会从 SeaTunnel 字段类型自动推断。只有在 `schema.field` 中把字段显式映射为
`LIST`，且输入值是可遍历对象时才适用。
+- `DATE` 和 `TIMESTAMP` 转换使用系统默认时区。
## 配置选项
@@ -76,10 +75,11 @@ import ChangeLog from '../changelog/connector-aerospike.md';
- **kv**: 每个非主键字段存储为独立的 bin，此时不使用 `bin_name`
当 `data_format` 为 `map` 或 `string` 时，必须配置 `bin_name`，否则写入器无法判断打包后的整行数据应该写入哪个
Aerospike bin。
+`key` 指定的字段必须存在于输入 schema 中，因为每次写入都会用它作为 Aerospike 记录主键。
### schema.field 配置说明
-`schema.field` 对 `map` 和 `string` 格式是可选配置。需要明确控制每个字段写入 Aerospike 时的类型时再配置。
+`schema.field` 对 `map` 和 `string` 格式是可选配置。不配置时，连接器会写入所有输入字段，并根据 SeaTunnel
字段类型自动映射。需要明确控制每个字段写入 Aerospike 时的类型时再配置。
当 `data_format` 为 `kv` 时，请在 `schema.field` 中列出需要写成独立 Aerospike bin 的字段。写入器会遍历
`schema.field`，未列出的字段不会在 `kv` 模式下写入。
diff --git a/docs/zh/connectors/sink/AmazonDynamoDB.md
b/docs/zh/connectors/sink/AmazonDynamoDB.md
index c222c1139d..c8e15f55df 100644
--- a/docs/zh/connectors/sink/AmazonDynamoDB.md
+++ b/docs/zh/connectors/sink/AmazonDynamoDB.md
@@ -18,19 +18,19 @@ Amazon DynamoDB Sink 连接器用于将 SeaTunnel 数据行写入 DynamoDB 表
## 选项
-| 名称 | 类型 | 必填 | 默认值 |
-|---------------------|--------|------|--------|
-| url | string | 是 | - |
-| region | string | 是 | - |
-| access_key_id | string | 是 | - |
-| secret_access_key | string | 是 | - |
-| table | string | 是 | - |
-| batch_size | int | 否 | 25 |
-| multi_table_sink_replica | int | 否 | - |
-| max_retries | int | 否 | 10 |
-| retry_base_delay_ms | long | 否 | 100 |
-| retry_max_delay_ms | long | 否 | 5000 |
-| common-options | | 否 | - |
+| 名称 | 类型 | 必填 | 默认值 | 说明 |
+|---------------------|--------|------|--------|------------------------------|
+| url | string | 是 | - | DynamoDB 服务地址。 |
+| region | string | 是 | - | DynamoDB 所在的 AWS 区域。 |
+| access_key_id | string | 是 | - | AWS access key ID。 |
+| secret_access_key | string | 是 | - | AWS secret access key。 |
+| table | string | 是 | - | 要写入的 DynamoDB 表名。 |
+| batch_size | int | 否 | 25 | 一次批量写入请求缓存的记录数。 |
+| multi_table_sink_replica | int | 否 | - | 每张表对应的 Sink Writer 副本数。 |
+| max_retries | int | 否 | 10 | 未处理 item 的最大重试次数。 |
+| retry_base_delay_ms | long | 否 | 100 | 初始重试等待时间，单位毫秒。 |
+| retry_max_delay_ms | long | 否 | 5000 | 最大重试等待时间，单位毫秒。 |
+| common-options | object | 否 | - | Sink 插件通用参数。 |
### url [string]
@@ -61,6 +61,7 @@ DynamoDB 所在的 AWS 区域，例如 `us-east-1`。
一次 DynamoDB 批量写入请求缓存的记录数。
DynamoDB batch write 每次最多支持 25 条写请求，所以默认值为 `25`。
+不要把该值设置为大于 `25`，否则会超过 DynamoDB batch write API 的限制。
### multi_table_sink_replica [int]
diff --git a/docs/zh/connectors/sink/AmazonSqs.md
b/docs/zh/connectors/sink/AmazonSqs.md
index 533cf5f73f..f0e65ad6f6 100644
--- a/docs/zh/connectors/sink/AmazonSqs.md
+++ b/docs/zh/connectors/sink/AmazonSqs.md
@@ -36,6 +36,8 @@ Amazon SQS Sink 连接器用于把每条输入的 SeaTunnel 行数据写入一
| field_delimiter | String | 否 | , | 当 `format = text` 时使用的字段分隔符。
|
| common-options | | 否 | - | Sink 插件通用参数，详见 [Sink Common
Options](../common-options/sink-common-options.md)。
|
+`url` 可以指向 AWS SQS，也可以指向兼容 SQS 的本地服务，例如
`http://sqs-host:4566/000000000000/sink_queue`。
+
## 格式说明
- `json`：把每行数据写成 JSON 对象。
diff --git a/docs/zh/connectors/source/AmazonDynamoDB.md
b/docs/zh/connectors/source/AmazonDynamoDB.md
index c3e6dbab3f..771bcaf01e 100644
--- a/docs/zh/connectors/source/AmazonDynamoDB.md
+++ b/docs/zh/connectors/source/AmazonDynamoDB.md
@@ -23,17 +23,17 @@ Amazon DynamoDB 源连接器通过 DynamoDB scan 请求读取已有表中的数
## 选项
-| 名称 | 类型 | 必填 | 默认值 |
-|-----------------------|--------|------|--------|
-| url | string | 是 | - |
-| region | string | 是 | - |
-| access_key_id | string | 是 | - |
-| secret_access_key | string | 是 | - |
-| table | string | 是 | - |
-| schema | config | 是 | - |
-| scan_item_limit | int | 否 | 1 |
-| parallel_scan_threads | int | 否 | 2 |
-| common-options | | 否 | - |
+| 名称 | 类型 | 必填 | 默认值 | 说明 |
+|-----------------------|--------|------|--------|----------------------------|
+| url | string | 是 | - | DynamoDB 服务地址。 |
+| region | string | 是 | - | DynamoDB 所在的 AWS 区域。 |
+| access_key_id | string | 是 | - | AWS access key ID。 |
+| secret_access_key | string | 是 | - | AWS secret access key。 |
+| table | string | 是 | - | 要扫描的 DynamoDB 表名。 |
+| schema | config | 是 | - | 要读取的 SeaTunnel 字段。 |
+| scan_item_limit | int | 否 | 1 | 每次 scan 请求返回的最大 item 数。 |
+| parallel_scan_threads | int | 否 | 2 | parallel scan 的逻辑分片数。 |
+| common-options | object | 否 | - | Source 插件通用参数。 |
### url [string]
@@ -89,6 +89,8 @@ schema = {
每次 DynamoDB scan 请求最多返回的 item 数量。
+较大的值可以减少请求次数，但也会增加每个读取批次占用的内存。
+
### parallel_scan_threads [int]
DynamoDB parallel scan 使用的逻辑分片数量。
diff --git a/docs/zh/connectors/source/AmazonSqs.md
b/docs/zh/connectors/source/AmazonSqs.md
index d10afd9ab9..ff95bba747 100644
--- a/docs/zh/connectors/source/AmazonSqs.md
+++ b/docs/zh/connectors/source/AmazonSqs.md
@@ -44,6 +44,8 @@ Amazon SQS 源连接器用于从一个 Amazon SQS 队列 URL 读取消息。连
| debezium_record_include_schema | Boolean | 否 | true | Debezium JSON
消息是否包含 schema。仅在 `format = debezium_json` 时使用。
|
| common-options | | 否 | - | 源插件通用参数，详见 [Source
Common Options](../common-options/source-common-options.md)。
|
+`url` 可以指向 AWS SQS，也可以指向兼容 SQS 的本地服务，例如
`http://sqs-host:4566/000000000000/source_queue`。
+
## 格式说明
- `json`：把每条消息体按 JSON 对象解析，并要求字段能对应到 `schema`。
