This is an automated email from the ASF dual-hosted git repository.
zhangshenghang pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/seatunnel.git
The following commit(s) were added to refs/heads/dev by this push:
new ff91889978 [Docs][Connector-V2] Improve Aerospike DynamoDB and SQS
docs (#11343)
ff91889978 is described below
commit ff918899784265b008f3046661c0898362edd247
Author: Daniel Carter <[email protected]>
AuthorDate: Mon Jul 6 22:33:01 2026 +0800
[Docs][Connector-V2] Improve Aerospike DynamoDB and SQS docs (#11343)
Co-authored-by: DanielCarter-stack
<[email protected]>
---
docs/en/connectors/sink/Aerospike.md | 13 +++++++--
docs/en/connectors/sink/AmazonDynamoDB.md | 13 +++++++++
docs/en/connectors/sink/AmazonSqs.md | 33 ++++++++++++++++++++++
docs/en/connectors/source/AmazonDynamoDB.md | 13 +++++++++
docs/en/connectors/source/AmazonSqs.md | 33 ++++++++++++++++++++++
docs/zh/connectors/sink/Aerospike.md | 11 +++++++-
docs/zh/connectors/sink/AmazonDynamoDB.md | 17 ++++++++++--
docs/zh/connectors/sink/AmazonSqs.md | 43 +++++++++++++++++++++++++----
docs/zh/connectors/source/AmazonDynamoDB.md | 17 ++++++++++--
docs/zh/connectors/source/AmazonSqs.md | 33 ++++++++++++++++++++++
10 files changed, 214 insertions(+), 12 deletions(-)
diff --git a/docs/en/connectors/sink/Aerospike.md
b/docs/en/connectors/sink/Aerospike.md
index 5b43eed5f4..6442da5356 100644
--- a/docs/en/connectors/sink/Aerospike.md
+++ b/docs/en/connectors/sink/Aerospike.md
@@ -4,11 +4,11 @@ import ChangeLog from '../changelog/connector-aerospike.md';
> Aerospike sink connector
-## Support Those Engines
+## Supported Engines
> Spark<br/>
> Flink<br/>
-> Seatunnel Zeta<br/>
+> SeaTunnel Zeta<br/>
## License Compatibility Notice
@@ -27,6 +27,7 @@ When using this connector, you need to comply with AGPL 3.0
license terms.
Sink connector for Aerospike database. The connector writes records to one
Aerospike namespace and set. It uses the configured `key` field as the
Aerospike record key.
The connector writes to one fixed target set. It does not route records to
different Aerospike sets by table name.
+When a record with the same Aerospike key already exists, the connector
updates that record's bins.
## Supported DataSource Info
@@ -85,6 +86,14 @@ For `kv` format, configure `schema.field` for the fields you
want to write as in
Supported Aerospike type names include `STRING`, `INTEGER`, `LONG`, `DOUBLE`,
`BOOLEAN`, `BYTEARRAY`, and `LIST`.
+## Usage Notes
+
+- `key` must name an existing input field. The field value is converted to a
string and used as the Aerospike record key.
+- `data_format = "string"` stores the selected fields as one JSON string in
`bin_name`.
+- `data_format = "map"` stores the selected fields as one Aerospike map in
`bin_name`.
+- `data_format = "kv"` writes each configured field as a separate Aerospike
bin and ignores `bin_name`.
+- Empty `username` and `password` values are only suitable when Aerospike
authentication is disabled.
+
## Task Example
### Write FakeSource Data To Aerospike
diff --git a/docs/en/connectors/sink/AmazonDynamoDB.md
b/docs/en/connectors/sink/AmazonDynamoDB.md
index ad1e626932..8567832c4d 100644
--- a/docs/en/connectors/sink/AmazonDynamoDB.md
+++ b/docs/en/connectors/sink/AmazonDynamoDB.md
@@ -10,6 +10,12 @@ The Amazon DynamoDB sink connector writes SeaTunnel rows to
a DynamoDB table.
The target table must already exist. The connector writes each row as a
DynamoDB item and uses batch write requests. It supports single-table writes
and multi-table writes when the upstream row carries a table id.
+## Supported Engines
+
+> Spark<br/>
+> Flink<br/>
+> SeaTunnel Zeta<br/>
+
## Key Features
- [ ] [exactly-once](../../introduction/concepts/connector-v2-features.md)
@@ -83,6 +89,13 @@ The maximum delay, in milliseconds, between retries.
Sink plugin common parameters, please refer to [Sink Common
Options](../common-options/sink-common-options.md) for details.
+## Usage Notes
+
+- Create the target DynamoDB table before starting the SeaTunnel job. The sink
does not create tables or key schemas.
+- `access_key_id` and `secret_access_key` are required by this connector. For
DynamoDB Local, use dummy values accepted by the local service.
+- DynamoDB accepts at most 25 write requests in one batch write call, so keep
`batch_size` at or below `25`.
+- The sink retries unprocessed items with exponential backoff. It does not
provide exactly-once guarantees.
+
## Data Type Mapping
| SeaTunnel Data Type | DynamoDB Attribute Type |
diff --git a/docs/en/connectors/sink/AmazonSqs.md
b/docs/en/connectors/sink/AmazonSqs.md
index e8b4de683b..3aa659fb64 100644
--- a/docs/en/connectors/sink/AmazonSqs.md
+++ b/docs/en/connectors/sink/AmazonSqs.md
@@ -46,9 +46,42 @@ is serialized by the configured `format`, and the serialized
value is sent as th
- `debezium_json` writes Debezium JSON messages. For details, see [Debezium
JSON](../formats/debezium-json.md).
- The sink sends only the message body. It does not expose SQS message
attributes, delay seconds, deduplication ID, or message group ID options.
- `access_key_id` and `secret_access_key` are optional, but they must be
configured together when static AWS credentials are used.
+- The sink sends each SeaTunnel row as one SQS message. It does not batch
multiple rows into one SQS request.
## Task Examples
+### Copy Messages Between Local-Compatible Queues
+
+```hocon
+env {
+ parallelism = 1
+ job.mode = "BATCH"
+}
+
+source {
+ AmazonSqs {
+ url = "http://sqs-host:4566/000000000000/source_queue";
+ access_key_id = "1234"
+ secret_access_key = "abcd"
+ region = "us-east-1"
+ schema = {
+ fields {
+ name = "string"
+ }
+ }
+ }
+}
+
+sink {
+ AmazonSqs {
+ url = "http://sqs-host:4566/000000000000/sink_queue";
+ access_key_id = "1234"
+ secret_access_key = "abcd"
+ region = "us-east-1"
+ }
+}
+```
+
### Write JSON Messages
```hocon
diff --git a/docs/en/connectors/source/AmazonDynamoDB.md
b/docs/en/connectors/source/AmazonDynamoDB.md
index 835ac9b52e..5b7d2422cf 100644
--- a/docs/en/connectors/source/AmazonDynamoDB.md
+++ b/docs/en/connectors/source/AmazonDynamoDB.md
@@ -12,6 +12,12 @@ The connector is a batch source. DynamoDB does not expose
field types in the sam
This source reads the current table data with scan requests. It does not read
DynamoDB Streams or CDC change events.
+## Supported Engines
+
+> Spark<br/>
+> Flink<br/>
+> SeaTunnel Zeta<br/>
+
## Key Features
- [x] [batch](../../introduction/concepts/connector-v2-features.md)
@@ -103,6 +109,13 @@ For small tables, keep the default value. For large
tables, increase it together
Source plugin common parameters, please refer to [Source Common
Options](../common-options/source-common-options.md) for details.
+## Usage Notes
+
+- The source uses DynamoDB scan requests, so it reads the current table
snapshot rather than change events.
+- `access_key_id` and `secret_access_key` are required by this connector. For
DynamoDB Local, use dummy values accepted by the local service.
+- `parallel_scan_threads` controls the number of DynamoDB scan segments.
Increase it together with job parallelism for larger tables.
+- `scan_item_limit` is the page limit used on each scan request, not the total
number of rows in the job.
+
## Data Type Mapping
| SeaTunnel Data Type | DynamoDB Attribute Type |
diff --git a/docs/en/connectors/source/AmazonSqs.md
b/docs/en/connectors/source/AmazonSqs.md
index 6c6648acbf..b44775f11d 100644
--- a/docs/en/connectors/source/AmazonSqs.md
+++ b/docs/en/connectors/source/AmazonSqs.md
@@ -55,9 +55,42 @@ Each receive request asks SQS for up to 10 messages. If more
messages are waitin
- `debezium_json` reads Debezium JSON messages. For details, see [Debezium
JSON](../formats/debezium-json.md).
- `delete_message = true` removes consumed messages from SQS. Keep the default
`false` when you only want to inspect or copy messages without deleting them.
- `access_key_id` and `secret_access_key` are optional, but they must be
configured together when static AWS credentials are used.
+- The source performs one receive request, with up to 10 messages, and then
finishes the bounded job.
## Task Examples
+### Copy Messages Between Local-Compatible Queues
+
+```hocon
+env {
+ parallelism = 1
+ job.mode = "BATCH"
+}
+
+source {
+ AmazonSqs {
+ url = "http://sqs-host:4566/000000000000/source_queue";
+ access_key_id = "1234"
+ secret_access_key = "abcd"
+ region = "us-east-1"
+ schema = {
+ fields {
+ name = "string"
+ }
+ }
+ }
+}
+
+sink {
+ AmazonSqs {
+ url = "http://sqs-host:4566/000000000000/sink_queue";
+ access_key_id = "1234"
+ secret_access_key = "abcd"
+ region = "us-east-1"
+ }
+}
+```
+
### Read JSON Messages
```hocon
diff --git a/docs/zh/connectors/sink/Aerospike.md
b/docs/zh/connectors/sink/Aerospike.md
index e2aaf5fd2e..fd409a679f 100644
--- a/docs/zh/connectors/sink/Aerospike.md
+++ b/docs/zh/connectors/sink/Aerospike.md
@@ -18,7 +18,7 @@ import ChangeLog from '../changelog/connector-aerospike.md';
## 主要特性
- [ ] [精确一次](../../introduction/concepts/connector-v2-features.md)
-- [ ] [CDC](../../introduction/concepts/connector-v2-features.md)
+- [ ] [变更数据捕获](../../introduction/concepts/connector-v2-features.md)
- [ ] [支持多表写入](../../introduction/concepts/connector-v2-features.md)
- [ ] [定时刷新](../../introduction/concepts/connector-v2-features.md)
@@ -27,6 +27,7 @@ import ChangeLog from '../changelog/connector-aerospike.md';
用于向 Aerospike 数据库写入数据的连接器。该连接器会把数据写入一个指定的 Aerospike 命名空间和集合，并使用 `key` 指定的字段作为
Aerospike 记录主键。
该连接器只写入一个固定的目标集合，不会按表名自动把数据路由到不同的 Aerospike 集合。
+如果同一个 Aerospike 主键已经存在，连接器会更新这条记录的 bin。
## 支持的数据源
@@ -85,6 +86,14 @@ import ChangeLog from '../changelog/connector-aerospike.md';
支持的 Aerospike 类型名称包括
`STRING`、`INTEGER`、`LONG`、`DOUBLE`、`BOOLEAN`、`BYTEARRAY`、`LIST`。
+## 使用说明
+
+- `key` 必须是输入数据中已经存在的字段。该字段的值会转成字符串，并作为 Aerospike 记录主键。
+- `data_format = "string"` 会把选中的字段作为一个 JSON 字符串写入 `bin_name`。
+- `data_format = "map"` 会把选中的字段作为一个 Aerospike map 写入 `bin_name`。
+- `data_format = "kv"` 会把每个配置字段写成独立 Aerospike bin，并且不会使用 `bin_name`。
+- 只有 Aerospike 未启用认证时，才适合把 `username` 和 `password` 留空。
+
## 任务示例
### 将 FakeSource 数据写入 Aerospike
diff --git a/docs/zh/connectors/sink/AmazonDynamoDB.md
b/docs/zh/connectors/sink/AmazonDynamoDB.md
index c8e15f55df..10bfbb06dc 100644
--- a/docs/zh/connectors/sink/AmazonDynamoDB.md
+++ b/docs/zh/connectors/sink/AmazonDynamoDB.md
@@ -2,14 +2,20 @@ import ChangeLog from
'../changelog/connector-amazondynamodb.md';
# AmazonDynamoDB
-> Amazon DynamoDB Sink 连接器
+> Amazon DynamoDB 写入连接器
## 描述
-Amazon DynamoDB Sink 连接器用于将 SeaTunnel 数据行写入 DynamoDB 表。
+Amazon DynamoDB 写入连接器用于将 SeaTunnel 数据行写入 DynamoDB 表。
目标表必须提前创建。连接器会把每一行写成一个 DynamoDB item，并使用批量写入请求。它支持单表写入，也支持上游数据行携带表名时的多表写入。
+## 支持的引擎
+
+> Spark<br/>
+> Flink<br/>
+> SeaTunnel Zeta<br/>
+
## 主要特性
- [ ] [精确一次](../../introduction/concepts/connector-v2-features.md)
@@ -83,6 +89,13 @@ DynamoDB batch write 每次最多支持 25 条写请求，所以默认值为 `25
Sink 连接器通用参数，请参考 [Sink 通用选项](../common-options/sink-common-options.md)。
+## 使用说明
+
+- 启动 SeaTunnel 任务前，需要先创建目标 DynamoDB 表。该写入连接器不会自动创建表或主键结构。
+- `access_key_id` 和 `secret_access_key` 是必填项。使用 DynamoDB Local
时，可以填写本地服务接受的占位值。
+- DynamoDB 每次 batch write 最多接受 25 条写请求，所以 `batch_size` 不要超过 `25`。
+- 写入连接器会对未处理 item 做指数退避重试，但不提供精确一次保证。
+
## 数据类型映射
| SeaTunnel 数据类型 | DynamoDB 属性类型 |
diff --git a/docs/zh/connectors/sink/AmazonSqs.md
b/docs/zh/connectors/sink/AmazonSqs.md
index f0e65ad6f6..c76450acc9 100644
--- a/docs/zh/connectors/sink/AmazonSqs.md
+++ b/docs/zh/connectors/sink/AmazonSqs.md
@@ -2,11 +2,11 @@ import ChangeLog from '../changelog/connector-amazonsqs.md';
# AmazonSqs
-> Amazon SQS Sink 连接器
+> Amazon SQS 写入连接器
## 描述
-Amazon SQS Sink 连接器用于把每条输入的 SeaTunnel 行数据写入一个 Amazon SQS 队列 URL。
+Amazon SQS 写入连接器用于把每条输入的 SeaTunnel 行数据写入一个 Amazon SQS 队列 URL。
连接器会按照 `format` 序列化数据，并把序列化后的内容作为 SQS 消息体发送。
## 支持的引擎
@@ -20,11 +20,11 @@ Amazon SQS Sink 连接器用于把每条输入的 SeaTunnel 行数据写入一
- [x] [批处理](../../introduction/concepts/connector-v2-features.md)
- [x] [流处理](../../introduction/concepts/connector-v2-features.md)
- [ ] [精确一次](../../introduction/concepts/connector-v2-features.md)
-- [ ] [cdc](../../introduction/concepts/connector-v2-features.md)
+- [ ] [变更数据捕获](../../introduction/concepts/connector-v2-features.md)
- [ ] [支持多表写入](../../introduction/concepts/connector-v2-features.md)
- [ ] [定时刷新](../../introduction/concepts/connector-v2-features.md)
-## Sink 选项
+## 写入选项
| 名称 | 类型 | 是否必填 | 默认值 | 描述
|
|-------------------|--------|------|------|----------------------------------------------------------------------------------------------------------------------|
@@ -44,11 +44,44 @@ Amazon SQS Sink 连接器用于把每条输入的 SeaTunnel 行数据写入一
- `text`：用 `field_delimiter` 拼接每行中的字段。
- `canal_json`：写出 Canal JSON 消息，详见 [Canal JSON](../formats/canal-json.md)。
- `debezium_json`：写出 Debezium JSON 消息，详见 [Debezium
JSON](../formats/debezium-json.md)。
-- 当前 sink 只发送消息体，不提供 SQS message attributes、delay seconds、deduplication ID 或
message group ID 等配置。
+- 当前写入连接器只发送消息体，不提供 SQS message attributes、delay seconds、deduplication ID 或
message group ID 等配置。
- `access_key_id` 和 `secret_access_key` 是可选项；如果使用静态 AWS 凭证，需要两个一起配置。
+- 该写入连接器会把每条 SeaTunnel 行数据发送成一条 SQS 消息，不会把多行数据合并到一次 SQS 请求里。
## 任务示例
+### 在本地兼容队列之间复制消息
+
+```hocon
+env {
+ parallelism = 1
+ job.mode = "BATCH"
+}
+
+source {
+ AmazonSqs {
+ url = "http://sqs-host:4566/000000000000/source_queue";
+ access_key_id = "1234"
+ secret_access_key = "abcd"
+ region = "us-east-1"
+ schema = {
+ fields {
+ name = "string"
+ }
+ }
+ }
+}
+
+sink {
+ AmazonSqs {
+ url = "http://sqs-host:4566/000000000000/sink_queue";
+ access_key_id = "1234"
+ secret_access_key = "abcd"
+ region = "us-east-1"
+ }
+}
+```
+
### 写入 JSON 消息
```hocon
diff --git a/docs/zh/connectors/source/AmazonDynamoDB.md
b/docs/zh/connectors/source/AmazonDynamoDB.md
index 771bcaf01e..44839930bb 100644
--- a/docs/zh/connectors/source/AmazonDynamoDB.md
+++ b/docs/zh/connectors/source/AmazonDynamoDB.md
@@ -10,7 +10,13 @@ Amazon DynamoDB 源连接器通过 DynamoDB scan 请求读取已有表中的数
该连接器是批处理源。DynamoDB 不像关系型数据库那样提供完整字段类型信息，所以必须在 SeaTunnel 中显式配置 schema。
-该 Source 使用 scan 请求读取表中当前已有的数据，不读取 DynamoDB Streams 或 CDC 变更事件。
+该源连接器使用 scan 请求读取表中当前已有的数据，不读取 DynamoDB Streams 或 CDC 变更事件。
+
+## 支持的引擎
+
+> Spark<br/>
+> Flink<br/>
+> SeaTunnel Zeta<br/>
## 主要特性
@@ -33,7 +39,7 @@ Amazon DynamoDB 源连接器通过 DynamoDB scan 请求读取已有表中的数
| schema | config | 是 | - | 要读取的 SeaTunnel 字段。 |
| scan_item_limit | int | 否 | 1 | 每次 scan 请求返回的最大 item 数。 |
| parallel_scan_threads | int | 否 | 2 | parallel scan 的逻辑分片数。 |
-| common-options | object | 否 | - | Source 插件通用参数。 |
+| common-options | object | 否 | - | 源插件通用参数。 |
### url [string]
@@ -103,6 +109,13 @@ DynamoDB parallel scan 使用的逻辑分片数量。
源连接器通用参数，请参考[源通用选项](../common-options/source-common-options.md)。
+## 使用说明
+
+- 该源连接器使用 DynamoDB scan 请求，所以读取的是当前表快照，不是变更事件。
+- `access_key_id` 和 `secret_access_key` 是必填项。使用 DynamoDB Local
时，可以填写本地服务接受的占位值。
+- `parallel_scan_threads` 控制 DynamoDB scan 的逻辑分片数。大表可以结合任务并行度一起调大。
+- `scan_item_limit` 是每次 scan 请求的分页限制，不是整个任务最多读取的总行数。
+
## 数据类型映射
| SeaTunnel 数据类型 | DynamoDB 属性类型 |
diff --git a/docs/zh/connectors/source/AmazonSqs.md
b/docs/zh/connectors/source/AmazonSqs.md
index ff95bba747..736cc4ca6b 100644
--- a/docs/zh/connectors/source/AmazonSqs.md
+++ b/docs/zh/connectors/source/AmazonSqs.md
@@ -54,9 +54,42 @@ Amazon SQS 源连接器用于从一个 Amazon SQS 队列 URL 读取消息。连
- `debezium_json`：读取 Debezium JSON 消息，详见 [Debezium
JSON](../formats/debezium-json.md)。
- `delete_message = true` 会删除已经消费的 SQS 消息。如果只是检查或复制消息，建议保留默认值 `false`。
- `access_key_id` 和 `secret_access_key` 是可选项；如果使用静态 AWS 凭证，需要两个一起配置。
+- 该源连接器只执行一次 receive 请求，最多读取 10 条消息，然后结束这个有界任务。
## 任务示例
+### 在本地兼容队列之间复制消息
+
+```hocon
+env {
+ parallelism = 1
+ job.mode = "BATCH"
+}
+
+source {
+ AmazonSqs {
+ url = "http://sqs-host:4566/000000000000/source_queue";
+ access_key_id = "1234"
+ secret_access_key = "abcd"
+ region = "us-east-1"
+ schema = {
+ fields {
+ name = "string"
+ }
+ }
+ }
+}
+
+sink {
+ AmazonSqs {
+ url = "http://sqs-host:4566/000000000000/sink_queue";
+ access_key_id = "1234"
+ secret_access_key = "abcd"
+ region = "us-east-1"
+ }
+}
+```
+
### 读取 JSON 消息
```hocon
