This is an automated email from the ASF dual-hosted git repository.
zhangshenghang pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/seatunnel.git
The following commit(s) were added to refs/heads/dev by this push:
new 507cd5060f [Docs][Connector-V2] Improve Amazon SQS connector docs
(#11270)
507cd5060f is described below
commit 507cd5060ff23ea29442c6a88b33ed022132a155
Author: Daniel Carter <[email protected]>
AuthorDate: Fri Jul 3 08:17:59 2026 +0800
[Docs][Connector-V2] Improve Amazon SQS connector docs (#11270)
Co-authored-by: DanielCarter-stack
<[email protected]>
Co-authored-by: Shenghang <[email protected]>
---
docs/en/connectors/sink/AmazonSqs.md | 120 ++++++++++++++++++-------------
docs/en/connectors/source/AmazonSqs.md | 107 ++++++++++++++++-----------
docs/zh/connectors/sink/AmazonSqs.md | 127 +++++++++++++++++++--------------
docs/zh/connectors/source/AmazonSqs.md | 109 +++++++++++++++++-----------
4 files changed, 277 insertions(+), 186 deletions(-)
diff --git a/docs/en/connectors/sink/AmazonSqs.md
b/docs/en/connectors/sink/AmazonSqs.md
index 1951afc255..7991f3168c 100644
--- a/docs/en/connectors/sink/AmazonSqs.md
+++ b/docs/en/connectors/sink/AmazonSqs.md
@@ -4,86 +4,104 @@ import ChangeLog from '../changelog/connector-amazonsqs.md';
> Amazon SQS sink connector
+## Description
+
+The Amazon SQS sink connector writes each incoming SeaTunnel row to one Amazon
SQS queue URL. The row
+is serialized by the configured `format`, and the serialized value is sent as
the SQS message body.
+
## Support Those Engines
> Spark<br/>
> Flink<br/>
> SeaTunnel Zeta<br/>
-## Description
-
-Write data to Amazon SQS
-
## Key Features
- [x] [batch](../../introduction/concepts/connector-v2-features.md)
- [x] [stream](../../introduction/concepts/connector-v2-features.md)
-- [x] [exactly-once](../../introduction/concepts/connector-v2-features.md)
-- [ ] [column projection](../../introduction/concepts/connector-v2-features.md)
-- [ ] [parallelism](../../introduction/concepts/connector-v2-features.md)
-- [ ] [support user-defined
split](../../introduction/concepts/connector-v2-features.md)
+- [ ] [exactly-once](../../introduction/concepts/connector-v2-features.md)
+- [ ] [cdc](../../introduction/concepts/connector-v2-features.md)
+- [ ] [support multiple table
write](../../introduction/concepts/connector-v2-features.md)
- [ ] [timer flush](../../introduction/concepts/connector-v2-features.md)
## Sink Options
-| Name | Type | Required | Default |
Description
|
-|-------------------------|--------|----------|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| url | String | Yes | - | The Queue URL to
read from Amazon SQS.
|
-| region | String | No | - | The AWS region for
the SQS service
|
-| format | String | No | json | Data format. The
default format is json. Optional text format, canal-json and debezium-json.If
you use json or text format. The default field separator is ", ". If you
customize the delimiter, add the "field_delimiter" option.If you use canal
format, please refer to [canal-json](../formats/canal-json.md) for details.If
you use debezium format, please refer to
[debezium-json](../formats/debezium-json.md) for details. |
-| format_error_handle_way | String | No | fail | The processing
method of data format error. The default value is fail, and the optional value
is (fail, skip). When fail is selected, data format error will block and an
exception will be thrown. When skip is selected, data format error will skip
this line data.
|
-| field_delimiter | String | No | , | Customize the field
delimiter for data format.
|
+| Name | Type | Required | Default | Description
|
+|-------------------|--------|----------|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| url | String | Yes | - | Full SQS queue URL to
write to, for example
`https://sqs.us-east-1.amazonaws.com/123456789012/sink_queue`.
|
+| region | String | Yes | - | AWS region of the SQS
queue, for example `us-east-1`.
|
+| access_key_id | String | No | - | AWS access key ID. Set it
together with `secret_access_key` to use static credentials. Leave both unset
to use the AWS default credential provider chain. |
+| secret_access_key | String | No | - | AWS secret access key. Set
it together with `access_key_id` to use static credentials.
|
+| format | String | No | json | Message body format.
Supported values are `json`, `text`, `canal_json`, and `debezium_json`.
|
+| field_delimiter | String | No | , | Field delimiter used when
`format = text`.
|
+| common-options | | No | - | Sink plugin common
parameters. For details, see [Sink Common
Options](../common-options/sink-common-options.md).
|
+
+## Format Notes
+
+- `json` writes each row as a JSON object.
+- `text` joins row fields by `field_delimiter`.
+- `canal_json` writes Canal JSON messages. For details, see [Canal
JSON](../formats/canal-json.md).
+- `debezium_json` writes Debezium JSON messages. For details, see [Debezium
JSON](../formats/debezium-json.md).
+- The sink sends only the message body. It does not expose SQS message
attributes, delay seconds, deduplication ID, or message group ID options.
+
+## Task Examples
+
+### Write JSON Messages
+
+```hocon
+env {
+ parallelism = 1
+ job.mode = "BATCH"
+}
+
+source {
+ FakeSource {
+ row.num = 1
+ schema = {
+ fields {
+ name = string
+ }
+ }
+ rows = [
+ {
+ kind = INSERT
+ fields = ["test_name"]
+ }
+ ]
+ }
+}
+
+sink {
+ AmazonSqs {
+ url = "https://sqs.us-east-1.amazonaws.com/123456789012/sink_queue";
+ region = "us-east-1"
+ access_key_id = "AKIA..."
+ secret_access_key = "SECRET..."
+ }
+}
+```
-## Task Example
+### Write Text Messages With a Custom Delimiter
-```bash
+```hocon
source {
FakeSource {
schema = {
fields {
- c_map = "map<string, string>"
- c_array = "array<int>"
- c_string = string
- c_boolean = boolean
- c_tinyint = tinyint
- c_smallint = smallint
- c_int = int
- c_bigint = bigint
- c_float = float
- c_double = double
- c_bytes = bytes
- c_date = date
- c_decimal = "decimal(38, 18)"
- c_timestamp = timestamp
- c_row = {
- c_map = "map<string, string>"
- c_array = "array<int>"
- c_string = string
- c_boolean = boolean
- c_tinyint = tinyint
- c_smallint = smallint
- c_int = int
- c_bigint = bigint
- c_float = float
- c_double = double
- c_bytes = bytes
- c_date = date
- c_decimal = "decimal(38, 18)"
- c_timestamp = timestamp
- }
+ artist = string
+ album = string
+ release_year = int
}
}
- plugin_output = "fake"
}
}
sink {
AmazonSqs {
- url = "http://127.0.0.1:8000";
+ url = "https://sqs.us-east-1.amazonaws.com/123456789012/sink_queue";
region = "us-east-1"
- queue = "queueName"
format = text
- field_delimiter = "|"
+ field_delimiter = "|"
}
}
```
diff --git a/docs/en/connectors/source/AmazonSqs.md
b/docs/en/connectors/source/AmazonSqs.md
index 67b179d561..702673a345 100644
--- a/docs/en/connectors/source/AmazonSqs.md
+++ b/docs/en/connectors/source/AmazonSqs.md
@@ -2,7 +2,15 @@ import ChangeLog from '../changelog/connector-amazonsqs.md';
# AmazonSqs
-> AmazonSqs source connector
+> Amazon SQS source connector
+
+## Description
+
+The Amazon SQS source connector reads messages from one Amazon SQS queue URL.
Each message body is
+deserialized by the configured `format` and `schema`, then emitted as
SeaTunnel rows.
+
+The connector uses a single reader and finishes the job after the current
receive request is
+processed. It is suitable for bounded reads from a queue.
## Support Those Engines
@@ -13,71 +21,90 @@ import ChangeLog from '../changelog/connector-amazonsqs.md';
## Key Features
- [x] [batch](../../introduction/concepts/connector-v2-features.md)
-- [x] [stream](../../introduction/concepts/connector-v2-features.md)
+- [ ] [stream](../../introduction/concepts/connector-v2-features.md)
- [ ] [exactly-once](../../introduction/concepts/connector-v2-features.md)
-- [ ] [column projection](../../introduction/concepts/connector-v2-features.md)
+- [x] [column projection](../../introduction/concepts/connector-v2-features.md)
- [ ] [parallelism](../../introduction/concepts/connector-v2-features.md)
- [ ] [support user-defined
split](../../introduction/concepts/connector-v2-features.md)
-## Description
+## Source Options
-Read data from Amazon SQS.
+| Name | Type | Required | Default | Description
|
+|--------------------------------|---------|----------|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| url | String | Yes | - | Full SQS
queue URL to read from, for example
`https://sqs.us-east-1.amazonaws.com/123456789012/source_queue`.
|
+| region | String | Yes | - | AWS region
of the SQS queue, for example `us-east-1`.
|
+| schema | Config | Yes | - | Message body
schema. For more details, see [Schema
Feature](../../introduction/concepts/schema-feature.md).
|
+| access_key_id | String | No | - | AWS access
key ID. Set it together with `secret_access_key` to use static credentials.
Leave both unset to use the AWS default credential provider chain. |
+| secret_access_key | String | No | - | AWS secret
access key. Set it together with `access_key_id` to use static credentials.
|
+| format | String | No | json | Message body
format. Supported values are `json`, `text`, `canal_json`, and `debezium_json`.
|
+| field_delimiter | String | No | , | Field
delimiter used when `format = text`.
|
+| delete_message | Boolean | No | false | Whether to
delete a message from the queue after it is read and deserialized successfully.
|
+| message_group_id | String | No | - | Message
group ID option kept for compatibility. It is not required for normal SQS
reads. |
+| debezium_record_include_schema | Boolean | No | true | Whether
Debezium JSON messages include a schema. This option is used only when `format
= debezium_json`. |
+| common-options | | No | - | Source
plugin common parameters. For details, see [Source Common
Options](../common-options/source-common-options.md).
|
+
+## Format Notes
+
+- `json` reads each message body as a JSON object that matches `schema`.
+- `text` splits each message body by `field_delimiter` and maps the values to
fields in `schema` order.
+- `canal_json` reads Canal JSON messages. For details, see [Canal
JSON](../formats/canal-json.md).
+- `debezium_json` reads Debezium JSON messages. For details, see [Debezium
JSON](../formats/debezium-json.md).
+- `delete_message = true` removes consumed messages from SQS. Keep the default
`false` when you only want to inspect or copy messages without deleting them.
+
+## Task Examples
+
+### Read JSON Messages
+
+```hocon
+env {
+ parallelism = 1
+ job.mode = "BATCH"
+}
-## Source Options
+source {
+ AmazonSqs {
+ url = "https://sqs.us-east-1.amazonaws.com/123456789012/source_queue";
+ region = "us-east-1"
+ access_key_id = "AKIA..."
+ secret_access_key = "SECRET..."
+ schema = {
+ fields {
+ name = string
+ }
+ }
+ }
+}
-| Name | Type | Required | Default |
Description
|
-|-------------------------|--------|----------|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| url | String | Yes | - | The Queue URL to
read from Amazon SQS.
|
-| region | String | No | - | The AWS region for
the SQS service
|
-| schema | Config | No | - | The structure of the
data, including field names and field types. For more details, please refer to
[Schema Feature](../../introduction/concepts/schema-feature.md).
[...]
-| format | String | No | json | Data format. The
default format is json. Optional text format, canal-json and debezium-json.If
you use json or text format. The default field separator is ", ". If you
customize the delimiter, add the "field_delimiter" option.If you use canal
format, please refer to [canal-json](../formats/canal-json.md) for details.If
you use debezium format, please refer to
[debezium-json](../formats/debezium-json.md) for details. |
-| format_error_handle_way | String | No | fail | The processing
method of data format error. The default value is fail, and the optional value
is (fail, skip). When fail is selected, data format error will block and an
exception will be thrown. When skip is selected, data format error will skip
this line data.
|
-| field_delimiter | String | No | , | Customize the field
delimiter for data format.
|
-| common-options | | No | - | Source plugin common
parameters, please refer to [Source Common
Options](../common-options/source-common-options.md) for details
|
+sink {
+ Console {}
+}
+```
-## Task Example
+### Read Text Messages With a Custom Delimiter
-```bash
+```hocon
source {
AmazonSqs {
- url = "http://127.0.0.1:4566";
+ url = "https://sqs.us-east-1.amazonaws.com/123456789012/source_queue";
region = "us-east-1"
format = text
field_delimiter = "#"
+ delete_message = true
schema = {
fields {
artist = string
- c_map = "map<string, array<int>>"
- c_array = "array<int>"
- c_string = string
- c_boolean = boolean
- c_tinyint = tinyint
- c_smallint = smallint
- c_int = int
- c_bigint = bigint
- c_float = float
- c_double = double
- c_decimal = "decimal(30, 8)"
- c_null = "null"
- c_bytes = bytes
- c_date = date
- c_timestamp = timestamp
+ album = string
+ release_year = int
}
}
}
}
-transform {
- # If you would like to get more information about how to configure
seatunnel and see full list of transform plugins,
- # please go to https://seatunnel.apache.org/docs/transforms/sql
-}
-
sink {
- Console {}
+ Console {}
}
```
## Changelog
<ChangeLog />
-
diff --git a/docs/zh/connectors/sink/AmazonSqs.md
b/docs/zh/connectors/sink/AmazonSqs.md
index 76aff4863a..103cd455ec 100644
--- a/docs/zh/connectors/sink/AmazonSqs.md
+++ b/docs/zh/connectors/sink/AmazonSqs.md
@@ -2,87 +2,106 @@ import ChangeLog from '../changelog/connector-amazonsqs.md';
# AmazonSqs
-> Amazon SQS 接收器连接器
+> Amazon SQS Sink 连接器
-## 支持以下引擎
+## 描述
+
+Amazon SQS Sink 连接器用于把每条输入的 SeaTunnel 行数据写入一个 Amazon SQS 队列 URL。
+连接器会按照 `format` 序列化数据，并把序列化后的内容作为 SQS 消息体发送。
+
+## 支持的引擎
> Spark<br/>
> Flink<br/>
> SeaTunnel Zeta<br/>
-## 描述
-
-将数据写入 Amazon SQS
-
-## 关键特性
+## 主要特性
- [x] [批处理](../../introduction/concepts/connector-v2-features.md)
- [x] [流处理](../../introduction/concepts/connector-v2-features.md)
-- [x] [精确一次](../../introduction/concepts/connector-v2-features.md)
-- [ ] [列映射](../../introduction/concepts/connector-v2-features.md)
-- [ ] [并行性](../../introduction/concepts/connector-v2-features.md)
-- [ ] [支持用户定义的拆分](../../introduction/concepts/connector-v2-features.md)
+- [ ] [精确一次](../../introduction/concepts/connector-v2-features.md)
+- [ ] [cdc](../../introduction/concepts/connector-v2-features.md)
+- [ ] [支持多表写入](../../introduction/concepts/connector-v2-features.md)
+- [ ] [定时刷新](../../introduction/concepts/connector-v2-features.md)
+
+## Sink 选项
+
+| 名称 | 类型 | 是否必填 | 默认值 | 描述
|
+|-------------------|--------|------|------|----------------------------------------------------------------------------------------------------------------------|
+| url | String | 是 | - | 要写入的完整 SQS 队列 URL，例如
`https://sqs.us-east-1.amazonaws.com/123456789012/sink_queue`。
|
+| region | String | 是 | - | SQS 队列所在的 AWS 区域，例如 `us-east-1`。
|
+| access_key_id | String | 否 | - | AWS access key ID。和
`secret_access_key` 一起配置时使用静态凭证；两者都不配置时使用 AWS 默认凭证链。
|
+| secret_access_key | String | 否 | - | AWS secret access key。和
`access_key_id` 一起配置时使用静态凭证。
|
+| format | String | 否 | json | 消息体格式。支持
`json`、`text`、`canal_json`、`debezium_json`。
|
+| field_delimiter | String | 否 | , | 当 `format = text` 时使用的字段分隔符。
|
+| common-options | | 否 | - | Sink 插件通用参数，详见 [Sink Common
Options](../common-options/sink-common-options.md)。
|
-## 参数和选项
+## 格式说明
-| 名称 | 类型 | 必需 | 默认值 |
Description
|
-|-------------------------|--------|--|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| url | String | 是 | - | 从Amazon SQS读取的队列URL.
|
-| region | String | 否 | - | SQS服务的AWS区域
|
-| format | String | 否 | json | 数据格式。默认格式为json。可选文本格式，canal
json和debezium
json。如果你使用json或文本格式。默认字段分隔符为“，”。如果自定义分隔符，请添加“field_delimiter”选项。如果您使用canal格式，请参阅[canal-json](../formats/canal-json.md)了解详细信息。如果您使用debezium格式，请参阅[debezium-json](../formats/debezium-json.md)了解详细信息.
|
-| format_error_handle_way | String | 否 | fail |
数据格式错误的处理方法。默认值为fail，可选值为（fail，skip）。当选择失败时，数据格式错误将被阻止，并引发异常。当选择跳过时，数据格式错误将跳过此行数据.
|
-| field_delimiter | String | 否 | , | 自定义数据格式的字段分隔符.
|
+- `json`：把每行数据写成 JSON 对象。
+- `text`：用 `field_delimiter` 拼接每行中的字段。
+- `canal_json`：写出 Canal JSON 消息，详见 [Canal JSON](../formats/canal-json.md)。
+- `debezium_json`：写出 Debezium JSON 消息，详见 [Debezium
JSON](../formats/debezium-json.md)。
+- 当前 sink 只发送消息体，不提供 SQS message attributes、delay seconds、deduplication ID 或
message group ID 等配置。
## 任务示例
-```bash
+### 写入 JSON 消息
+
+```hocon
+env {
+ parallelism = 1
+ job.mode = "BATCH"
+}
+
+source {
+ FakeSource {
+ row.num = 1
+ schema = {
+ fields {
+ name = string
+ }
+ }
+ rows = [
+ {
+ kind = INSERT
+ fields = ["test_name"]
+ }
+ ]
+ }
+}
+
+sink {
+ AmazonSqs {
+ url = "https://sqs.us-east-1.amazonaws.com/123456789012/sink_queue";
+ region = "us-east-1"
+ access_key_id = "AKIA..."
+ secret_access_key = "SECRET..."
+ }
+}
+```
+
+### 使用自定义分隔符写入文本消息
+
+```hocon
source {
FakeSource {
schema = {
fields {
- c_map = "map<string, string>"
- c_array = "array<int>"
- c_string = string
- c_boolean = boolean
- c_tinyint = tinyint
- c_smallint = smallint
- c_int = int
- c_bigint = bigint
- c_float = float
- c_double = double
- c_bytes = bytes
- c_date = date
- c_decimal = "decimal(38, 18)"
- c_timestamp = timestamp
- c_row = {
- c_map = "map<string, string>"
- c_array = "array<int>"
- c_string = string
- c_boolean = boolean
- c_tinyint = tinyint
- c_smallint = smallint
- c_int = int
- c_bigint = bigint
- c_float = float
- c_double = double
- c_bytes = bytes
- c_date = date
- c_decimal = "decimal(38, 18)"
- c_timestamp = timestamp
- }
+ artist = string
+ album = string
+ release_year = int
}
}
- plugin_output = "fake"
}
}
sink {
AmazonSqs {
- url = "http://127.0.0.1:8000";
+ url = "https://sqs.us-east-1.amazonaws.com/123456789012/sink_queue";
region = "us-east-1"
- queue = "queueName"
format = text
- field_delimiter = "|"
+ field_delimiter = "|"
}
}
```
diff --git a/docs/zh/connectors/source/AmazonSqs.md
b/docs/zh/connectors/source/AmazonSqs.md
index 2cf1ba0b9c..47bfe59dc5 100644
--- a/docs/zh/connectors/source/AmazonSqs.md
+++ b/docs/zh/connectors/source/AmazonSqs.md
@@ -2,78 +2,105 @@ import ChangeLog from '../changelog/connector-amazonsqs.md';
# AmazonSqs
-> AmazonSqs 源连接器
+> Amazon SQS 源连接器
-## 支持一下引擎
+## 描述
+
+Amazon SQS 源连接器用于从一个 Amazon SQS 队列 URL 读取消息。连接器会按照 `format` 和
+`schema` 解析每条消息的消息体，然后输出为 SeaTunnel 行数据。
+
+该连接器使用单个 reader，处理完本次接收到的消息后任务会结束，适合从队列中做有界读取。
+
+## 支持的引擎
> Spark<br/>
> Flink<br/>
> SeaTunnel Zeta<br/>
-## 关键特性
+## 主要特性
- [x] [批处理](../../introduction/concepts/connector-v2-features.md)
-- [x] [流处理](../../introduction/concepts/connector-v2-features.md)
+- [ ] [流处理](../../introduction/concepts/connector-v2-features.md)
- [ ] [精确一次](../../introduction/concepts/connector-v2-features.md)
-- [ ] [列投影](../../introduction/concepts/connector-v2-features.md)
+- [x] [列投影](../../introduction/concepts/connector-v2-features.md)
- [ ] [并行度](../../introduction/concepts/connector-v2-features.md)
- [ ] [支持用户自定义分片](../../introduction/concepts/connector-v2-features.md)
-## 描述
-
-从 Amazon SQS 读取数据.
-
## 源选项
-| 名称 | 类型 | 必需 | 默认值 | 描述
|
-|-------------------------|--------|----|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| url | String | 是 | - | 从 Amazon SQ S读取的队列 URL.
|
-| region | String | 否 | - | SQS 服务的 AWS 分区
|
-| schema | Config | 否 | - | 数据的结构，包括字段名和字段类型。更多详情请参考
[Schema 特性](../../introduction/concepts/schema-feature.md)。
|
-| format | String | 否 | json |
数据格式。默认格式为json。可选文本格式，canal-json和debezium-json。如果你使用json或text格式。默认字段分隔符为 ", ".
如果自定义分隔符，请添加"field_delimiter"选项。如果使用 canal
格式,详见[canal-json](../formats/canal-json.md).如果使用 debezium
格式,详见[debezium-json](../formats/debezium-json.md).. |
-| format_error_handle_way | String | 否 | fail | 数据格式错误的处理方法.
默认值为fail，可选值为（fail，skip）. 当选择失败时，数据格式错误将被阻止，并引发异常. 当选择跳过时，数据格式错误将跳过此行数据.
|
-| field_delimiter | String | 否 | , | 自定义数据格式的字段分隔符.
|
-| common-options | | 否 | - | 源插件常用参数, 详见
[源通用选项](../common-options/source-common-options.md)
|
+| 名称 | 类型 | 是否必填 | 默认值 | 描述
|
+|--------------------------------|---------|------|-------|------------------------------------------------------------------------------------------------------------------------|
+| url | String | 是 | - | 要读取的完整 SQS 队列
URL，例如 `https://sqs.us-east-1.amazonaws.com/123456789012/source_queue`。
|
+| region | String | 是 | - | SQS 队列所在的 AWS
区域，例如 `us-east-1`。
|
+| schema | Config | 是 | - |
消息体结构，包含字段名和字段类型。更多说明请参考 [Schema
特性](../../introduction/concepts/schema-feature.md)。
|
+| access_key_id | String | 否 | - | AWS access key
ID。和 `secret_access_key` 一起配置时使用静态凭证；两者都不配置时使用 AWS 默认凭证链。
|
+| secret_access_key | String | 否 | - | AWS secret access
key。和 `access_key_id` 一起配置时使用静态凭证。
|
+| format | String | 否 | json | 消息体格式。支持
`json`、`text`、`canal_json`、`debezium_json`。
|
+| field_delimiter | String | 否 | , | 当 `format = text`
时使用的字段分隔符。
|
+| delete_message | Boolean | 否 | false |
读取并成功解析消息后，是否从队列中删除该消息。
|
+| message_group_id | String | 否 | - | 为兼容保留的消息分组 ID
选项，普通 SQS 读取不需要配置。
|
+| debezium_record_include_schema | Boolean | 否 | true | Debezium JSON
消息是否包含 schema。仅在 `format = debezium_json` 时使用。
|
+| common-options | | 否 | - | 源插件通用参数，详见 [Source
Common Options](../common-options/source-common-options.md)。
|
+
+## 格式说明
+
+- `json`：把每条消息体按 JSON 对象解析，并要求字段能对应到 `schema`。
+- `text`：按 `field_delimiter` 切分消息体，并按 `schema` 中字段顺序映射。
+- `canal_json`：读取 Canal JSON 消息，详见 [Canal JSON](../formats/canal-json.md)。
+- `debezium_json`：读取 Debezium JSON 消息，详见 [Debezium
JSON](../formats/debezium-json.md)。
+- `delete_message = true` 会删除已经消费的 SQS 消息。如果只是检查或复制消息，建议保留默认值 `false`。
## 任务示例
-```bash
+### 读取 JSON 消息
+
+```hocon
+env {
+ parallelism = 1
+ job.mode = "BATCH"
+}
+
source {
AmazonSqs {
- url = "http://127.0.0.1:4566";
+ url = "https://sqs.us-east-1.amazonaws.com/123456789012/source_queue";
+ region = "us-east-1"
+ access_key_id = "AKIA..."
+ secret_access_key = "SECRET..."
+ schema = {
+ fields {
+ name = string
+ }
+ }
+ }
+}
+
+sink {
+ Console {}
+}
+```
+
+### 使用自定义分隔符读取文本消息
+
+```hocon
+source {
+ AmazonSqs {
+ url = "https://sqs.us-east-1.amazonaws.com/123456789012/source_queue";
region = "us-east-1"
format = text
field_delimiter = "#"
+ delete_message = true
schema = {
fields {
artist = string
- c_map = "map<string, array<int>>"
- c_array = "array<int>"
- c_string = string
- c_boolean = boolean
- c_tinyint = tinyint
- c_smallint = smallint
- c_int = int
- c_bigint = bigint
- c_float = float
- c_double = double
- c_decimal = "decimal(30, 8)"
- c_null = "null"
- c_bytes = bytes
- c_date = date
- c_timestamp = timestamp
+ album = string
+ release_year = int
}
}
}
}
-transform {
- # 如果你想了解更多关于如何配置seatunnel的信息，并查看转换插件的完整列表,
- # 请前往 https://seatunnel.apache.org/docs/transforms/sql
-}
-
sink {
- Console {}
+ Console {}
}
```
