This is an automated email from the ASF dual-hosted git repository.
davidzollo pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/seatunnel.git
The following commit(s) were added to refs/heads/dev by this push:
new c87e1ff9b3 [Docs][Connector-V2] Improve Email Databend and FtpFile
docs (#11298)
c87e1ff9b3 is described below
commit c87e1ff9b3b61b3afdeb9d218ff95398f8af17e7
Author: Daniel Carter <[email protected]>
AuthorDate: Fri Jul 3 23:00:13 2026 +0800
[Docs][Connector-V2] Improve Email Databend and FtpFile docs (#11298)
Co-authored-by: DanielCarter-stack
<[email protected]>
---
docs/en/connectors/sink/Databend.md | 5 ++++-
docs/en/connectors/sink/Email.md | 11 +++++------
docs/en/connectors/sink/FtpFile.md | 9 ++++++++-
docs/en/connectors/source/Databend.md | 35 ++++++++++++++++++++++-------------
docs/en/connectors/source/FtpFile.md | 1 +
docs/zh/connectors/sink/Databend.md | 12 +++++++-----
docs/zh/connectors/sink/Email.md | 11 +++++------
docs/zh/connectors/sink/FtpFile.md | 13 ++++++++++---
docs/zh/connectors/source/Databend.md | 35 ++++++++++++++++++++++-------------
docs/zh/connectors/source/FtpFile.md | 3 ++-
10 files changed, 86 insertions(+), 49 deletions(-)
diff --git a/docs/en/connectors/sink/Databend.md
b/docs/en/connectors/sink/Databend.md
index 92704b1632..1d43a39b4e 100644
--- a/docs/en/connectors/sink/Databend.md
+++ b/docs/en/connectors/sink/Databend.md
@@ -157,6 +157,9 @@ sink {
### CDC mode
+Set `conflict_key` to the primary-key column used to merge update/delete
events. Set
+`enable_delete = true` only when DELETE events should remove rows from
Databend.
+
```hocon
sink {
Databend {
@@ -181,4 +184,4 @@ sink {
## Changelog
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
diff --git a/docs/en/connectors/sink/Email.md b/docs/en/connectors/sink/Email.md
index a1418e468d..ad80c17055 100644
--- a/docs/en/connectors/sink/Email.md
+++ b/docs/en/connectors/sink/Email.md
@@ -92,8 +92,8 @@ Sink plugin common parameters, please refer to [Sink Common
Options](../common-o
### Send one table to multiple recipients
-This example follows the Email e2e job. It uses a test SMTP server without
authentication and sends
-one email to each address in `email_to_address`.
+This example uses an SMTP server without authentication and sends one email to
each address in
+`email_to_address`.
```hocon
env {
@@ -132,7 +132,7 @@ sink {
EmailSink {
email_from_address = "[email protected]"
email_to_address = "[email protected],[email protected]"
- email_host = "email-e2e"
+ email_host = "smtp.example.com"
email_transport_protocol = "smtp"
email_smtp_auth = false
email_smtp_port = 3025
@@ -147,8 +147,7 @@ sink {
### Send multiple tables
-Email sink supports multi-table input. In the e2e job, two upstream tables
create two emails for
-each recipient.
+Email sink supports multi-table input. Two upstream tables create two emails
for each recipient.
```hocon
source {
@@ -184,7 +183,7 @@ sink {
EmailSink {
email_from_address = "[email protected]"
email_to_address = "[email protected],[email protected]"
- email_host = "email-e2e"
+ email_host = "smtp.example.com"
email_transport_protocol = "smtp"
email_smtp_auth = false
email_smtp_port = 3025
diff --git a/docs/en/connectors/sink/FtpFile.md
b/docs/en/connectors/sink/FtpFile.md
index d305c5a851..8098416911 100644
--- a/docs/en/connectors/sink/FtpFile.md
+++ b/docs/en/connectors/sink/FtpFile.md
@@ -48,6 +48,7 @@ By default, we use 2PC commit to ensure `exactly-once`
| tmp_path | string | yes | /tmp/seatunnel
| The result file will write to a tmp path first and
then use `mv` to submit tmp dir to target dir. Need a FTP dir.
|
| connection_mode | string | no | active_local
| The target ftp connection mode
|
| remote_verification_enabled | boolean | no | true
| Whether to enable remote host verification for FTP
data channels
|
+| control_encoding | string | no | UTF-8
| Character encoding for the FTP control connection,
useful for paths with spaces or non-ASCII characters
|
| custom_filename | boolean | no | false
| Whether you need custom the filename
|
| file_name_expression | string | no |
"${transactionId}" | Only used when custom_filename is
true
|
| filename_time_format | string | no | "yyyy.MM.dd"
| Only used when custom_filename is true
|
@@ -110,6 +111,13 @@ The target ftp connection mode , default is active mode,
supported as the follow
Whether to enable remote host verification for FTP data channels, default is
`true`.
+### control_encoding [string]
+
+Character encoding for the FTP control connection. Default is `UTF-8`.
+
+When file paths contain special characters, spaces, or non-ASCII characters,
keep this value as
+`UTF-8` unless your FTP server requires another control-channel encoding.
+
### custom_filename [boolean]
Whether custom the filename
@@ -136,7 +144,6 @@ When the format in the `file_name_expression` parameter is
`xxxx-${now}` , `file
| d | Day of month |
| H | Hour in day (0-23) |
| m | Minute in hour |
-| schema_evolution_enabled | boolean | no | false
| Enable schema evolution support for CDC pipelines.
When true, ADD/DROP/RENAME/MODIFY column events from the source are applied to
the sink without a job restart. Not supported for binary format. |
| s | Second in minute |
### file_format_type [string]
diff --git a/docs/en/connectors/source/Databend.md
b/docs/en/connectors/source/Databend.md
index d60f4e610b..e2df8c82a4 100644
--- a/docs/en/connectors/source/Databend.md
+++ b/docs/en/connectors/source/Databend.md
@@ -70,20 +70,14 @@ Basic Configuration:
| password | String | Yes | - | Databend database password |
| database | String | No | - | Databend database name, defaults to the
database name specified in the connection URL |
| table | String | No | - | Databend table name |
-| query | String | No | - | Databend query statement, if set will override
database and table settings |
-| fetch_size | Integer | No | 0 | Number of records to fetch from database at
once, set to 0 to use JDBC driver default value |
+| query | String | No | - | Databend query statement. If set, it overrides
database and table settings |
+| sql | String | No | - | Alias-style custom SQL statement. If both `sql` and
`query` are set, `sql` takes precedence |
+| fetch_size | Integer | No | 1 | Number of records to fetch from Databend at
once. Set it higher for large reads |
+| ssl | Boolean | No | false | Whether to use SSL for the Databend connection |
| jdbc_config | Map | No | - | Additional JDBC connection configuration, such
as load balancing strategies |
-Table List Configuration:
-
-| Name | Type | Required | Default Value | Description |
-|------|------|----------|---------------|-------------|
-| database | String | Yes | - | Database name |
-| table | String | Yes | - | Table name |
-| query | String | No | - | Custom query statement |
-| fetch_size | Integer | No | 0 | Number of records to fetch from database at
once |
-
-Note: When this configuration corresponds to a single table, you can flatten
the configuration items from table_list to the outer level.
+You must configure either `sql`, `query`, or both `database` and `table`. The
connector does not
+currently support `table_list`, so configure one Databend source block for
each table.
## Task Examples
@@ -123,6 +117,21 @@ source {
}
```
+### Using SSL
+
+```hocon
+source {
+ Databend {
+ url = "jdbc:databend://databend.example.com:8000/default"
+ username = "root"
+ password = ""
+ sql = "SELECT * FROM default.users"
+ ssl = true
+ fetch_size = 1000
+ }
+}
+```
+
## Related Links
- [Databend Official Website](https://databend.rs/)
@@ -130,4 +139,4 @@ source {
## Changelog
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
diff --git a/docs/en/connectors/source/FtpFile.md
b/docs/en/connectors/source/FtpFile.md
index 87589f5945..faa9517a3b 100644
--- a/docs/en/connectors/source/FtpFile.md
+++ b/docs/en/connectors/source/FtpFile.md
@@ -55,6 +55,7 @@ If you use SeaTunnel Engine, It automatically integrated the
hadoop jar when you
| file_format_type | string | yes | -
|
| connection_mode | string | no | active_local
|
| remote_verification_enabled | boolean | no | true
|
+| control_encoding | string | no | UTF-8
|
| delimiter/field_delimiter | string | no | \001 for text and , for
csv |
| row_delimiter | string | no | \n
|
| read_columns | list | no | -
|
diff --git a/docs/zh/connectors/sink/Databend.md
b/docs/zh/connectors/sink/Databend.md
index 3d8670d7f1..259af9e62c 100644
--- a/docs/zh/connectors/sink/Databend.md
+++ b/docs/zh/connectors/sink/Databend.md
@@ -16,7 +16,7 @@ import ChangeLog from '../changelog/connector-databend.md';
- [x] [精确一次](../../introduction/concepts/connector-v2-features.md)
- [x] [cdc](../../introduction/concepts/connector-v2-features.md)
- [x] [并行度](../../introduction/concepts/connector-v2-features.md)
-- [ ] [timer flush](../../introduction/concepts/connector-v2-features.md)
+- [ ] [定时刷新](../../introduction/concepts/connector-v2-features.md)
## 描述
@@ -155,7 +155,10 @@ sink {
}
```
-### CDC mode
+### CDC 模式
+
+`conflict_key` 用来指定合并更新/删除事件的主键列。只有当 DELETE 事件需要删除
+Databend 中的数据时，才需要设置 `enable_delete = true`。
```hocon
sink {
@@ -166,9 +169,8 @@ sink {
database = "default"
table = "sink_table"
- # Enable CDC mode
+ # 开启 CDC 写入模式
batch_size = 1
- interval = 3
conflict_key = "id"
enable_delete = true
}
@@ -182,4 +184,4 @@ sink {
## Changelog
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
diff --git a/docs/zh/connectors/sink/Email.md b/docs/zh/connectors/sink/Email.md
index ec90dffaa2..473a98fabe 100644
--- a/docs/zh/connectors/sink/Email.md
+++ b/docs/zh/connectors/sink/Email.md
@@ -15,7 +15,7 @@ import ChangeLog from '../changelog/connector-email.md';
## 主要特性
- [ ] [精确一次](../../introduction/concepts/connector-v2-features.md)
-- [ ] [timer flush](../../introduction/concepts/connector-v2-features.md)
+- [ ] [定时刷新](../../introduction/concepts/connector-v2-features.md)
## 选项
@@ -92,8 +92,7 @@ Sink插件常用参数，请参考 [Sink常用选项](../common-options/sink-com
### 发送单表数据到多个收件人
-这个示例来自 Email e2e 任务。示例使用不需要认证的测试 SMTP 服务，并给 `email_to_address`
-中的每个邮箱各发送一封邮件。
+这个示例使用不需要认证的 SMTP 服务，并给 `email_to_address` 中的每个邮箱各发送一封邮件。
```hocon
env {
@@ -132,7 +131,7 @@ sink {
EmailSink {
email_from_address = "[email protected]"
email_to_address = "[email protected],[email protected]"
- email_host = "email-e2e"
+ email_host = "smtp.example.com"
email_transport_protocol = "smtp"
email_smtp_auth = false
email_smtp_port = 3025
@@ -147,7 +146,7 @@ sink {
### 发送多表数据
-Email sink 支持多表输入。在 e2e 任务中，两个上游表会让每个收件人收到两封邮件。
+Email sink 支持多表输入。两个上游表会让每个收件人收到两封邮件。
```hocon
source {
@@ -183,7 +182,7 @@ sink {
EmailSink {
email_from_address = "[email protected]"
email_to_address = "[email protected],[email protected]"
- email_host = "email-e2e"
+ email_host = "smtp.example.com"
email_transport_protocol = "smtp"
email_smtp_auth = false
email_smtp_port = 3025
diff --git a/docs/zh/connectors/sink/FtpFile.md
b/docs/zh/connectors/sink/FtpFile.md
index d3a8660a70..571fcb5843 100644
--- a/docs/zh/connectors/sink/FtpFile.md
+++ b/docs/zh/connectors/sink/FtpFile.md
@@ -35,7 +35,7 @@ import ChangeLog from '../changelog/connector-file-ftp.md';
- [x] excel
- [x] xml
- [x] binary
-- [ ] [timer flush](../../introduction/concepts/connector-v2-features.md)
+- [ ] [定时刷新](../../introduction/concepts/connector-v2-features.md)
## 选项
@@ -49,6 +49,7 @@ import ChangeLog from '../changelog/connector-file-ftp.md';
| tmp_path | string | 是 | /tmp/seatunnel
| 结果文件将首先写入一个临时路径，然后使用 `mv` 命令将临时目录提交到目标目录。需要是一个FTP目录。
|
| connection_mode | string | 否 | active_local
| 目标FTP连接模式
|
| remote_verification_enabled | boolean | 否 | true
| 是否启用FTP数据通道的远程主机验证
|
+| control_encoding | string | 否 | UTF-8
| FTP 控制连接的字符编码，路径包含空格或非 ASCII 字符时很有用
|
| custom_filename | boolean | 否 | false
| 是否需要自定义文件名
|
| file_name_expression | string | 否 | "${transactionId}"
| 仅在 `custom_filename` 为 `true` 时使用
|
| filename_time_format | string | 否 | "yyyy.MM.dd"
| 仅在 `custom_filename` 为 `true` 时使用
|
@@ -110,6 +111,13 @@ import ChangeLog from '../changelog/connector-file-ftp.md';
是否启用FTP数据通道的远程主机验证。默认值为 `true`。
+### control_encoding [string]
+
+FTP 控制连接的字符编码。默认值为 `UTF-8`。
+
+当文件路径包含特殊字符、空格或非 ASCII 字符时，除非 FTP 服务端要求其他控制通道编码，否则建议保持
+`UTF-8`。
+
### custom_filename [boolean]
是否自定义文件名
@@ -137,7 +145,6 @@ import ChangeLog from '../changelog/connector-file-ftp.md';
| d | Day of month |
| H | Hour in day (0-23) |
| m | Minute in hour |
-| schema_evolution_enabled | boolean | 否 | false
| 开启 Schema 演变支持，适用于 CDC 管道。为 true 时，来自上游的
ADD/DROP/RENAME/MODIFY 列事件无需重启作业即可应用到 Sink。不支持 binary 格式。 |
| s | Second in minute |
### file_format_type [string]
@@ -389,4 +396,4 @@ LocalFile {
## 变更日志
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
diff --git a/docs/zh/connectors/source/Databend.md
b/docs/zh/connectors/source/Databend.md
index 1c69453489..2b3c3b25a4 100644
--- a/docs/zh/connectors/source/Databend.md
+++ b/docs/zh/connectors/source/Databend.md
@@ -69,20 +69,14 @@ import ChangeLog from '../changelog/connector-databend.md';
| password | String | 是 | - | Databend 数据库密码 |
| database | String | 否 | - | Databend 数据库名称，默认使用连接 URL 中指定的数据库名 |
| table | String | 否 | - | Databend 表名称 |
-| query | String | 否 | - | Databend 查询语句，如果设置将覆盖 database 和 table 的设置 |
-| fetch_size | Integer | 否 | 0 | 一次从数据库中获取的记录数，设置为0使用JDBC驱动默认值 |
+| query | String | 否 | - | Databend 查询语句。如果设置，会覆盖 database 和 table 的设置 |
+| sql | String | 否 | - | 自定义 SQL 语句。若同时配置 `sql` 和 `query`，优先使用 `sql` |
+| fetch_size | Integer | 否 | 1 | 每次从 Databend 拉取的记录数。读取大量数据时可以适当调大 |
+| ssl | Boolean | 否 | false | 是否使用 SSL 连接 Databend |
| jdbc_config | Map | 否 | - | 额外的 JDBC 连接配置，如加载均衡策略等 |
-表清单配置:
-
-| 名称 | 类型 | 是否必须 | 默认值 | 描述 |
-|------|------|----------|--------|------|
-| database | String | 是 | - | 数据库名称 |
-| table | String | 是 | - | 表名称 |
-| query | String | 否 | - | 自定义查询语句 |
-| fetch_size | Integer | 否 | 0 | 一次从数据库中获取的记录数 |
-
-注意: 当此配置对应于单个表时，您可以将 table_list 中的配置项展平到外层。
+必须配置 `sql`、`query`、或同时配置 `database` 和 `table`。当前连接器不支持
+`table_list`，如果要读多张表，请为每张表分别配置一个 Databend source。
## 任务示例
@@ -122,6 +116,21 @@ source {
}
```
+### 使用 SSL
+
+```hocon
+source {
+ Databend {
+ url = "jdbc:databend://databend.example.com:8000/default"
+ username = "root"
+ password = ""
+ sql = "SELECT * FROM default.users"
+ ssl = true
+ fetch_size = 1000
+ }
+}
+```
+
## 相关链接
- [Databend 官方网站](https://databend.rs/)
@@ -129,4 +138,4 @@ source {
## Changelog
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
diff --git a/docs/zh/connectors/source/FtpFile.md
b/docs/zh/connectors/source/FtpFile.md
index 088f1c5783..9cd145cad0 100644
--- a/docs/zh/connectors/source/FtpFile.md
+++ b/docs/zh/connectors/source/FtpFile.md
@@ -52,7 +52,8 @@ import ChangeLog from '../changelog/connector-file-ftp.md';
| path | string | 是 | - |
| file_format_type | string | 是 | - |
| connection_mode | string | 否 | active_local |
-| remote_verification_enabled | boolean | no | true |
+| remote_verification_enabled | boolean | 否 | true |
+| control_encoding | string | 否 | UTF-8 |
| delimiter/field_delimiter | string | 否 | \001 |
| read_columns | list | 否 | - |
| parse_partition_from_path | boolean | 否 | true |
