This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/dev/pr-13050-a7955d6355a7d7536f8f672db98ef2b8dec114a2 in repository https://gitbox.apache.org/repos/asf/seatunnel.git
commit 20a8300495a5577d0c53d7471fa4467e83ed0756 Author: Jast <[email protected]> AuthorDate: Sat Nov 7 06:38:09 2026 +0000 [Docs] Fix connector and transform doc inaccuracies verified against source code (#13050) Co-authored-by: zhangshenghang <[email protected]> Co-authored-by: jast <[email protected]> --- docs/en/connectors/sink/Clickhouse.md | 7 +++++-- docs/en/connectors/sink/Doris.md | 3 ++- docs/en/connectors/sink/Mysql.md | 13 ++++++++----- docs/en/connectors/sink/Oracle.md | 9 ++++++--- docs/en/connectors/sink/PostgreSql.md | 5 +++-- docs/en/connectors/sink/Redis.md | 2 +- docs/en/connectors/sink/SqlServer.md | 2 +- docs/en/connectors/sink/StarRocks.md | 8 ++++---- docs/en/connectors/sink/TDengine.md | 2 +- docs/en/connectors/source/Clickhouse.md | 6 ++++-- docs/en/connectors/source/Http-Linear.md | 4 ++-- docs/en/connectors/source/InfluxDB.md | 2 +- docs/en/connectors/source/Milvus.md | 2 +- docs/en/connectors/source/Phoenix.md | 2 +- docs/en/connectors/source/Redis.md | 2 +- docs/en/connectors/source/TDengine.md | 4 ++-- docs/en/transforms/copy.md | 2 +- docs/en/transforms/jsonpath.md | 6 +++--- docs/en/transforms/llm.md | 2 +- docs/zh/connectors/sink/Clickhouse.md | 7 +++++-- docs/zh/connectors/sink/Doris.md | 7 ++++--- docs/zh/connectors/sink/Mysql.md | 15 +++++++++------ docs/zh/connectors/sink/Oracle.md | 9 ++++++--- docs/zh/connectors/sink/PostgreSql.md | 5 +++-- docs/zh/connectors/sink/Redis.md | 2 +- docs/zh/connectors/sink/SqlServer.md | 2 +- docs/zh/connectors/sink/StarRocks.md | 12 ++++++------ docs/zh/connectors/sink/TDengine.md | 2 +- docs/zh/connectors/source/Clickhouse.md | 6 ++++-- docs/zh/connectors/source/Http-Linear.md | 4 ++-- docs/zh/connectors/source/InfluxDB.md | 2 +- docs/zh/connectors/source/Milvus.md | 2 +- docs/zh/connectors/source/Phoenix.md | 2 +- docs/zh/connectors/source/TDengine.md | 4 ++-- docs/zh/transforms/copy.md | 2 +- docs/zh/transforms/jsonpath.md | 6 +++--- docs/zh/transforms/llm.md | 2 +- 37 files changed, 100 insertions(+), 74 deletions(-) diff --git a/docs/en/connectors/sink/Clickhouse.md b/docs/en/connectors/sink/Clickhouse.md index 3196726eb63..e6d1623af53 100644 --- a/docs/en/connectors/sink/Clickhouse.md +++ b/docs/en/connectors/sink/Clickhouse.md @@ -62,13 +62,16 @@ They can be downloaded via install-plugin.sh or from the Maven central repositor | SeaTunnel Data Type | Clickhouse Data Type | |---------------------|-----------------------------------------------------------------------------------------------------------------------------------------------| | STRING | String / Int128 / UInt128 / Int256 / UInt256 / Point / Ring / Polygon MultiPolygon | -| INT | Int8 / UInt8 / Int16 / UInt16 / Int32 | +| TINYINT | Int8 | +| SMALLINT | Int16 | +| INT | Int32 | | BIGINT | UInt64 / Int64 / IntervalYear / IntervalQuarter / IntervalMonth / IntervalWeek / IntervalDay / IntervalHour / IntervalMinute / IntervalSecond | | DOUBLE | Float64 | | DECIMAL | Decimal | | FLOAT | Float32 | | DATE | Date | -| TIME | DateTime | +| TIME | String | +| TIMESTAMP | DateTime64 | | ARRAY | Array | | MAP | Map | diff --git a/docs/en/connectors/sink/Doris.md b/docs/en/connectors/sink/Doris.md index 958b35c10fe..18cc768f91f 100644 --- a/docs/en/connectors/sink/Doris.md +++ b/docs/en/connectors/sink/Doris.md @@ -705,7 +705,8 @@ align the target schema explicitly rather than relying on undocumented options. ### What data format does Doris Stream Load use? -Doris Sink uses JSON format for Stream Load by default. Configure it explicitly if needed: +Doris Sink requires an explicit Stream Load `format` in `doris.config` (`json` or `csv`); +there is no default and the job fails to start without it: ```hocon doris.config { diff --git a/docs/en/connectors/sink/Mysql.md b/docs/en/connectors/sink/Mysql.md index 388571c2e2b..0633e88dfe1 100644 --- a/docs/en/connectors/sink/Mysql.md +++ b/docs/en/connectors/sink/Mysql.md @@ -49,8 +49,10 @@ semantics (using XA transaction guarantee). | Mysql Data Type | SeaTunnel Data Type | |-----------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------| -| BIT(1)<br/>INT UNSIGNED | BOOLEAN | -| TINYINT<br/>TINYINT UNSIGNED<br/>SMALLINT<br/>SMALLINT UNSIGNED<br/>MEDIUMINT<br/>MEDIUMINT UNSIGNED<br/>INT<br/>INTEGER<br/>YEAR | INT | +| BIT(1) | BOOLEAN | +| TINYINT | TINYINT | +| TINYINT UNSIGNED<br/>SMALLINT | SMALLINT | +| SMALLINT UNSIGNED<br/>MEDIUMINT<br/>MEDIUMINT UNSIGNED<br/>INT<br/>INTEGER<br/>YEAR | INT | | INT UNSIGNED<br/>INTEGER UNSIGNED<br/>BIGINT | BIGINT | | BIGINT UNSIGNED | DECIMAL(20,0) | | DECIMAL(x,y)(Get the designated column's specified column size.<38) | DECIMAL(x,y) | @@ -61,9 +63,10 @@ semantics (using XA transaction guarantee). | CHAR<br/>VARCHAR<br/>TINYTEXT<br/>MEDIUMTEXT<br/>TEXT<br/>LONGTEXT<br/>JSON | STRING | | DATE | DATE | | TIME | TIME | -| DATETIME<br/>TIMESTAMP | TIMESTAMP | -| TINYBLOB<br/>MEDIUMBLOB<br/>BLOB<br/>LONGBLOB<br/>BINARY<br/>VARBINAR<br/>BIT(n) | BYTES | -| GEOMETRY<br/>UNKNOWN | Not supported yet | +| DATETIME | TIMESTAMP | +| TIMESTAMP | TIMESTAMP_TZ | +| TINYBLOB<br/>MEDIUMBLOB<br/>BLOB<br/>LONGBLOB<br/>BINARY<br/>VARBINAR<br/>BIT(n)<br/>GEOMETRY | BYTES | +| UNKNOWN | Not supported yet | ## Sink Options diff --git a/docs/en/connectors/sink/Oracle.md b/docs/en/connectors/sink/Oracle.md index e99db4a64ac..2923fb483fc 100644 --- a/docs/en/connectors/sink/Oracle.md +++ b/docs/en/connectors/sink/Oracle.md @@ -55,13 +55,16 @@ semantics (using XA transaction guarantee). | FLOAT | DECIMAL(38, 18) | | NUMBER(precision <= 9, scale == 0) | INT | | NUMBER(9 < precision <= 18, scale == 0) | BIGINT | -| NUMBER(18 < precision, scale == 0) | DECIMAL(38, 0) | -| NUMBER(scale != 0) | DECIMAL(38, 18) | +| NUMBER(18 < precision < 38, scale == 0) | DECIMAL(precision, 0) | +| NUMBER(precision >= 38, scale == 0) | DECIMAL(38, 0) | +| NUMBER(scale > 0) | DECIMAL(precision, min(scale, 18)) | +| NUMBER(scale < 0) | Normalized to `DECIMAL(precision - scale, 0)`, or `LONG`/`INT`/`BOOLEAN` when decimal narrowing applies. Missing precision/scale default to 38/18. | | BINARY_DOUBLE | DOUBLE | | BINARY_FLOAT<br/>REAL | FLOAT | | CHAR<br/>NCHAR<br/>NVARCHAR2<br/>VARCHAR2<br/>LONG<br/>ROWID<br/>NCLOB<br/>CLOB<br/> | STRING | | DATE | TIMESTAMP | -| TIMESTAMP<br/>TIMESTAMP WITH LOCAL TIME ZONE | TIMESTAMP | +| TIMESTAMP<br/>TIMESTAMP(s) | TIMESTAMP(s) | +| TIMESTAMP WITH TIME ZONE<br/>TIMESTAMP WITH LOCAL TIME ZONE | TIMESTAMP_TZ | | BLOB<br/>RAW<br/>LONG RAW<br/>BFILE | BYTES | ## Options diff --git a/docs/en/connectors/sink/PostgreSql.md b/docs/en/connectors/sink/PostgreSql.md index e9916f3484d..47d28fc9766 100644 --- a/docs/en/connectors/sink/PostgreSql.md +++ b/docs/en/connectors/sink/PostgreSql.md @@ -65,8 +65,9 @@ Exactly-once requires XA transaction support in the target database. Set | BOOL<br/> | BOOLEAN | | _BOOL<br/> | ARRAY<BOOLEAN> | | BYTEA<br/> | BYTES | -| _BYTEA<br/> | ARRAY<TINYINT> | -| INT2<br/>SMALLSERIAL<br/>INT4<br/>SERIAL<br/> | INT | +| _BYTEA<br/> | NOT SUPPORTED YET | +| INT2<br/>SMALLSERIAL<br/> | SMALLINT | +| INT4<br/>SERIAL<br/> | INT | | _INT2<br/>_INT4<br/> | ARRAY<INT> | | INT8<br/>BIGSERIAL<br/> | BIGINT | | _INT8<br/> | ARRAY<BIGINT> | diff --git a/docs/en/connectors/sink/Redis.md b/docs/en/connectors/sink/Redis.md index ddebaf9d80a..a713e986277 100644 --- a/docs/en/connectors/sink/Redis.md +++ b/docs/en/connectors/sink/Redis.md @@ -103,7 +103,7 @@ the legacy `{field}` style are supported. - `KEY` and `STRING`: write one Redis string value for each row. Later rows overwrite earlier rows with the same key. - `HASH`: write one or more fields into a Redis hash. Configure `hash_key_field` to choose the hash field name. -- `LIST`: append each row value to a Redis list. +- `LIST`: add each row value to a Redis list. Values are pushed to the head of the list (`LPUSH`), so they are returned in reverse insertion order by `LRANGE`. - `SET`: add each row value to a Redis set. - `ZSET`: add each row value to a Redis sorted set with score `1`. diff --git a/docs/en/connectors/sink/SqlServer.md b/docs/en/connectors/sink/SqlServer.md index 760e663d73d..f8ea96ceb1f 100644 --- a/docs/en/connectors/sink/SqlServer.md +++ b/docs/en/connectors/sink/SqlServer.md @@ -58,7 +58,7 @@ semantics (using XA transaction guarantee). | TINYINT<br/>SMALLINT | SHORT | | INTEGER | INT | | BIGINT | LONG | -| DECIMAL<br />NUMERIC<br />MONEY<br />SMALLMONEY | DECIMAL((Get the designated column's specified column size)+1,<br/>(Gets the designated column's number of digits to right of the <br />decimal point.))) | +| DECIMAL<br />NUMERIC<br />MONEY<br />SMALLMONEY | DECIMAL(precision, scale) | | REAL | FLOAT | | FLOAT | DOUBLE | | CHAR<br />NCHAR<br />VARCHAR<br />NTEXT<br />NVARCHAR<br />TEXT | STRING | diff --git a/docs/en/connectors/sink/StarRocks.md b/docs/en/connectors/sink/StarRocks.md index f86f0b0946f..6d26e96bc2c 100644 --- a/docs/en/connectors/sink/StarRocks.md +++ b/docs/en/connectors/sink/StarRocks.md @@ -227,12 +227,12 @@ sink { | FLOAT | FLOAT | | DOUBLE | DOUBLE | | DECIMAL | DECIMAL | -| DATE | STRING | +| DATE | DATE | | TIME | STRING | -| DATETIME | STRING | +| DATETIME | TIMESTAMP | | STRING | STRING | -| ARRAY | STRING | -| MAP | STRING | +| ARRAY | ARRAY | +| MAP | MAP | | BYTES | STRING | #### Supported import data formats diff --git a/docs/en/connectors/sink/TDengine.md b/docs/en/connectors/sink/TDengine.md index c508060ddfc..b3316f3ae84 100644 --- a/docs/en/connectors/sink/TDengine.md +++ b/docs/en/connectors/sink/TDengine.md @@ -29,7 +29,7 @@ sub table name. ## Key features -- [x] [exactly-once](../../introduction/concepts/connector-v2-features.md) +- [ ] [exactly-once](../../introduction/concepts/connector-v2-features.md) - [ ] [cdc](../../introduction/concepts/connector-v2-features.md) - [x] [support multiple table write](../../introduction/concepts/connector-v2-features.md) - [ ] [timer flush](../../introduction/concepts/connector-v2-features.md) diff --git a/docs/en/connectors/source/Clickhouse.md b/docs/en/connectors/source/Clickhouse.md index 1ebc1766f7a..55853c74b7b 100644 --- a/docs/en/connectors/source/Clickhouse.md +++ b/docs/en/connectors/source/Clickhouse.md @@ -40,13 +40,15 @@ They can be downloaded via install-plugin.sh or from the Maven central repositor | Clickhouse Data Type | SeaTunnel Data Type | |-----------------------------------------------------------------------------------------------------------------------------------------------|---------------------| | String / Int128 / UInt128 / Int256 / UInt256 / Point / Ring / Polygon MultiPolygon | STRING | -| Int8 / UInt8 / Int16 / UInt16 / Int32 | INT | +| Int8 | TINYINT | +| UInt8 / Int16 | SMALLINT | +| UInt16 / Int32 | INT | | UInt64 / Int64 / IntervalYear / IntervalQuarter / IntervalMonth / IntervalWeek / IntervalDay / IntervalHour / IntervalMinute / IntervalSecond | BIGINT | | Float64 | DOUBLE | | Decimal | DECIMAL | | Float32 | FLOAT | | Date | DATE | -| DateTime | TIME | +| DateTime | TIMESTAMP | | Array | ARRAY | | Map | MAP | diff --git a/docs/en/connectors/source/Http-Linear.md b/docs/en/connectors/source/Http-Linear.md index 612c87d8e7c..b6d42649851 100644 --- a/docs/en/connectors/source/Http-Linear.md +++ b/docs/en/connectors/source/Http-Linear.md @@ -27,7 +27,7 @@ The Linear source connector reads data from the [Linear](https://linear.app/) Gr | headers | Map | No | - | Extra HTTP headers. Any `Authorization` entry here is always replaced by the value of `api_key`. | | params | Map | No | - | Query parameters sent with the request. | | body | String | No | - | HTTP request body. Usually used with `method = "POST"` to send a GraphQL query. | -| format | String | No | text | Response format. Use `json` when reading the GraphQL response into a SeaTunnel schema; use `text` to return the raw response as `content`. The shared HTTP option also accepts `binary`, but `binary` is not supported by the `Linear` source and `binary_chunk_size` has no effect here. | +| format | String | No | text | Response format. Use `json` when reading the GraphQL response into a SeaTunnel schema; use `text` to return the raw response as `content`. Only response-format options are used by the Linear reader: the shared HTTP `binary` format and `binary_chunk_size` are **not** applied here (the Linear reader always deserializes JSON or text), so do not rely on them. | | schema | Config | No | - | Output schema. Required when `format = "json"`. | | schema.fields | Config | No | - | Field names and SeaTunnel data types used to parse the JSON response. | | content_field | String | No | - | JSONPath used to select a nested part of the response before parsing it with `schema`, for example `$.data.issues.nodes.*`. | @@ -54,7 +54,7 @@ The Linear source connector reads data from the [Linear](https://linear.app/) Gr - Set `format = "json"` and configure `schema` when you want typed SeaTunnel rows. - Use `content_field` to unwrap the GraphQL envelope, for example `$.data.issues.nodes.*`, before parsing with `schema`. - Use `json_field` only when each output field needs its own JSONPath expression. -- Pagination is not supported by this connector. Each job reads only what a single request returns; the HTTP `pageing` block and its options have no effect here. +- Pagination is **not** effective in Linear mode: the reader issues a single request and ignores the HTTP `pageing` block, so a multi-page API will only return its first page. Reading more data than one request returns requires a separate code change, not configuration. - `api_key` is sent verbatim as the `Authorization` header value and always overrides any `Authorization` entry in `headers`. Put only other custom headers in `headers`. ## Task Examples diff --git a/docs/en/connectors/source/InfluxDB.md b/docs/en/connectors/source/InfluxDB.md index 6e5fbbd0c5f..205b58c4a22 100644 --- a/docs/en/connectors/source/InfluxDB.md +++ b/docs/en/connectors/source/InfluxDB.md @@ -20,7 +20,7 @@ Use `tables_configs` to read multiple queries, databases, and output schemas thr - [x] [batch](../../introduction/concepts/connector-v2-features.md) - [ ] [stream](../../introduction/concepts/connector-v2-features.md) -- [x] [exactly-once](../../introduction/concepts/connector-v2-features.md) +- [ ] [exactly-once](../../introduction/concepts/connector-v2-features.md) - [x] [column projection](../../introduction/concepts/connector-v2-features.md) - [x] [parallelism](../../introduction/concepts/connector-v2-features.md) - [x] [support user-defined split](../../introduction/concepts/connector-v2-features.md) diff --git a/docs/en/connectors/source/Milvus.md b/docs/en/connectors/source/Milvus.md index 28235f75282..5763701de91 100644 --- a/docs/en/connectors/source/Milvus.md +++ b/docs/en/connectors/source/Milvus.md @@ -27,7 +27,7 @@ Common use cases: ## Key Features - [x] [batch](../../introduction/concepts/connector-v2-features.md) -- [x] [exactly-once](../../introduction/concepts/connector-v2-features.md) +- [ ] [exactly-once](../../introduction/concepts/connector-v2-features.md) - [x] [column projection](../../introduction/concepts/connector-v2-features.md) - [x] [parallelism](../../introduction/concepts/connector-v2-features.md) - [x] [support multiple table read](../../introduction/concepts/connector-v2-features.md) diff --git a/docs/en/connectors/source/Phoenix.md b/docs/en/connectors/source/Phoenix.md index afe158d0d9b..07de5c8e732 100644 --- a/docs/en/connectors/source/Phoenix.md +++ b/docs/en/connectors/source/Phoenix.md @@ -29,7 +29,7 @@ There are two ways to connect Phoenix through Java JDBC: ## Key features - [x] [batch](../../introduction/concepts/connector-v2-features.md) -- [x] [stream](../../introduction/concepts/connector-v2-features.md) +- [ ] [stream](../../introduction/concepts/connector-v2-features.md) - [ ] [exactly-once](../../introduction/concepts/connector-v2-features.md) - [x] [column projection](../../introduction/concepts/connector-v2-features.md) - [ ] [parallelism](../../introduction/concepts/connector-v2-features.md) diff --git a/docs/en/connectors/source/Redis.md b/docs/en/connectors/source/Redis.md index ed81dfd978e..da6fd95fc7e 100644 --- a/docs/en/connectors/source/Redis.md +++ b/docs/en/connectors/source/Redis.md @@ -410,7 +410,7 @@ Redis { } ``` -read string type keys write append to list +read string type keys and write them to a list (LPUSH) ```hocon source { diff --git a/docs/en/connectors/source/TDengine.md b/docs/en/connectors/source/TDengine.md index fba19132669..532f6120435 100644 --- a/docs/en/connectors/source/TDengine.md +++ b/docs/en/connectors/source/TDengine.md @@ -27,8 +27,8 @@ TDengine sink when writing rows back to TDengine. - [x] [batch](../../introduction/concepts/connector-v2-features.md) - [ ] [stream](../../introduction/concepts/connector-v2-features.md) -- [x] [exactly-once](../../introduction/concepts/connector-v2-features.md) -- [x] [column projection](../../introduction/concepts/connector-v2-features.md) +- [ ] [exactly-once](../../introduction/concepts/connector-v2-features.md) +- [ ] [column projection](../../introduction/concepts/connector-v2-features.md) - [x] [parallelism](../../introduction/concepts/connector-v2-features.md) - [ ] [support user-defined split](../../introduction/concepts/connector-v2-features.md) diff --git a/docs/en/transforms/copy.md b/docs/en/transforms/copy.md index 7daccdc6fb4..8c54b454814 100644 --- a/docs/en/transforms/copy.md +++ b/docs/en/transforms/copy.md @@ -10,7 +10,7 @@ Copy a field to a new field. | name | type | required | default value | |--------|--------|----------|---------------| -| fields | Object | yes (or the deprecated `src_field`+`dest_field` pair) | | +| fields | Map | yes (or the deprecated `src_field`+`dest_field` pair) | | | src_field | String | no | | | dest_field | String | no | | diff --git a/docs/en/transforms/jsonpath.md b/docs/en/transforms/jsonpath.md index 69149b86f35..2621408c812 100644 --- a/docs/en/transforms/jsonpath.md +++ b/docs/en/transforms/jsonpath.md @@ -234,9 +234,9 @@ transform { Then the data result table `fake1` will like this -| name | age | col | other | -|------|-----|----------|-------| -| a | 18 | ["a",18] | ... | +| col | other | name | age | +|----------|-------|------|-----| +| ["a",18] | ... | a | 18 | ## Configure error data handle way diff --git a/docs/en/transforms/llm.md b/docs/en/transforms/llm.md index 1b7f169092c..80c7ba10186 100644 --- a/docs/en/transforms/llm.md +++ b/docs/en/transforms/llm.md @@ -19,7 +19,7 @@ more. | prompt | string | yes | | | inference_columns | list | no | | | model | string | yes | | -| api_key | string | yes | | +| api_key | string | conditional (required for OPENAI, DOUBAO, MICROSOFT, QIANFAN; QIANFAN also needs `secret_key` and `oauth_path`) | | | api_path | string | no | | | custom_config | map | no | | | custom_response_parse | string | no | | diff --git a/docs/zh/connectors/sink/Clickhouse.md b/docs/zh/connectors/sink/Clickhouse.md index eeccde24628..15de70d7a65 100644 --- a/docs/zh/connectors/sink/Clickhouse.md +++ b/docs/zh/connectors/sink/Clickhouse.md @@ -56,13 +56,16 @@ Clickhouse source 和 ClickhouseFile sink 的行为不变。 | SeaTunnel 数据类型 | Clickhouse 数据类型 | |----------------|-----------------------------------------------------------------------------------------------------------------------------------------------| | STRING | String / Int128 / UInt128 / Int256 / UInt256 / Point / Ring / Polygon MultiPolygon | -| INT | Int8 / UInt8 / Int16 / UInt16 / Int32 | +| TINYINT | Int8 | +| SMALLINT | Int16 | +| INT | Int32 | | BIGINT | UInt64 / Int64 / IntervalYear / IntervalQuarter / IntervalMonth / IntervalWeek / IntervalDay / IntervalHour / IntervalMinute / IntervalSecond | | DOUBLE | Float64 | | DECIMAL | Decimal | | FLOAT | Float32 | | DATE | Date | -| TIME | DateTime | +| TIME | String | +| TIMESTAMP | DateTime64 | | ARRAY | Array | | MAP | Map | diff --git a/docs/zh/connectors/sink/Doris.md b/docs/zh/connectors/sink/Doris.md index 54593709487..52ab38e39f1 100644 --- a/docs/zh/connectors/sink/Doris.md +++ b/docs/zh/connectors/sink/Doris.md @@ -115,7 +115,7 @@ Doris Sink连接器的内部实现是通过stream load批量缓存和导入的 默认模板: ```sql -CREATE TABLE IF NOT EXISTS `${database}`.`${table_name}` ( +CREATE TABLE IF NOT EXISTS `${database}`.`${table}` ( ${rowtype_primary_key}, ${rowtype_fields} ) ENGINE=OLAP @@ -133,7 +133,7 @@ DISTRIBUTED BY HASH (${rowtype_primary_key}) 如果模板中填写了自定义字段,例如添加 id 字段 ```sql -CREATE TABLE IF NOT EXISTS `${database}`.`${table_name}` +CREATE TABLE IF NOT EXISTS `${database}`.`${table}` ( id, ${rowtype_fields} @@ -699,7 +699,8 @@ Doris 通过 Stream Load label 去重,防止重复提交。开启 2PC 后重 ### Doris Stream Load 使用什么数据格式? -默认使用 JSON 格式,可按需显式配置: +Doris Sink 必须在 `doris.config` 中显式配置 Stream Load 的 `format`(`json` 或 `csv`); +该项没有默认值,缺少时作业会启动失败: ```hocon doris.config { diff --git a/docs/zh/connectors/sink/Mysql.md b/docs/zh/connectors/sink/Mysql.md index 1943ba5e188..7e6eb7b9967 100644 --- a/docs/zh/connectors/sink/Mysql.md +++ b/docs/zh/connectors/sink/Mysql.md @@ -49,21 +49,24 @@ import ChangeLog from '../changelog/connector-jdbc.md'; | Mysql 数据类型 | SeaTunnel 数据类型 | |-----------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------| -| BIT(1)<br/>INT UNSIGNED | BOOLEAN | -| TINYINT<br/>TINYINT UNSIGNED<br/>SMALLINT<br/>SMALLINT UNSIGNED<br/>MEDIUMINT<br/>MEDIUMINT UNSIGNED<br/>INT<br/>INTEGER<br/>YEAR | INT | +| BIT(1) | BOOLEAN | +| TINYINT | TINYINT | +| TINYINT UNSIGNED<br/>SMALLINT | SMALLINT | +| SMALLINT UNSIGNED<br/>MEDIUMINT<br/>MEDIUMINT UNSIGNED<br/>INT<br/>INTEGER<br/>YEAR | INT | | INT UNSIGNED<br/>INTEGER UNSIGNED<br/>BIGINT | BIGINT | | BIGINT UNSIGNED | DECIMAL(20,0) | | DECIMAL(x,y)(获取指定列的列大小<38) | DECIMAL(x,y) | | DECIMAL(x,y)(获取指定列的列大小>38) | DECIMAL(38,18) | -| DECIMAL UNSIGNED | DECIMAL((DECIMAL((获取指定列的列大小)+1,<br/>(获取指定列的小数点右侧的位数))) | +| DECIMAL UNSIGNED | DECIMAL((获取指定列的列大小)+1,<br/>(获取指定列的小数点右侧的位数))) | | FLOAT<br/>FLOAT UNSIGNED | FLOAT | | DOUBLE<br/>DOUBLE UNSIGNED | DOUBLE | | CHAR<br/>VARCHAR<br/>TINYTEXT<br/>MEDIUMTEXT<br/>TEXT<br/>LONGTEXT<br/>JSON | STRING | | DATE | DATE | | TIME | TIME | -| DATETIME<br/>TIMESTAMP | TIMESTAMP | -| TINYBLOB<br/>MEDIUMBLOB<br/>BLOB<br/>LONGBLOB<br/>BINARY<br/>VARBINAR<br/>BIT(n) | BYTES | -| GEOMETRY<br/>UNKNOWN | Not supported yet | +| DATETIME | TIMESTAMP | +| TIMESTAMP | TIMESTAMP_TZ | +| TINYBLOB<br/>MEDIUMBLOB<br/>BLOB<br/>LONGBLOB<br/>BINARY<br/>VARBINAR<br/>BIT(n)<br/>GEOMETRY | BYTES | +| UNKNOWN | Not supported yet | ## Sink 参数 diff --git a/docs/zh/connectors/sink/Oracle.md b/docs/zh/connectors/sink/Oracle.md index 6e8e2cecf07..58559c812c5 100644 --- a/docs/zh/connectors/sink/Oracle.md +++ b/docs/zh/connectors/sink/Oracle.md @@ -54,13 +54,16 @@ import ChangeLog from '../changelog/connector-jdbc.md'; | FLOAT | DECIMAL(38, 18) | | NUMBER(precision <= 9, scale == 0) | INT | | NUMBER(9 < precision <= 18, scale == 0) | BIGINT | -| NUMBER(18 < precision, scale == 0) | DECIMAL(38, 0) | -| NUMBER(scale != 0) | DECIMAL(38, 18) | +| NUMBER(18 < precision < 38, scale == 0) | DECIMAL(precision, 0) | +| NUMBER(precision >= 38, scale == 0) | DECIMAL(38, 0) | +| NUMBER(scale > 0) | DECIMAL(precision, min(scale, 18)) | +| NUMBER(scale < 0) | 规范化为 `DECIMAL(precision - scale, 0)`;启用 decimal narrowing 时为 `LONG`/`INT`/`BOOLEAN`。未指定 precision/scale 时默认按 38/18 处理。 | | BINARY_DOUBLE | DOUBLE | | BINARY_FLOAT<br/>REAL | FLOAT | | CHAR<br/>NCHAR<br/>NVARCHAR2<br/>VARCHAR2<br/>LONG<br/>ROWID<br/>NCLOB<br/>CLOB<br/> | STRING | | DATE | TIMESTAMP | -| TIMESTAMP<br/>TIMESTAMP WITH LOCAL TIME ZONE | TIMESTAMP | +| TIMESTAMP<br/>TIMESTAMP(s) | TIMESTAMP(s) | +| TIMESTAMP WITH TIME ZONE<br/>TIMESTAMP WITH LOCAL TIME ZONE | TIMESTAMP_TZ | | BLOB<br/>RAW<br/>LONG RAW<br/>BFILE | BYTES | ## 参数 diff --git a/docs/zh/connectors/sink/PostgreSql.md b/docs/zh/connectors/sink/PostgreSql.md index ef6d9517b69..eb226023642 100644 --- a/docs/zh/connectors/sink/PostgreSql.md +++ b/docs/zh/connectors/sink/PostgreSql.md @@ -60,8 +60,9 @@ PostgreSQL Sink 连接器通过 JDBC 驱动将行写入 PostgreSQL 数据库。 | BOOL<br/> | BOOLEAN | | _BOOL<br/> | ARRAY<BOOLEAN> | | BYTEA<br/> | BYTES | -| _BYTEA<br/> | ARRAY<TINYINT> | -| INT2<br/>SMALLSERIAL<br/>INT4<br/>SERIAL<br/> | INT | +| _BYTEA<br/> | 目前不支持 | +| INT2<br/>SMALLSERIAL<br/> | SMALLINT | +| INT4<br/>SERIAL<br/> | INT | | _INT2<br/>_INT4<br/> | ARRAY<INT> | | INT8<br/>BIGSERIAL<br/> | BIGINT | | _INT8<br/> | ARRAY<BIGINT> | diff --git a/docs/zh/connectors/sink/Redis.md b/docs/zh/connectors/sink/Redis.md index 64c5bd7edb4..a68ba95d463 100644 --- a/docs/zh/connectors/sink/Redis.md +++ b/docs/zh/connectors/sink/Redis.md @@ -97,7 +97,7 @@ Zeta 的 `--dry-run connect` 会校验 Redis 是否可达以及是否接受配 - `KEY` 和 `STRING`:每行写入一个 Redis 字符串。同一个 key 下,后写入的数据会覆盖先写入的数据。 - `HASH`:写入 Redis hash。配置 `hash_key_field` 可以指定 hash field。 -- `LIST`:把每行数据追加到 Redis list。 +- `LIST`:把每行数据写入 Redis list。数据通过 `LPUSH` 插入到列表头部,因此 `LRANGE` 读取顺序与写入顺序相反。 - `SET`:把每行数据加入 Redis set。 - `ZSET`:把每行数据加入 Redis zset,score 固定为 `1`。 diff --git a/docs/zh/connectors/sink/SqlServer.md b/docs/zh/connectors/sink/SqlServer.md index 495f32741fe..7b872cff078 100644 --- a/docs/zh/connectors/sink/SqlServer.md +++ b/docs/zh/connectors/sink/SqlServer.md @@ -56,7 +56,7 @@ import ChangeLog from '../changelog/connector-jdbc.md'; | TINYINT<br/>SMALLINT | SHORT | | INTEGER | INT | | BIGINT | LONG | -| DECIMAL<br />NUMERIC<br />MONEY<br />SMALLMONEY | DECIMAL((获取指定列的列大小)+1,<br/>(获取指定列的小数点右侧的位数))) | +| DECIMAL<br />NUMERIC<br />MONEY<br />SMALLMONEY | DECIMAL(precision, scale)(精度与小数位取自源列) | | REAL | FLOAT | | FLOAT | DOUBLE | | CHAR<br />NCHAR<br />VARCHAR<br />NTEXT<br />NVARCHAR<br />TEXT | STRING | diff --git a/docs/zh/connectors/sink/StarRocks.md b/docs/zh/connectors/sink/StarRocks.md index ae0303daa94..2441971f23e 100644 --- a/docs/zh/connectors/sink/StarRocks.md +++ b/docs/zh/connectors/sink/StarRocks.md @@ -66,7 +66,7 @@ StarRocks数据接收器使用模板,在需求需要的时候也可以修改 默认模板如下: ```sql -CREATE TABLE IF NOT EXISTS `${database}`.`${table_name}` ( +CREATE TABLE IF NOT EXISTS `${database}`.`${table}` ( ${rowtype_primary_key}, ${rowtype_fields} ) ENGINE=OLAP @@ -80,7 +80,7 @@ DISTRIBUTED BY HASH (${rowtype_primary_key})PROPERTIES ( 在模板中添加自定义字段,比如说加上`id`字段的修改模板如下: ```sql -CREATE TABLE IF NOT EXISTS `${database}`.`${table_name}` +CREATE TABLE IF NOT EXISTS `${database}`.`${table}` ( id, ${rowtype_fields} @@ -222,12 +222,12 @@ sink { | FLOAT | FLOAT | | DOUBLE | DOUBLE | | DECIMAL | DECIMAL | -| DATE | STRING | +| DATE | DATE | | TIME | STRING | -| DATETIME | STRING | +| DATETIME | TIMESTAMP | | STRING | STRING | -| ARRAY | STRING | -| MAP | STRING | +| ARRAY | ARRAY | +| MAP | MAP | | BYTES | STRING | #### 支持导入的数据格式 diff --git a/docs/zh/connectors/sink/TDengine.md b/docs/zh/connectors/sink/TDengine.md index 0ed60e53130..d12cc225a50 100644 --- a/docs/zh/connectors/sink/TDengine.md +++ b/docs/zh/connectors/sink/TDengine.md @@ -22,7 +22,7 @@ import ChangeLog from '../changelog/connector-tdengine.md'; ## 主要特性 -- [x] [精确一次](../../introduction/concepts/connector-v2-features.md) +- [ ] [精确一次](../../introduction/concepts/connector-v2-features.md) - [ ] [变更数据捕获](../../introduction/concepts/connector-v2-features.md) - [x] [支持多表写入](../../introduction/concepts/connector-v2-features.md) - [ ] [定时刷新](../../introduction/concepts/connector-v2-features.md) diff --git a/docs/zh/connectors/source/Clickhouse.md b/docs/zh/connectors/source/Clickhouse.md index a061a4841ce..4296482de06 100644 --- a/docs/zh/connectors/source/Clickhouse.md +++ b/docs/zh/connectors/source/Clickhouse.md @@ -39,13 +39,15 @@ import ChangeLog from '../changelog/connector-clickhouse.md'; | Clickhouse 数据类型 | SeaTunnel 数据类型 | |-----------------------------------------------------------------------------------------------------------------------------------------------|---------------------| | String / Int128 / UInt128 / Int256 / UInt256 / Point / Ring / Polygon MultiPolygon | STRING | -| Int8 / UInt8 / Int16 / UInt16 / Int32 | INT | +| Int8 | TINYINT | +| UInt8 / Int16 | SMALLINT | +| UInt16 / Int32 | INT | | UInt64 / Int64 / IntervalYear / IntervalQuarter / IntervalMonth / IntervalWeek / IntervalDay / IntervalHour / IntervalMinute / IntervalSecond | BIGINT | | Float64 | DOUBLE | | Decimal | DECIMAL | | Float32 | FLOAT | | Date | DATE | -| DateTime | TIME | +| DateTime | TIMESTAMP | | Array | ARRAY | | Map | MAP | diff --git a/docs/zh/connectors/source/Http-Linear.md b/docs/zh/connectors/source/Http-Linear.md index cc093dc37c1..fbaf7c9affb 100644 --- a/docs/zh/connectors/source/Http-Linear.md +++ b/docs/zh/connectors/source/Http-Linear.md @@ -27,7 +27,7 @@ Linear 源连接器用于读取 [Linear](https://linear.app/) GraphQL API 数据 | headers | Map | 否 | - | 额外的 HTTP 请求头。此处任何 `Authorization` 条目都会被 `api_key` 的值覆盖。 | | params | Map | 否 | - | 请求附带的查询参数。 | | body | String | 否 | - | HTTP 请求体,通常与 `method = "POST"` 一起使用以发送 GraphQL 查询。 | -| format | String | 否 | text | 响应格式,`json` 时需要配合 `schema`;`text` 时返回原始响应。共享的 HTTP 选项也接受 `binary`,但 `Linear` 源连接器不支持 `binary`,`binary_chunk_size` 在此处无效。 | +| format | String | 否 | text | 响应格式,`json` 时需要配合 `schema`;`text` 时返回原始响应。Linear 读取器只使用响应格式相关选项:共享 HTTP 选项中的 `binary` 格式与 `binary_chunk_size` **在此不生效**(Linear 读取器始终按 JSON 或文本反序列化),请勿依赖它们。 | | schema | Config | 否 | - | 输出数据结构,`format = "json"` 时必填。 | | schema.fields | Config | 否 | - | 字段名与 SeaTunnel 数据类型,用于解析 JSON 响应。 | | content_field | String | 否 | - | 在 `schema` 解析之前先通过 JSONPath 抽取一段 JSON,例如 `$.data.issues.nodes.*`。 | @@ -54,7 +54,7 @@ Linear 源连接器用于读取 [Linear](https://linear.app/) GraphQL API 数据 - 需要带类型的 SeaTunnel 数据行时,请设置 `format = "json"` 并配置 `schema`。 - 响应带有 GraphQL 外层结构时,先用 `content_field`(例如 `$.data.issues.nodes.*`)解包,再交给 `schema` 解析。 - 仅当每个输出字段需要独立的 JSONPath 时才使用 `json_field`。 -- 该连接器不支持分页,每个作业只读取单次请求返回的数据;HTTP 的 `pageing` 配置块及其选项在此无效。 +- 分页在 Linear 模式下**不生效**:读取器只发起单次请求并忽略 HTTP 的 `pageing` 配置块,因此多页接口只会返回第一页。读取超过单次请求的数据量需要独立的代码改动,而非配置。 - `api_key` 会原样作为 `Authorization` 请求头的值发送,并始终覆盖 `headers` 中的任何 `Authorization` 条目,其余自定义请求头放在 `headers` 中即可。 ## 任务示例 diff --git a/docs/zh/connectors/source/InfluxDB.md b/docs/zh/connectors/source/InfluxDB.md index 0f78b412733..1fb41cff298 100644 --- a/docs/zh/connectors/source/InfluxDB.md +++ b/docs/zh/connectors/source/InfluxDB.md @@ -20,7 +20,7 @@ import ChangeLog from '../changelog/connector-influxdb.md'; - [x] [批处理](../../introduction/concepts/connector-v2-features.md) - [ ] [流处理](../../introduction/concepts/connector-v2-features.md) -- [x] [精确一次](../../introduction/concepts/connector-v2-features.md) +- [ ] [精确一次](../../introduction/concepts/connector-v2-features.md) - [x] [列投影](../../introduction/concepts/connector-v2-features.md) - [x] [并行度](../../introduction/concepts/connector-v2-features.md) - [x] [支持用户自定义切分](../../introduction/concepts/connector-v2-features.md) diff --git a/docs/zh/connectors/source/Milvus.md b/docs/zh/connectors/source/Milvus.md index 70ae8c0cd2d..800c61f462f 100644 --- a/docs/zh/connectors/source/Milvus.md +++ b/docs/zh/connectors/source/Milvus.md @@ -27,7 +27,7 @@ Milvus 源连接器用于从 Milvus 或 Zilliz Cloud 读取数据。它可以读 ## 主要特性 - [x] [批处理](../../introduction/concepts/connector-v2-features.md) -- [x] [精确一次](../../introduction/concepts/connector-v2-features.md) +- [ ] [精确一次](../../introduction/concepts/connector-v2-features.md) - [x] [列投影](../../introduction/concepts/connector-v2-features.md) - [x] [并行读取](../../introduction/concepts/connector-v2-features.md) - [x] [支持多表读取](../../introduction/concepts/connector-v2-features.md) diff --git a/docs/zh/connectors/source/Phoenix.md b/docs/zh/connectors/source/Phoenix.md index d237600206d..7f623eb6b4c 100644 --- a/docs/zh/connectors/source/Phoenix.md +++ b/docs/zh/connectors/source/Phoenix.md @@ -29,7 +29,7 @@ import ChangeLog from '../changelog/connector-jdbc.md'; ## 关键特性 - [x] [批处理](../../introduction/concepts/connector-v2-features.md) -- [x] [流处理](../../introduction/concepts/connector-v2-features.md) +- [ ] [流处理](../../introduction/concepts/connector-v2-features.md) - [ ] [精确一次](../../introduction/concepts/connector-v2-features.md) - [x] [列投影](../../introduction/concepts/connector-v2-features.md) diff --git a/docs/zh/connectors/source/TDengine.md b/docs/zh/connectors/source/TDengine.md index a18a77f9297..99ef12f513e 100644 --- a/docs/zh/connectors/source/TDengine.md +++ b/docs/zh/connectors/source/TDengine.md @@ -22,8 +22,8 @@ import ChangeLog from '../changelog/connector-tdengine.md'; - [x] [批处理](../../introduction/concepts/connector-v2-features.md) - [ ] [流式](../../introduction/concepts/connector-v2-features.md) -- [x] [精确一次](../../introduction/concepts/connector-v2-features.md) -- [x] [列投影](../../introduction/concepts/connector-v2-features.md) +- [ ] [精确一次](../../introduction/concepts/connector-v2-features.md) +- [ ] [列投影](../../introduction/concepts/connector-v2-features.md) - [x] [并行度](../../introduction/concepts/connector-v2-features.md) - [ ] [支持用户自定义分片](../../introduction/concepts/connector-v2-features.md) diff --git a/docs/zh/transforms/copy.md b/docs/zh/transforms/copy.md index 0a9afd8d0b2..7631723d1c3 100644 --- a/docs/zh/transforms/copy.md +++ b/docs/zh/transforms/copy.md @@ -10,7 +10,7 @@ | 名称 | 类型 | 是否必须 | 默认值 | |--------|--------|------|-----| -| fields | Object | yes(或使用已弃用的 `src_field`+`dest_field` 组合) | | +| fields | Map | yes(或使用已弃用的 `src_field`+`dest_field` 组合) | | | src_field | String | no | | | dest_field | String | no | | diff --git a/docs/zh/transforms/jsonpath.md b/docs/zh/transforms/jsonpath.md index e7ba7d01e71..0d4c317d7d8 100644 --- a/docs/zh/transforms/jsonpath.md +++ b/docs/zh/transforms/jsonpath.md @@ -233,9 +233,9 @@ transform { 那么数据结果表 `fake1` 将会像这样: -| name | age | col | other | -|------|-----|----------|-------| -| a | 18 | ["a",18] | ... | +| col | other | name | age | +|----------|-------|------|-----| +| ["a",18] | ... | a | 18 | diff --git a/docs/zh/transforms/llm.md b/docs/zh/transforms/llm.md index 7853d72eff7..c70e0cb4ef9 100644 --- a/docs/zh/transforms/llm.md +++ b/docs/zh/transforms/llm.md @@ -17,7 +17,7 @@ LLM 转换插件利用大型语言模型(LLM)的能力处理数据,将输 | prompt | string | yes | | | inference_columns | list | no | | | model | string | yes | | -| api_key | string | yes | | +| api_key | string | 条件必填(OPENAI、DOUBAO、MICROSOFT、QIANFAN 时必填;QIANFAN 还需 `secret_key` 与 `oauth_path`) | | | api_path | string | no | | | custom_config | map | no | | | custom_response_parse | string | no | |
