This is an automated email from the ASF dual-hosted git repository.
nzw921rx pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/seatunnel.git
The following commit(s) were added to refs/heads/dev by this push:
new 6b63aca173 [Docs] Fix Zeta REST API, engine ops, transform and FAQ doc
inaccuracies verified against source code (#12932)
6b63aca173 is described below
commit 6b63aca173eb8c056db0b78a87539aec3e4ad1a5
Author: Jast <[email protected]>
AuthorDate: Fri Oct 23 00:47:20 2026 +0800
[Docs] Fix Zeta REST API, engine ops, transform and FAQ doc inaccuracies
verified against source code (#12932)
Co-authored-by: zhangshenghang <[email protected]>
Co-authored-by: jast <[email protected]>
---
docs/en/engines/zeta/engine-jar-storage-mode.md | 36 ++++---
docs/en/engines/zeta/hybrid-cluster-deployment.md | 10 +-
docs/en/engines/zeta/rest-api-job-lifecycle.md | 35 +++---
docs/en/engines/zeta/rest-api-v1.md | 120 ++++++++++++++++-----
docs/en/engines/zeta/rest-api-v2.md | 96 ++++++++++++-----
docs/en/engines/zeta/security.md | 2 +-
.../engines/zeta/separated-cluster-deployment.md | 10 +-
docs/en/engines/zeta/tcp.md | 2 +-
docs/en/engines/zeta/telemetry.md | 2 +-
docs/en/engines/zeta/user-command.md | 12 ++-
docs/en/faq.md | 4 +-
docs/en/introduction/concepts/schema-feature.md | 18 ++--
docs/en/transforms/data-validator.md | 2 +-
docs/en/transforms/dynamic-compile.md | 2 +-
docs/en/transforms/filter.md | 2 +-
.../multi-table-transform-and-join-boundary.md | 21 ++--
docs/en/transforms/recipes.md | 2 +-
docs/en/transforms/transform-multi-table.md | 2 +-
docs/zh/engines/zeta/engine-jar-storage-mode.md | 36 ++++---
docs/zh/engines/zeta/hybrid-cluster-deployment.md | 10 +-
docs/zh/engines/zeta/rest-api-job-lifecycle.md | 30 ++----
docs/zh/engines/zeta/rest-api-v1.md | 118 +++++++++++++++-----
docs/zh/engines/zeta/rest-api-v2.md | 115 +++++++++++++++-----
docs/zh/engines/zeta/security.md | 2 +-
.../engines/zeta/separated-cluster-deployment.md | 10 +-
docs/zh/engines/zeta/tcp.md | 2 +-
docs/zh/engines/zeta/telemetry.md | 2 +-
docs/zh/engines/zeta/user-command.md | 13 ++-
docs/zh/faq.md | 6 +-
.../introduction/concepts/incompatible-changes.md | 19 ++++
docs/zh/introduction/concepts/schema-feature.md | 18 ++--
docs/zh/transforms/data-validator.md | 2 +-
docs/zh/transforms/dynamic-compile.md | 2 +-
docs/zh/transforms/filter.md | 2 +-
.../multi-table-transform-and-join-boundary.md | 19 ++--
docs/zh/transforms/recipes.md | 2 +-
docs/zh/transforms/transform-multi-table.md | 2 +-
37 files changed, 537 insertions(+), 251 deletions(-)
diff --git a/docs/en/engines/zeta/engine-jar-storage-mode.md
b/docs/en/engines/zeta/engine-jar-storage-mode.md
index 095448db34..41ff99faaf 100644
--- a/docs/en/engines/zeta/engine-jar-storage-mode.md
+++ b/docs/en/engines/zeta/engine-jar-storage-mode.md
@@ -35,9 +35,11 @@ Two different storage strategies provide a more flexible
storage mode for jar fi
## Related Configuration
+All options live under `seatunnel.engine.jar-storage` in `seatunnel.yaml`; a
top-level `jar-storage` block is not parsed.
+
| Parameter | Default Value |
Describe
|
|-------------------------------------|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------|
-| connector-jar-storage-enable | false | Whether to enable
uploading the connector jar package to the engine. The default enabled state is
false. |
+| enable | false | Whether to enable
uploading the connector jar package to the engine. The default enabled state is
false. |
| connector-jar-storage-mode | SHARED | Engine-side jar
package storage mode selection. There are two optional modes, SHARED and
ISOLATED. The default Jar package storage mode is SHARED. |
| connector-jar-storage-path | " " | User-defined jar
package storage path.
|
| connector-jar-cleanup-task-interval | 3600s | Engine-side jar
package cleaning scheduled task execution interval.
|
@@ -52,16 +54,18 @@ The jar package files required for the execution of a job
have no influence on o
Example:
```yaml
-jar-storage:
- connector-jar-storage-enable: true
- connector-jar-storage-mode: ISOLATED
- connector-jar-storage-path: ""
- connector-jar-cleanup-task-interval: 3600
- connector-jar-expiry-time: 600
+seatunnel:
+ engine:
+ jar-storage:
+ enable: true
+ connector-jar-storage-mode: ISOLATED
+ connector-jar-storage-path: ""
+ connector-jar-cleanup-task-interval: 3600
+ connector-jar-expiry-time: 600
```
Detailed explanation of configuration parameters:
-- connector-jar-storage-enable: Enable uploading the connector jar package
before executing the job.
+- enable: Enable uploading the connector jar package before executing the job.
- connector-jar-storage-mode: Connector jar package storage mode, two storage
modes are available: shared mode (SHARED) and isolation mode (ISOLATED).
- connector-jar-storage-path: The local storage path of the user-defined
connector jar package on the Zeta engine.
- connector-jar-cleanup-task-interval: Zeta engine connector jar package
scheduled cleanup task interval, the default is 3600 seconds.
@@ -77,16 +81,18 @@ The configuration in the following configuration file sets
the running time of t
Example:
```yaml
-jar-storage:
- connector-jar-storage-enable: true
- connector-jar-storage-mode: SHARED
- connector-jar-storage-path: ""
- connector-jar-cleanup-task-interval: 3600
- connector-jar-expiry-time: 600
+seatunnel:
+ engine:
+ jar-storage:
+ enable: true
+ connector-jar-storage-mode: SHARED
+ connector-jar-storage-path: ""
+ connector-jar-cleanup-task-interval: 3600
+ connector-jar-expiry-time: 600
```
Detailed explanation of configuration parameters:
-- connector-jar-storage-enable: Enable uploading the connector jar package
before executing the job.
+- enable: Enable uploading the connector jar package before executing the job.
- connector-jar-storage-mode: Connector jar package storage mode, two storage
modes are available: shared mode (SHARED) and isolation mode (ISOLATED).
- connector-jar-storage-path: The local storage path of the user-defined
connector jar package on the Zeta engine.
- connector-jar-cleanup-task-interval: Zeta engine connector jar package
scheduled cleanup task interval, the default is 3600 seconds.
diff --git a/docs/en/engines/zeta/hybrid-cluster-deployment.md
b/docs/en/engines/zeta/hybrid-cluster-deployment.md
index ad12feb0b8..e6cb31935d 100644
--- a/docs/en/engines/zeta/hybrid-cluster-deployment.md
+++ b/docs/en/engines/zeta/hybrid-cluster-deployment.md
@@ -197,14 +197,16 @@ The corePoolSize of seatunnel coordinator job's executor
cached thread pool
**max-thread-num**
-The max job count can be executed at same time
+The maximum number of threads in the coordinator service executor (the
`maximumPoolSize` of its thread pool). The default is `Integer.MAX_VALUE`.
Example
```yaml
-coordinator-service:
- core-thread-num: 30
- max-thread-num: 1000
+seatunnel:
+ engine:
+ coordinator-service:
+ core-thread-num: 30
+ max-thread-num: 1000
```
### 4.8 Job Metrics Partition Count (This parameter is invalid on the Worker
node)
diff --git a/docs/en/engines/zeta/rest-api-job-lifecycle.md
b/docs/en/engines/zeta/rest-api-job-lifecycle.md
index 915b9764b6..f0dcb302d6 100644
--- a/docs/en/engines/zeta/rest-api-job-lifecycle.md
+++ b/docs/en/engines/zeta/rest-api-job-lifecycle.md
@@ -149,7 +149,7 @@ Response fields:
|---|---|
| `jobId` | Unique job identifier |
| `jobName` | Human-readable job name |
-| `jobStatus` | `RUNNING`, `FINISHED`, `FAILED`, `CANCELLED` |
+| `jobStatus` | `RUNNING`, `FINISHED`, `FAILED`, `CANCELED` |
| `envOptions` | Env configuration applied |
| `createTime` | Job creation timestamp |
| `jobDag` | DAG structure |
@@ -225,12 +225,10 @@ curl -X POST "http://<master>:8080/stop-job" \
-d '{"jobId": "733584788375093248", "isStopWithSavePoint": true}'
```
-The savepoint path is printed in the job log and returned in the job final
state:
-
-```bash
-curl http://<master>:8080/job-info/733584788375093248 | \
- python3 -c "import sys,json; d=json.load(sys.stdin);
print(d.get('savepointPath', 'N/A'))"
-```
+The savepoint is written to the configured checkpoint storage (see the
checkpoint storage settings of your cluster). The `/job-info/:jobId` response
has no
+`savepointPath` field; to resume later, resubmit with `restoreMode=SAVEPOINT`
(and
+`restoreSourceJobId`), which picks up the latest savepoint of that job. For
the full
+request example, see section 6.2.
### 5.3 Cancel (force)
@@ -325,22 +323,13 @@ curl -X POST
"http://<master>:8080/submit-job?restoreMode=SAVEPOINT&restoreSourc
}'
```
-### 6.3 Restart from a specific savepoint path
-```bash
-curl -X POST http://<master>:8080/submit-job \
- -H "Content-Type: application/json" \
- -d '{
- "env": {
- "job.name": "my-cdc-job-restored",
- "job.mode": "STREAMING",
- "checkpoint.interval": 30000,
- "restore.mode": "savepoint",
- "savepoint.path":
"/seatunnel/checkpoint/savepoint/733584788375093248/1748595600000"
- },
- "source": [ ... ],
- "sink": [ ... ]
- }'
-```
+### 6.3 Restoring from an explicit savepoint path
+
+The REST API does not support pointing at a specific savepoint path. There are
no env options such
+as `restore.mode` or `savepoint.path`; restore is driven only by the
`restoreMode` and
+`restoreSourceJobId` request parameters, and `restoreMode=SAVEPOINT` always
picks up the **latest**
+savepoint of the referenced job (per pipeline, the one with the highest
checkpoint id). There is no
+supported way to select an older savepoint through the REST API.
---
diff --git a/docs/en/engines/zeta/rest-api-v1.md
b/docs/en/engines/zeta/rest-api-v1.md
index 6fba21b9d0..c5c3035527 100644
--- a/docs/en/engines/zeta/rest-api-v1.md
+++ b/docs/en/engines/zeta/rest-api-v1.md
@@ -46,7 +46,7 @@ network:
> | name | type | data type | description
> |
> |--------|----------|-----------|--------------------------------------------------------------------|
-> | type | required | string | plugin type, currently supports `source`
and `sink` |
+> | type | required | string | plugin type, supports `source`, `sink` and
`transform` |
> | plugin | required | string | connector factory identifier, for example
> `FakeSource` or `Console` |
#### Responses
@@ -108,6 +108,10 @@ network:
]
},
"expectValue": "TEMPLATE",
+ "compareOperator": null,
+ "compareOption": null,
+ "conditionOperator": "EQUAL",
+ "conditionOperatorCategory": "EQUALITY",
"operator": null,
"next": null
},
@@ -116,7 +120,29 @@ network:
}
}
],
- "conditionRules": []
+ "conditionRules": [],
+ "valueConstraints": [
+ {
+ "expression": "'row.num' >= 0",
+ "conditionTree": {
+ "option": {
+ "key": "row.num",
+ "type": "java.lang.Integer",
+ "defaultValue": 5,
+ "description": "The total number of data generated per degree of
parallelism",
+ "fallbackKeys": [],
+ "optionValues": null
+ },
+ "expectValue": 0,
+ "compareOperator": ">=",
+ "compareOption": null,
+ "conditionOperator": "GREATER_OR_EQUAL",
+ "conditionOperatorCategory": "NUMERIC",
+ "operator": null,
+ "next": null
+ }
+ }
+ ]
}
}
```
@@ -126,6 +152,9 @@ network:
- `requiredOptions[].ruleType` can be `ABSOLUTELY_REQUIRED`, `EXCLUSIVE`,
`BUNDLED`, or `CONDITIONAL`.
- `optionRule.conditionRules` recursively exposes nested conditional option
rules and is an empty array when the connector does not define nested rules.
- For conditional rules, both `expression` and `expressionTree` are returned
for dynamic form rendering.
+- `optionRule.valueConstraints` describes value-level validation rules such as
numeric ranges, string patterns, and cross-field comparisons. Each entry
provides a human-readable `expression` string alongside a structured
`conditionTree` for programmatic use. This array is empty when the connector
does not define any value constraints.
+- Within `conditionTree`, the `compareOperator` field is `null` for `EQUAL`
and otherwise uses the operator symbol exposed by the runtime rule (for example
`>=`, `is not blank`, or `extension`). The `compareOption` field is populated
only for cross-field comparisons.
+- `conditionOperator` is a stable operator identifier. Possible values include
`EQUAL`, `GREATER_OR_EQUAL`, `NOT_BLANK`, `FIELD_LESS_THAN`, `EXTENSION`, etc.
`conditionOperatorCategory` indicates the operator category, such as `NUMERIC`,
`STRING`, `COLLECTION`, `EQUALITY`, `EXTENSION`, etc.
</details>
@@ -150,8 +179,9 @@ network:
"gitCommitAbbrev":"DeadD0d0",
"totalSlot":"0",
"unassignedSlot":"0",
- "works":"1",
+ "workers":"1",
"runningJobs":"0",
+ "pendingJobs":"0",
"finishedJobs":"0",
"failedJobs":"0",
"cancelledJobs":"0"
@@ -160,7 +190,7 @@ network:
**Notes:**
- If you use `dynamic-slot`, the `totalSlot` and `unassignedSlot` always be
`0`. when you set it to fix slot number, it will return the correct total and
unassigned slot number
-- If the url has tag filter, the `works`, `totalSlot` and `unassignedSlot`
will return the result on the matched worker. but the job related metric will
always return the cluster level information.
+- If the url has tag filter, the `workers`, `totalSlot` and `unassignedSlot`
will return the result on the matched worker. but the job related metric will
always return the cluster level information.
</details>
@@ -277,13 +307,45 @@ None.
],
"isStartWithSavePoint": false,
"metrics": {
- "sourceReceivedCount": "",
- "sinkWriteCount": ""
+ "SourceReceivedCount": "",
+ "SinkWriteCount": ""
}
}
]
```
+**Notes:**
+- The `metrics` object keys follow the engine metric names: besides
`SourceReceivedCount` and `SinkWriteCount` it also contains the
`SinkCommitted*`, `*Bytes`, `*Qps`, `IntermediateQueueSize`, `FlushSignal*` and
`Table*` entries when the job exposes them.
+
+</details>
+
+------------------------------------------------------------------------------------------
+
+### Get Running Jobs Summary
+
+<details>
+ <summary><code>GET</code>
<code><b>/hazelcast/rest/maps/running-jobs/summary</b></code> <code>(Returns a
lightweight running-job list without DAG or metrics.)</code></summary>
+
+#### Parameters
+
+None.
+
+#### Responses
+
+```json
+[
+ {
+ "jobId": "733584788375666689",
+ "jobName": "rest_api_test",
+ "jobStatus": "RUNNING",
+ "createTime": "2024-08-01 12:00:00"
+ }
+]
+```
+
+**Notes:**
+- This is a cheap variant of `/running-jobs` used by the Web UI: it returns
only `jobId`, `jobName`, `jobStatus` and `createTime` per job and skips DAG and
metrics decoding.
+
</details>
------------------------------------------------------------------------------------------
@@ -307,6 +369,7 @@ None.
"jobName": "",
"jobStatus": "",
"createTime": "",
+ "startTime": "",
"jobDag": {
"jobId": "",
"envOptions": [],
@@ -323,10 +386,10 @@ None.
"pipelineEdges": {}
},
"metrics": {
- "sourceReceivedCount": "",
- "sinkWriteCount": ""
+ "SourceReceivedCount": "",
+ "SinkWriteCount": ""
},
- "finishedTime": "",
+ "finishTime": "",
"errorMsg": null,
"envOptions": {
},
@@ -338,7 +401,7 @@ None.
`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` always be
returned.
`envOptions`, `pluginJarsUrls`, `isStartWithSavePoint` will return when job is
running.
-`finishedTime`, `errorMsg` will return when job is finished.
+`finishTime`, `errorMsg` will return when job is finished.
A running job also returns a `diagnostics` block (state timestamps and
per-pipeline restore counts),
see [REST API V2](rest-api-v2.md) for its fields. Only this endpoint returns
it, `/running-jobs` does
not.
@@ -359,15 +422,16 @@ not.
| SinkCommittedQPS | Sink committed rate (rows/s) |
| SinkCommittedBytes | Sink committed bytes after checkpoint succeeds |
| SinkCommittedBytesPerSeconds | Sink committed rate (bytes/s) |
+| IntermediateQueueSize | Records currently buffered in the intermediate
queues |
| TableSourceReceived* | Per-table source metrics, key format
`TableSourceReceivedXXX#<table>` |
| TableSinkWrite* | Per-table sink write attempts, key format
`TableSinkWriteXXX#<table>` |
| TableSinkCommitted* | Per-table sink committed metrics, key format
`TableSinkCommittedXXX#<table>` |
-When we can't get the job info, the response will be:
+When we can't get the job info, the response contains only the requested
`jobId`:
```json
{
- "jobId" : ""
+ "jobId" : "733584788375666689"
}
```
@@ -396,6 +460,7 @@ This API has been deprecated, please use
/hazelcast/rest/maps/job-info/:jobId in
"jobName": "",
"jobStatus": "",
"createTime": "",
+ "startTime": "",
"jobDag": {
"jobId": "",
"envOptions": [],
@@ -424,6 +489,7 @@ This API has been deprecated, please use
/hazelcast/rest/maps/job-info/:jobId in
"SinkCommittedQPS": "",
"SinkCommittedBytes": "",
"SinkCommittedBytesPerSeconds": "",
+ "IntermediateQueueSize": "",
"TableSourceReceivedCount": {},
"TableSourceReceivedBytes": {},
"TableSourceReceivedBytesPerSeconds": {},
@@ -437,7 +503,7 @@ This API has been deprecated, please use
/hazelcast/rest/maps/job-info/:jobId in
"TableSinkCommittedBytes": {},
"TableSinkCommittedBytesPerSeconds": {}
},
- "finishedTime": "",
+ "finishTime": "",
"errorMsg": null,
"envOptions": {
},
@@ -449,13 +515,13 @@ This API has been deprecated, please use
/hazelcast/rest/maps/job-info/:jobId in
`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` always be
returned.
`envOptions`, `pluginJarsUrls`, `isStartWithSavePoint` will return when job is
running.
-`finishedTime`, `errorMsg` will return when job is finished.
+`finishTime`, `errorMsg` will return when job is finished.
-When we can't get the job info, the response will be:
+When we can't get the job info, the response contains only the requested
`jobId`:
```json
{
- "jobId" : ""
+ "jobId" : "733584788375666689"
}
```
@@ -629,7 +695,7 @@ Each member is asked in parallel and awaited against one
shared deadline (`seatu
```json
{
- "jobId": 733584788375666689,
+ "jobId": "733584788375666689",
"jobName": "rest_api_test"
}
```
@@ -766,7 +832,7 @@ Each member is asked in parallel and awaited against one
shared deadline (`seatu
```json
{
-"jobId": 733584788375666689
+"jobId": "733584788375666689"
}
```
@@ -973,13 +1039,11 @@ If the parameter is an empty `Map` object, it means that
the tags of the current
#### Request Parameters
-#### Parameters (Add in the `params` field of the request body)
-
> | Parameter Name | Required | Type | Description
> |
> |----------------------|------------|---------|---------------------------------|
-> | jobId | optional | string | job id
|
+> | jobId | optional | string | job id, appended as the path
segment `/logs/{jobId}` |
-When `jobId` is empty, it returns log information for all nodes; otherwise, it
returns the log list of the specified `jobId` across all nodes.
+When `jobId` is omitted, it returns log information for all nodes; otherwise,
it returns the log list of the specified `jobId` across all nodes.
#### Response
@@ -993,18 +1057,22 @@ If you'd like to view the log list first, you can use a
`GET` request to retriev
```json
[
{
- "node": "localhost:5801",
- "logLink":
"http://localhost:5801/hazelcast/rest/maps/logs/job-899485770241277953.log";,
+ "node": "localhost:8080",
+ "logLink": "http://localhost:8080/logs/job-899485770241277953.log";,
"logName": "job-899485770241277953.log"
},
{
- "node": "localhost:5801",
- "logLink":
"http://localhost:5801/hazelcast/rest/maps/logs/job-899470314109468673.log";,
+ "node": "localhost:8080",
+ "logLink": "http://localhost:8080/logs/job-899470314109468673.log";,
"logName": "job-899470314109468673.log"
}
]
```
+`logLink` and `node` point to the Jetty HTTP service of each node (port `8080`
by default, see
+`seatunnel.engine.http`), not to the Hazelcast REST port. The endpoints require
+`seatunnel.engine.http.enable-http: true`, which is `false` by default.
+
The supported formats are `json` and `html`, with `html` as the default.
#### Response Size Limit
diff --git a/docs/en/engines/zeta/rest-api-v2.md
b/docs/en/engines/zeta/rest-api-v2.md
index 97fc473174..1228ba2481 100644
--- a/docs/en/engines/zeta/rest-api-v2.md
+++ b/docs/en/engines/zeta/rest-api-v2.md
@@ -176,7 +176,7 @@ Please refer [security](security.md)
"conditionRules": [],
"valueConstraints": [
{
- "expression": "'row.num' >= 1",
+ "expression": "'row.num' >= 0",
"conditionTree": {
"option": {
"key": "row.num",
@@ -186,7 +186,7 @@ Please refer [security](security.md)
"fallbackKeys": [],
"optionValues": null
},
- "expectValue": 1,
+ "expectValue": 0,
"compareOperator": ">=",
"compareOption": null,
"conditionOperator": "GREATER_OR_EQUAL",
@@ -253,7 +253,7 @@ Please refer [security](security.md)
"gitCommitAbbrev":"DeadD0d0",
"totalSlot":"0",
"unassignedSlot":"0",
- "works":"1",
+ "workers":"1",
"runningJobs":"0",
"pendingJobs":"0",
"finishedJobs":"0",
@@ -264,7 +264,7 @@ Please refer [security](security.md)
**Notes:**
- If you use `dynamic-slot`, the `totalSlot` and `unassignedSlot` always be
`0`. when you set it to fix slot number, it will return the correct total and
unassigned slot number
-- If the url has tag filter, the `works`, `totalSlot` and `unassignedSlot`
will return the result on the matched worker. but the job related metric will
always return the cluster level information.
+- If the url has tag filter, the `workers`, `totalSlot` and `unassignedSlot`
will return the result on the matched worker. but the job related metric will
always return the cluster level information.
</details>
@@ -374,13 +374,46 @@ None.
],
"isStartWithSavePoint": false,
"metrics": {
- "sourceReceivedCount": "",
- "sinkWriteCount": ""
+ "SourceReceivedCount": "",
+ "SinkWriteCount": ""
}
}
]
```
+**Notes:**
+- The `metrics` object keys follow the engine metric names: besides
`SourceReceivedCount` and `SinkWriteCount` it also contains the
`SinkCommitted*`, `*Bytes`, `*Qps`, `IntermediateQueueSize`, `FlushSignal*` and
`Table*` entries when the job exposes them.
+- When the `page` parameter is supplied, the response is wrapped as `{"data":
[...], "total": n}` (same as `/finished-jobs`); without paging the bare array
above is returned.
+
+</details>
+
+------------------------------------------------------------------------------------------
+
+### Get Running Jobs Summary
+
+<details>
+ <summary><code>GET</code> <code><b>/running-jobs/summary</b></code>
<code>(Returns a lightweight running-job list without DAG or
metrics.)</code></summary>
+
+#### Parameters
+
+None.
+
+#### Responses
+
+```json
+[
+ {
+ "jobId": "733584788375666689",
+ "jobName": "rest_api_test",
+ "jobStatus": "RUNNING",
+ "createTime": "2024-08-01 12:00:00"
+ }
+]
+```
+
+**Notes:**
+- This is a cheap variant of `/running-jobs` used by the Web UI: it returns
only `jobId`, `jobName`, `jobStatus` and `createTime` per job and skips DAG and
metrics decoding.
+
</details>
------------------------------------------------------------------------------------------
@@ -562,6 +595,7 @@ This endpoint helps troubleshoot why jobs stay in `PENDING`
by showing the pendi
"jobName": "",
"jobStatus": "",
"createTime": "",
+ "startTime": "",
"jobDag": {
"jobId": "",
"envOptions": [],
@@ -604,7 +638,7 @@ This endpoint helps troubleshoot why jobs stay in `PENDING`
by showing the pendi
"TableSinkCommittedBytes": {},
"TableSinkCommittedBytesPerSeconds": {}
},
- "finishedTime": "",
+ "finishTime": "",
"errorMsg": null,
"envOptions": {
},
@@ -642,7 +676,7 @@ This endpoint helps troubleshoot why jobs stay in `PENDING`
by showing the pendi
`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` always be
returned.
`envOptions`, `pluginJarsUrls`, `isStartWithSavePoint` will return when job is
running.
-`finishedTime`, `errorMsg` will return when job is finished.
+`finishTime`, `errorMsg` will return when job is finished.
`diagnostics` will return when the job is running and its diagnostics can be
read from the master
node. It is auxiliary information: if it can not be obtained, the field is
omitted instead of
failing the request. Only this endpoint returns it; `/running-jobs` does not,
because collecting it
@@ -682,11 +716,11 @@ for every running job would cost one more round trip to
the master per job.
| pipelines[].stateTimestamps | Epoch millis when the pipeline entered each
state. After a restore, the timestamps of the new attempt overwrite the
previous ones |
| totalPipelineRestoreCount | Sum of `restoreCount` over all pipelines of the
job |
-When we can't get the job info, the response will be:
+When we can't get the job info, the response contains only the requested
`jobId`:
```json
{
- "jobId" : ""
+ "jobId" : "733584788375666689"
}
```
@@ -715,6 +749,7 @@ This API has been deprecated, please use /job-info/:jobId
instead
"jobName": "",
"jobStatus": "",
"createTime": "",
+ "startTime": "",
"jobDag": {
"jobId": "",
"envOptions": [],
@@ -740,6 +775,10 @@ This API has been deprecated, please use /job-info/:jobId
instead
"SinkWriteQPS": "",
"SinkWriteBytes": "",
"SinkWriteBytesPerSeconds": "",
+ "SinkCommittedCount": "",
+ "SinkCommittedQPS": "",
+ "SinkCommittedBytes": "",
+ "SinkCommittedBytesPerSeconds": "",
"TableSourceReceivedCount": {},
"TableSourceReceivedBytes": {},
"TableSourceReceivedBytesPerSeconds": {},
@@ -749,7 +788,7 @@ This API has been deprecated, please use /job-info/:jobId
instead
"TableSinkWriteBytes": {},
"TableSinkWriteBytesPerSeconds": {}
},
- "finishedTime": "",
+ "finishTime": "",
"errorMsg": null,
"envOptions": {
},
@@ -761,13 +800,13 @@ This API has been deprecated, please use /job-info/:jobId
instead
`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` always be
returned.
`envOptions`, `pluginJarsUrls`, `isStartWithSavePoint` will return when job is
running.
-`finishedTime`, `errorMsg` will return when job is finished.
+`finishTime`, `errorMsg` will return when job is finished.
-When we can't get the job info, the response will be:
+When we can't get the job info, the response contains only the requested
`jobId`:
```json
{
- "jobId" : ""
+ "jobId" : "733584788375666689"
}
```
@@ -1026,7 +1065,7 @@ INSERT INTO console_sink SELECT * FROM fake_source;
```json
{
- "jobId": 733584788375666689,
+ "jobId": "733584788375666689",
"jobName": "rest_api_test"
}
```
@@ -1075,7 +1114,7 @@ curl --location
'http://127.0.0.1:8080/submit-job/upload?restoreMode=CHECKPOINT&;
```json
{
- "jobId": 733584788375666689,
+ "jobId": "733584788375666689",
"jobName": "SeaTunnel_Job"
}
```
@@ -1212,7 +1251,7 @@ curl --location
'http://127.0.0.1:8080/submit-job/upload?restoreMode=CHECKPOINT&;
```json
{
-"jobId": 733584788375666689
+"jobId": "733584788375666689"
}
```
@@ -1419,13 +1458,11 @@ If the parameter is an empty `Map` object, it means
that the tags of the current
#### Request Parameters
-#### Parameters (to be added in the `params` field of the request body)
-
> | Parameter Name | Required | Type | Description
> |
> |-----------------------|--------------|---------|------------------------------------|
-> | jobId | optional | string | job id
|
+> | jobId | optional | string | job id, appended as the
path segment `/logs/{jobId}` |
-If `jobId` is empty, the request will return logs from all nodes. Otherwise,
it will return the list of logs for the specified `jobId` from all nodes.
+If `jobId` is omitted, the request will return logs from all nodes. Otherwise,
it will return the list of logs for the specified `jobId` from all nodes.
#### Response
@@ -1502,8 +1539,8 @@ Returns a list of logs from the requested node.
#### Examples
-To get a list of logs from the current node: `http://localhost:5801/log`
-To get the content of a log file:
`http://localhost:5801/log/job-898380162133917698.log`
+To get a list of logs from the current node: `http://localhost:8080/log`
+To get the content of a log file:
`http://localhost:8080/log/job-898380162133917698.log`
Log content is limited by `seatunnel.engine.http.log-response-max-size-mb` in
the same way as the
all-node endpoint above.
@@ -1864,10 +1901,21 @@ See [Realtime Observability](realtime-observability.md)
for configuration and me
"retentionMinutes": 3,
"latestBucketStartMs": 1700000000000
}
- ]
+ ],
+ "collector": {
+ "pollIntervalMs": 5000,
+ "fetchTimeoutMs": 3000,
+ "lastCollectStartMs": 1700000000000,
+ "lastCollectEndMs": 1700000001200,
+ "lastRawMetricsFetchCostMs": 120,
+ "lastRawMetricsBlobs": 2,
+ "collectFailureCount": 0
+ }
}
```
+The `collector` object reports the realtime collector lifecycle on the master
(poll interval, last collection timings and the failure counter).
+
</details>
<details>
diff --git a/docs/en/engines/zeta/security.md b/docs/en/engines/zeta/security.md
index b3c1b4257c..c851ee62e8 100644
--- a/docs/en/engines/zeta/security.md
+++ b/docs/en/engines/zeta/security.md
@@ -27,7 +27,7 @@ You can secure your REST-API-V2 service by enabling HTTPS.
Both HTTP and HTTPS c
| Parameter Name | Required | Description |
|----------------|----------|-------------|
-| `enable-http` | No | Whether to enable HTTP service, default is `true` |
+| `enable-http` | No | Whether to enable HTTP service, default is `false` (the
shipped `config/seatunnel.yaml` sets it to `true`) |
| `port` | No | HTTP service port, default is `8080` |
| `enable-https` | No | Whether to enable HTTPS service, default is `false` |
| `https-port` | No | HTTPS service port, default is `8443` |
diff --git a/docs/en/engines/zeta/separated-cluster-deployment.md
b/docs/en/engines/zeta/separated-cluster-deployment.md
index 4a36c03747..a04d6f8074 100644
--- a/docs/en/engines/zeta/separated-cluster-deployment.md
+++ b/docs/en/engines/zeta/separated-cluster-deployment.md
@@ -426,14 +426,16 @@ The corePoolSize of seatunnel coordinator job's executor
cached thread pool
**max-thread-num**
-The max job count can be executed at same time
+The maximum number of threads in the coordinator service executor (the
`maximumPoolSize` of its thread pool). The default is `Integer.MAX_VALUE`.
Example
```yaml
-coordinator-service:
- core-thread-num: 30
- max-thread-num: 1000
+seatunnel:
+ engine:
+ coordinator-service:
+ core-thread-num: 30
+ max-thread-num: 1000
```
### 4.9 Job Metrics Partition Count (This parameter is invalid on the Worker
node)
diff --git a/docs/en/engines/zeta/tcp.md b/docs/en/engines/zeta/tcp.md
index 899a26e3bf..16b10cb568 100644
--- a/docs/en/engines/zeta/tcp.md
+++ b/docs/en/engines/zeta/tcp.md
@@ -33,4 +33,4 @@ Instead of providing members line-by-line as shown above, you
also have the opti
`<members>192.168.1.0-7,192.168.1.21</members>`
-If you do not provide ports for the members, Hazelcast automatically tries the
ports `5701`, `5702` and so on.
+If you do not provide ports for the members, Hazelcast tries the port
configured under `network.port` (`5801` in the SeaTunnel default
`hazelcast.yaml`, `5802` for the worker template). Whether subsequent ports are
also attempted is controlled by `network.port.auto-increment`, which SeaTunnel
ships as `false`.
diff --git a/docs/en/engines/zeta/telemetry.md
b/docs/en/engines/zeta/telemetry.md
index 1d81aed91a..1048758360 100644
--- a/docs/en/engines/zeta/telemetry.md
+++ b/docs/en/engines/zeta/telemetry.md
@@ -196,7 +196,7 @@ This metric is exported by the active master only. It
reports only an aggregate
| MetricName | Type | Labels
|
DESCRIPTION |
|------------|-------|-----------------------------------------------------------------------------------------------------------------------------|-------------------------------------|
-| job_count | Gauge | **type**, the type of job, including: "canceled"
"cancelling" "created" "failed" "failing" "finished" "running" "scheduled" |
All job counts of seatunnel cluster |
+| job_count | Gauge | **type**, the type of job, including: "canceled"
"cancelling" "created" "failed" "failing" "finished" "pending" "running"
"scheduled" | All job counts of seatunnel cluster |
### JVM Metrics
diff --git a/docs/en/engines/zeta/user-command.md
b/docs/en/engines/zeta/user-command.md
index 64a68c750c..7cd4fa2e70 100644
--- a/docs/en/engines/zeta/user-command.md
+++ b/docs/en/engines/zeta/user-command.md
@@ -22,6 +22,11 @@ Usage: seatunnel.sh [options]
-can, --cancel, --cancel-job Cancel the job(s) by JobId.
-f, --force-cancel, --force-cancel-job Force Cancel job(s) by JobId.
--check Whether to check the config
(default: false).
+ --checkpoint-history Get checkpoint history by
JobId.
+ --checkpoint-history-limit Limit checkpoint history size
(default: 20).
+ --checkpoint-history-pipeline Filter checkpoint history by
pipeline id.
+ --checkpoint-history-status Filter checkpoint history by
status: COMPLETED,FAILED,CANCELED.
+ --checkpoint-overview Get checkpoint overview by
JobId.
-cj, --close, --close-job Close the client and the task
will also be closed (default: true).
-cn, --cluster The name of the cluster.
-c, --config Config file.
@@ -40,6 +45,7 @@ Usage: seatunnel.sh [options]
-s, --savepoint, --savepoint-job Savepoint the job by jobId.
--sample-limit Maximum rows forwarded from
each source by sample dry-run mode (default: 10, max: 10000).
--sample-print-data Print sampled row values to
persistent logs (default: false).
+ --set-job-id Set custom job id for job.
-i, --variable Variable substitution, such as
-i city=beijing, or -i date=20190318. We use ',' as a separator. When inside
"", ',' are treated as normal characters instead of delimiters. (default: []).
```
@@ -258,9 +264,9 @@ This command will output detailed information about all
members in the cluster,
**Example output:**
```
Member ID Address Role
Version
-a1b2c3d4-e5f6-7890-abcd-ef1234567890 192.168.1.100:5701 ACTIVE MASTER
5.3.0
-b2c3d4e5-f6g7-8901-bcde-f23456789012 192.168.1.101:5701 MASTER
5.3.0
-c3d4e5f6-g7h8-9012-cdef-345678901234 192.168.1.102:5701 WORKER
5.3.0
+a1b2c3d4-e5f6-7890-abcd-ef1234567890 192.168.1.100:5801 ACTIVE MASTER
5.3.0
+b2c3d4e5-f6g7-8901-bcde-f23456789012 192.168.1.101:5801 MASTER
5.3.0
+c3d4e5f6-g7h8-9012-cdef-345678901234 192.168.1.102:5801 WORKER
5.3.0
```
**Note**: You must specify the cluster name with the `-cn` parameter. The
cluster must be running for this command to work.
diff --git a/docs/en/faq.md b/docs/en/faq.md
index 5cd23f2ba4..d7b7cc57e8 100644
--- a/docs/en/faq.md
+++ b/docs/en/faq.md
@@ -29,7 +29,7 @@ Yes, SeaTunnel supports custom data cleansing rules. You can
configure custom ru
SeaTunnel supports incremental data integration. For example, the CDC
connector allows real-time capture of data changes, which is ideal for
scenarios requiring real-time data integration.
## What CDC data sources are currently supported by SeaTunnel?
-SeaTunnel currently supports MongoDB CDC, MySQL CDC, OpenGauss CDC, Oracle
CDC, PostgreSQL CDC, SQL Server CDC, TiDB CDC, and more. For more details,
refer to the [Source List](./connectors/source).
+SeaTunnel currently supports MongoDB CDC, MySQL CDC, OpenGauss CDC, Oracle
CDC, PostgreSQL CDC, SQL Server CDC, TiDB CDC, DB2 CDC, OceanBase CDC, Vitess
CDC, and more. For more details, refer to the [Source
List](./connectors/source).
## How do I enable permissions required for SeaTunnel CDC integration?
Please refer to the official SeaTunnel documentation for the necessary steps
to enable permissions for each connector’s CDC functionality.
@@ -38,7 +38,7 @@ Please refer to the official SeaTunnel documentation for the
necessary steps to
Yes, SeaTunnel supports CDC from MySQL replicas by subscribing to binlog logs,
which are then parsed on the SeaTunnel server.
## Does SeaTunnel support CDC integration for tables without primary keys?
-SeaTunnel does not support CDC integration for tables without primary keys.
The reason is that if two identical records exist in the upstream and one is
deleted or modified, the downstream cannot determine which record to delete or
modify, leading to potential issues. Primary keys are essential to ensure data
uniqueness.
+Yes, with limitations. Without a unique key the downstream sink cannot tell
apart two identical rows, so updates and deletes may affect both of them. For a
table without a primary key, the JDBC-based CDC connectors (MySQL, OpenGauss,
PostgreSQL, Oracle, SQL Server, DB2) fall back to a non-nullable unique-key
column as the snapshot split column; when no suitable key column exists, the
table is read as a single snapshot split. You can also point the splitter at a
column explicitly with `ta [...]
## Does SeaTunnel support automatic table creation?
Before starting an integration task, you can select different handling schemes
for existing table structures on the target side, controlled via the
`schema_save_mode` parameter. Available options include:
diff --git a/docs/en/introduction/concepts/schema-feature.md
b/docs/en/introduction/concepts/schema-feature.md
index c9651a4d95..12f4bd35c5 100644
--- a/docs/en/introduction/concepts/schema-feature.md
+++ b/docs/en/introduction/concepts/schema-feature.md
@@ -155,7 +155,7 @@ columns = [
| timestamp_tz | `java.time.OffsetDateTime` | Stores
date and time information with an offset from UTC. It includes both the local
date-time and the offset from UTC, providing more precise temporal information
when working with multiple time zones.
|
| row | `org.apache.seatunnel.api.table.type.SeaTunnelRowType` | Row
type, can be nested.
|
| map | `java.util.Map` | A Map is
an object that maps keys to values. The key type includes `int` `string`
`boolean` `tinyint` `smallint` `bigint` `float` `double` `decimal` `date`
`time` `timestamp` `null` , and the value type includes `int` `string`
`boolean` `tinyint` `smallint` `bigint` `float` `double` `decimal` `date`
`time` `timestamp` `null` `array` `map` `row`. |
-| array | `ValueType[]` | A array
is a data type that represents a collection of elements. The element type
includes `int` `string` `boolean` `tinyint` `smallint` `bigint` `float`
`double`.
|
+| array | `ValueType[]` | A array
is a data type that represents a collection of elements. The element type
includes `int` `string` `boolean` `tinyint` `smallint` `bigint` `float`
`double` `map` and a nested `array`.
|
#### How to declare type supported
@@ -165,9 +165,9 @@ SeaTunnel provides a simple and direct way to declare basic
types. Basic type ke
When declaring complex types (such as **decimal**, **array**, **map**, and
**row**), pay attention to specific considerations.
- When declaring a decimal type, precision and scale settings are required,
and the type definition follows the format `decimal(precision, scale)`. It's
essential to emphasize that the declaration of the decimal type must be
enclosed in `"`; you cannot use the type name directly, as with basic types.
For example, when declaring a decimal field with precision 10 and scale 2, you
specify the field type as `"decimal(10,2)"`.
-- When declaring an array type, you need to specify the element type, and the
type definition follows the format `array<T>`, where `T` represents the element
type. The element type includes
`int`,`string`,`boolean`,`tinyint`,`smallint`,`bigint`,`float` and `double`.
Similar to the decimal type declaration, it also be enclosed in `"`. For
example, when declaring a field with an array of integers, you specify the
field type as `"array<int>"`.
+- When declaring an array type, you need to specify the element type, and the
type definition follows the format `array<T>`, where `T` represents the element
type. The element type includes
`int`,`string`,`boolean`,`tinyint`,`smallint`,`bigint`,`float`,`double`, a
`map<K,V>` and a nested `array<T>`. Similar to the decimal type declaration, it
also be enclosed in `"`. For example, when declaring a field with an array of
integers, you specify the field type as `"array<int>"`; an array of s [...]
- When declaring a map type, you need to specify the key and value types. The
map type definition follows the format `map<K,V>`, where `K` represents the key
type and `V` represents the value type. `K` can be any basic type and decimal
type, and `V` can be any type supported by SeaTunnel. Similar to previous type
declarations, the map type declaration must be enclosed in double quotes. For
example, when declaring a field with map type, where the key type is string and
the value type is i [...]
-- When declaring a row type, you need to define a
[HOCON](https://github.com/lightbend/config/blob/main/HOCON.md) object to
describe the fields and their types. The field types can be any type supported
by SeaTunnel. For example, when declaring a row type containing an integer
field `a` and a string field `b`, you can declare it as `{a = int, b =
string}`. Enclosing the definition in `"` as a string is also acceptable, so
`"{a = int, b = string}"` is equivalent to `{a = int, c = string}` [...]
+- When declaring a row type, you need to define a
[HOCON](https://github.com/lightbend/config/blob/main/HOCON.md) object to
describe the fields and their types. The field types can be any type supported
by SeaTunnel. For example, when declaring a row type containing an integer
field `a` and a string field `b`, you can declare it as `{a = int, b =
string}`. Enclosing the definition in `"` as a string is also acceptable, so
`"{a = int, b = string}"` is equivalent to `{a = int, b = string}` [...]
Here is an example of complex type declarations:
@@ -193,19 +193,19 @@ schema {
### PrimaryKey
-Primary key is a config used to define the primary key in schema, it contains
name, columns field.
+Primary key is a config used to define the primary key in schema, it contains
name, columnNames field.
```
primaryKey {
name = id
- columns = [id]
+ columnNames = [id]
}
```
-| Field | Required | Default Value | Description |
-|:--------|:---------|:--------------|-----------------------------------|
-| name | Yes | - | The name of the primaryKey |
-| columns | Yes | - | The column list in the primaryKey |
+| Field | Required | Default Value | Description |
+|:------------|:---------|:--------------|-----------------------------------|
+| name | Yes | - | The name of the primaryKey |
+| columnNames | Yes | - | The column list in the primaryKey |
### ConstraintKeys
diff --git a/docs/en/transforms/data-validator.md
b/docs/en/transforms/data-validator.md
index 83991d3f9a..e1cef97a01 100644
--- a/docs/en/transforms/data-validator.md
+++ b/docs/en/transforms/data-validator.md
@@ -12,7 +12,7 @@ The DataValidator transform validates field values according
to configured rules
|-----------------|--------|----------|---------------|
| row_error_handle_way| enum | no | FAIL |
| row_error_handle_way.error_table | string | no | |
-| field_rules | array | no | |
+| field_rules | array | yes (single-table mode) | |
### row_error_handle_way [enum]
diff --git a/docs/en/transforms/dynamic-compile.md
b/docs/en/transforms/dynamic-compile.md
index 113dc173de..bddeee430f 100644
--- a/docs/en/transforms/dynamic-compile.md
+++ b/docs/en/transforms/dynamic-compile.md
@@ -42,7 +42,7 @@ If it is a SOURCE-CODE enumeration; the SOURCE-CODE attribute
is required, and t
### absolute_path [string]
-The absolute path of Java or Groovy files on the server
+The absolute path of the Java, Groovy, or Scala source file on the server
### source_code [string]
diff --git a/docs/en/transforms/filter.md b/docs/en/transforms/filter.md
index 2fa60e6b1e..ad1f135220 100644
--- a/docs/en/transforms/filter.md
+++ b/docs/en/transforms/filter.md
@@ -27,7 +27,7 @@ The Filter transform decides which fields of a table are
passed to the next step
| exclude_fields | Array<String> | no | - | Fields
to remove. All other fields are kept. Mutually exclusive with `include_fields`.
|
| table_match_regex | String | no | `.*` | Regex of the
table paths that the transform level rule applies to.
|
| table_transform | Array<Map> | no | - |
Per-table rules, see [Multi-Table Transform](transform-multi-table.md).
|
-| rule_match_mode | String | no | - | How multiple
`table_transform` entries with the same `table_path` are evaluated.
|
+| rule_match_mode | String | no | `FIRST_MATCH` | How
multiple `table_transform` entries with the same `table_path` are evaluated.
|
| common options | - | no | - | Transform
plugin common parameters, see [Transform
Plugin](common-options/common-options.md). |
:::caution
diff --git a/docs/en/transforms/multi-table-transform-and-join-boundary.md
b/docs/en/transforms/multi-table-transform-and-join-boundary.md
index 58af2b78a5..9d0dda5161 100644
--- a/docs/en/transforms/multi-table-transform-and-join-boundary.md
+++ b/docs/en/transforms/multi-table-transform-and-join-boundary.md
@@ -51,7 +51,7 @@ flowchart LR
|---|---|---|
| Per-table field rename / map | ✅ Yes | Use `FieldMapper` with `plugin_input`
and `table_match_regex` |
| Per-table column filtering | ✅ Yes | Use `Filter` with `plugin_input` and
`table_match_regex` |
-| Per-table type casting | ✅ Yes | Use `FieldMapper` with `define_sink_type`
option |
+| Per-table type casting | ✅ Yes | Use the `DefineSinkType` transform with its
`columns` option (optionally scoped per table via `table_transform`) |
| Per-table SQL transform (single table) | ✅ Yes | Use `SQL` transform with
`plugin_input`; scope it with `table_match_regex` when needed |
| `TableMerge` — merge multiple tables into one | ✅ Yes | Tables must share
compatible schema |
| `TableRename` — rename tables in the stream | ✅ Yes | Works well for routing
to Sink by table name |
@@ -79,7 +79,7 @@ input tables must have the same (or compatible) schema. Use
it to route all tabl
"plugin_name": "TableMerge",
"plugin_input": ["orders_2023", "orders_2024"],
"plugin_output": "all_orders",
- "merge_by_field": true
+ "table": "all_orders"
}
```
@@ -177,13 +177,19 @@ rules:
{
"plugin_name": "Kafka",
"plugin_input": ["cdc_stream"],
- "topic": "products",
- "table_match_regex": "mydb\\.products"
+ "topic": "products"
}
]
}
```
+A transform with `table_match_regex` only rewrites the matching tables; every
other table of its
+input stream is passed through unchanged, and a sink receives every table of
the stream it
+subscribes to via `plugin_input` — sinks cannot filter by table name. In the
example above each
+Kafka sink therefore still sees all three tables of its input stream. To
deliver exactly one table
+to a sink, put a `TableFilter` transform (for example `database_pattern =
"mydb"`,
+`table_pattern = "products"`) in front of it.
+
---
## 6. Common Fields Example (Shared Schema)
@@ -197,6 +203,7 @@ apply a single transform:
"plugin_name": "TableMerge",
"plugin_input": ["cdc_stream"],
"plugin_output": "all_events",
+ "table": "all_events",
"table_match_regex": "mydb\\.(orders|payments|refunds)"
},
{
@@ -266,8 +273,10 @@ JOINs, use an external SQL engine after loading the data
into a sink.
**Q: What happens to tables that are not matched by any transform?**
-Unmatched tables continue flowing through the pipeline and can be captured by
a Sink that
-uses the right `plugin_input` and `table_match_regex`.
+Unmatched tables continue flowing through the pipeline unchanged. A sink
receives every table of
+the stream it subscribes to via `plugin_input` — sinks have no
`table_match_regex` option. If you
+need to select or route specific tables before the sink, use a transform with
+`table_match_regex`/`table_transform` (for example `TableFilter`,
`TableRename` or `TableMerge`).
**Q: Can I add a new table to an existing CDC pipeline without downtime?**
diff --git a/docs/en/transforms/recipes.md b/docs/en/transforms/recipes.md
index 23ae913bb5..0e65460ce4 100644
--- a/docs/en/transforms/recipes.md
+++ b/docs/en/transforms/recipes.md
@@ -134,7 +134,7 @@ SeaTunnel’s Transform V2 SPI supports both:
- map (1 input row -> 1 output row)
- flatMap (1 input row -> N output rows)
-However, there is no documented built-in “explode array into many rows”
transform today. If your JSON contains an array field like `items: [...]` and
you want to turn each item into a separate output row, you typically have these
options:
+To turn an array field like `items: [...]` into one output row per element,
use the `Sql` transform with `LATERAL VIEW EXPLODE` (or `OUTER EXPLODE` to keep
rows with empty arrays); see [SQL Functions](sql-functions.md). Other options:
- Pre-process upstream (e.g., Flink/Spark SQL, or upstream producers) to
output one JSON object per target row.
- Write a custom Transform V2 plugin that implements a flatMap transform.
diff --git a/docs/en/transforms/transform-multi-table.md
b/docs/en/transforms/transform-multi-table.md
index ff69768e15..29570ec164 100644
--- a/docs/en/transforms/transform-multi-table.md
+++ b/docs/en/transforms/transform-multi-table.md
@@ -19,7 +19,7 @@ Multi-table Transform has no limitations on Transform
capabilities; any Transfor
| table_match_regex | String | No | .* | A regular
expression to match the tables that require transformation. By default, it
matches all tables. Note that this table name refers to the actual upstream
table name, not `plugin_output`.
|
| table_transform | List | No | - | You can use a
list in `table_transform` to specify rules for individual tables. If a
transformation rule is configured for a specific table in `table_transform`,
the outer rules will not apply to that table. The rules in `table_transform`
take precedence. |
| table_transform.table_path | String | Yes | - | When configuring
a transformation rule for a table in `table_transform`, you need to specify the
table path using the `table_path` field. The table path should include
`databaseName[.schemaName].tableName` and is matched exactly. It must not be
blank. |
-| rule_match_mode | String | No | - | Controls how
multiple `table_transform` entries with the same exact `table_path` are
evaluated. Supported values are `FIRST_MATCH` and `ALL_MATCH`.
|
+| rule_match_mode | String | No | `FIRST_MATCH` | Controls
how multiple `table_transform` entries with the same exact `table_path` are
evaluated. Supported values are `FIRST_MATCH` and `ALL_MATCH`.
|
## Matching Logic
diff --git a/docs/zh/engines/zeta/engine-jar-storage-mode.md
b/docs/zh/engines/zeta/engine-jar-storage-mode.md
index b087f0a63f..d8b74436ee 100644
--- a/docs/zh/engines/zeta/engine-jar-storage-mode.md
+++ b/docs/zh/engines/zeta/engine-jar-storage-mode.md
@@ -36,9 +36,11 @@ COMMON_ PLUGIN_ JARS 指的是连接器所依赖的第三方 Jar 包， CONNECTO
## 相关配置
+所有选项都位于 `seatunnel.yaml` 的 `seatunnel.engine.jar-storage` 下；顶层的 `jar-storage`
块不会被解析。
+
| 参数 | 默认值 |
描述 |
|-------------------------------------|--------|-------------------------------------------------------------------------|
-| connector-jar-storage-enable | false | 是否启用上传连接器 Jar 包到引擎。默认启用状态为
false。 |
+| enable | false | 是否启用上传连接器 Jar 包到引擎。默认启用状态为
false。 |
| connector-jar-storage-mode | SHARED | 引擎端 Jar
包存储模式选择。有两个可选模式，SHARED（共享）和 ISOLATED（隔离）。默认的 Jar 包存储模式是 SHARED。 |
| connector-jar-storage-path | " " | 用户自定义的 Jar 包存储路径。
|
| connector-jar-cleanup-task-interval | 3600s | 引擎端 Jar 包清理定时任务执行间隔。
|
@@ -53,16 +55,18 @@ COMMON_ PLUGIN_ JARS 指的是连接器所依赖的第三方 Jar 包， CONNECTO
示例：
```yaml
-jar-storage:
- connector-jar-storage-enable: true
- connector-jar-storage-mode: ISOLATED
- connector-jar-storage-path: ""
- connector-jar-cleanup-task-interval: 3600
- connector-jar-expiry-time: 600
+seatunnel:
+ engine:
+ jar-storage:
+ enable: true
+ connector-jar-storage-mode: ISOLATED
+ connector-jar-storage-path: ""
+ connector-jar-cleanup-task-interval: 3600
+ connector-jar-expiry-time: 600
```
配置参数的详细解释：
-- connector-jar-storage-enable: 在执行作业前启用上传连接器 Jar 包的功能。
+- enable: 在执行作业前启用上传连接器 Jar 包的功能。
- connector-jar-storage-mode: 连接器 Jar
包的存储模式，有两种存储模式可供选择：共享模式（SHARED）和隔离模式（ISOLATED）。
- connector-jar-storage-path: 在 Zeta 引擎上用户自定义连接器 Jar 包的本地存储路径。
- connector-jar-cleanup-task-interval: Zeta 引擎连接器 Jar 包定时清理任务的间隔时间，默认为 3600 秒。
@@ -78,16 +82,18 @@ jar-storage:
示例:
```yaml
-jar-storage:
- connector-jar-storage-enable: true
- connector-jar-storage-mode: SHARED
- connector-jar-storage-path: ""
- connector-jar-cleanup-task-interval: 3600
- connector-jar-expiry-time: 600
+seatunnel:
+ engine:
+ jar-storage:
+ enable: true
+ connector-jar-storage-mode: SHARED
+ connector-jar-storage-path: ""
+ connector-jar-cleanup-task-interval: 3600
+ connector-jar-expiry-time: 600
```
配置参数的详细解释：
-- connector-jar-storage-enable: 在执行作业前启用上传连接器 Jar 包的功能。
+- enable: 在执行作业前启用上传连接器 Jar 包的功能。
- connector-jar-storage-mode: 连接器 Jar
包的存储模式，有两种存储模式可供选择：共享模式（SHARED）和隔离模式（ISOLATED）。
- connector-jar-storage-path: 在 Zeta 引擎上用户自定义连接器 Jar 包的本地存储路径。
- connector-jar-cleanup-task-interval: Zeta 引擎连接器 Jar 包定时清理任务的间隔时间，默认为 3600 秒。
diff --git a/docs/zh/engines/zeta/hybrid-cluster-deployment.md
b/docs/zh/engines/zeta/hybrid-cluster-deployment.md
index 9d7d4b05af..72c1cbeba4 100644
--- a/docs/zh/engines/zeta/hybrid-cluster-deployment.md
+++ b/docs/zh/engines/zeta/hybrid-cluster-deployment.md
@@ -194,14 +194,16 @@ CoordinatorService 提供了每个作业从 LogicalDag 到 ExecutionDag，再到
**max-thread-num**
-同时可执行的最大作业数量
+CoordinatorService 线程池的最大线程数（即线程池的 `maximumPoolSize`），默认值为 `Integer.MAX_VALUE`。
Example
```yaml
-coordinator-service:
- core-thread-num: 30
- max-thread-num: 1000
+seatunnel:
+ engine:
+ coordinator-service:
+ core-thread-num: 30
+ max-thread-num: 1000
```
### 4.8 作业指标分区数量（此参数在 Worker 节点上无效）
diff --git a/docs/zh/engines/zeta/rest-api-job-lifecycle.md
b/docs/zh/engines/zeta/rest-api-job-lifecycle.md
index 6932ed8231..f2d06f4180 100644
--- a/docs/zh/engines/zeta/rest-api-job-lifecycle.md
+++ b/docs/zh/engines/zeta/rest-api-job-lifecycle.md
@@ -148,7 +148,7 @@ curl http://<master>:8080/job-info/<jobId>
|---|---|
| `jobId` | 作业唯一标识 |
| `jobName` | 作业名称 |
-| `jobStatus` | `RUNNING`、`FINISHED`、`FAILED`、`CANCELLED` |
+| `jobStatus` | `RUNNING`、`FINISHED`、`FAILED`、`CANCELED` |
| `envOptions` | 生效的 env 配置 |
| `createTime` | 作业创建时间戳 |
| `jobDag` | DAG 拓扑结构 |
@@ -224,12 +224,9 @@ curl -X POST "http://<master>:8080/stop-job" \
-d '{"jobId": "733584788375093248", "isStopWithSavePoint": true}'
```
-Savepoint 路径会打印在作业日志中，也可通过查询已完成作业获取：
-
-```bash
-curl http://<master>:8080/job-info/733584788375093248 | \
- python3 -c "import sys,json; d=json.load(sys.stdin);
print(d.get('savepointPath', 'N/A'))"
-```
+Savepoint 会写入所配置的 checkpoint 存储（参见所用集群的 checkpoint 存储配置）。`/job-info/:jobId`
的响应中没有 `savepointPath` 字段；
+如需稍后恢复，请使用 `restoreMode=SAVEPOINT`（配合 `restoreSourceJobId`）重新提交，引擎会自动
+选取该作业最新的 Savepoint。完整请求示例见 6.2 节。
### 5.3 强制取消
@@ -319,21 +316,10 @@ curl -X POST
"http://<master>:8080/submit-job?restoreMode=SAVEPOINT&restoreSourc
### 6.3 从指定 Savepoint 路径恢复
-```bash
-curl -X POST http://<master>:8080/submit-job \
- -H "Content-Type: application/json" \
- -d '{
- "env": {
- "job.name": "my-cdc-job-restored",
- "job.mode": "STREAMING",
- "checkpoint.interval": 30000,
- "restore.mode": "savepoint",
- "savepoint.path":
"/seatunnel/checkpoint/savepoint/733584788375093248/1748595600000"
- },
- "source": [ ... ],
- "sink": [ ... ]
- }'
-```
+REST API 不支持指定具体的 Savepoint 路径进行恢复。不存在 `restore.mode` 或 `savepoint.path` 之类的
+env 选项；恢复行为只由请求参数 `restoreMode` 和 `restoreSourceJobId` 控制，并且
+`restoreMode=SAVEPOINT` 总是选取所引用作业**最新**的 Savepoint（按 pipeline 取 checkpoint id
最高的那
+个）。REST API 没有选择更早 Savepoint 的受支持方式。
---
diff --git a/docs/zh/engines/zeta/rest-api-v1.md
b/docs/zh/engines/zeta/rest-api-v1.md
index 96d23a3f5e..90f4675a4a 100644
--- a/docs/zh/engines/zeta/rest-api-v1.md
+++ b/docs/zh/engines/zeta/rest-api-v1.md
@@ -45,7 +45,7 @@ network:
> | 参数名称 | 是否必传 | 参数类型 | 参数描述 |
> |--------|------|------|-----------------------------------------|
-> | type | 是 | string | 插件类型，当前支持 `source` 和 `sink` |
+> | type | 是 | string | 插件类型，支持 `source`、`sink` 和 `transform` |
> | plugin | 是 | string | connector 的 factory identifier，例如 `FakeSource` 或
> `Console` |
#### 响应
@@ -107,6 +107,10 @@ network:
]
},
"expectValue": "TEMPLATE",
+ "compareOperator": null,
+ "compareOption": null,
+ "conditionOperator": "EQUAL",
+ "conditionOperatorCategory": "EQUALITY",
"operator": null,
"next": null
},
@@ -115,7 +119,29 @@ network:
}
}
],
- "conditionRules": []
+ "conditionRules": [],
+ "valueConstraints": [
+ {
+ "expression": "'row.num' >= 0",
+ "conditionTree": {
+ "option": {
+ "key": "row.num",
+ "type": "java.lang.Integer",
+ "defaultValue": 5,
+ "description": "The total number of data generated per degree of
parallelism",
+ "fallbackKeys": [],
+ "optionValues": null
+ },
+ "expectValue": 0,
+ "compareOperator": ">=",
+ "compareOption": null,
+ "conditionOperator": "GREATER_OR_EQUAL",
+ "conditionOperatorCategory": "NUMERIC",
+ "operator": null,
+ "next": null
+ }
+ }
+ ]
}
}
```
@@ -125,6 +151,9 @@ network:
- `requiredOptions[].ruleType` 可能是 `ABSOLUTELY_REQUIRED`、`EXCLUSIVE`、`BUNDLED`
或 `CONDITIONAL`。
- `optionRule.conditionRules` 会递归返回嵌套条件规则；当 connector 未定义嵌套规则时，该字段返回空数组。
- 对于条件规则，会同时返回 `expression` 和 `expressionTree`，便于 Web 做动态表单渲染。
+- `optionRule.valueConstraints` 描述取值级别的校验规则（例如数值范围、字符串模式、跨字段比较）。每个条目包含人类可读的
`expression` 字符串和结构化的 `conditionTree`。当 connector 没有取值约束时该数组为空。
+- 在 `conditionTree` 中，`compareOperator` 在 `EQUAL` 时为
`null`，其余情况下为运行时规则暴露的操作符符号（例如 `>=`、`is not blank`、`extension`）。`compareOption`
仅在跨字段比较时填充。
+- `conditionOperator` 是稳定的操作符标识，取值包括
`EQUAL`、`GREATER_OR_EQUAL`、`NOT_BLANK`、`FIELD_LESS_THAN`、`EXTENSION`
等；`conditionOperatorCategory` 表示操作符类别，例如
`NUMERIC`、`STRING`、`COLLECTION`、`EQUALITY`、`EXTENSION` 等。
</details>
@@ -149,8 +178,9 @@ network:
"gitCommitAbbrev":"DeadD0d0",
"totalSlot":"0",
"unassignedSlot":"0",
- "works":"1",
+ "workers":"1",
"runningJobs":"0",
+ "pendingJobs":"0",
"finishedJobs":"0",
"failedJobs":"0",
"cancelledJobs":"0"
@@ -159,7 +189,7 @@ network:
**注意:**
- 当你使用`dynamic-slot`时, 返回结果中的`totalSlot`和`unassignedSlot`将始终为0. 设置为固定的slot值后,
将正确返回集群中总共的slot数量以及未分配的slot数量.
-- 当添加标签过滤后, `works`, `totalSlot`, `unassignedSlot`将返回满足条件的节点的相关指标.
注意`runningJobs`等job相关指标为集群级别结果, 无法根据标签进行过滤.
+- 当添加标签过滤后, `workers`, `totalSlot`, `unassignedSlot`将返回满足条件的节点的相关指标.
注意`runningJobs`等job相关指标为集群级别结果, 无法根据标签进行过滤.
</details>
@@ -275,13 +305,45 @@ network:
],
"isStartWithSavePoint": false,
"metrics": {
- "sourceReceivedCount": "",
- "sinkWriteCount": ""
+ "SourceReceivedCount": "",
+ "SinkWriteCount": ""
}
}
]
```
+**说明:**
+- `metrics` 对象的键遵循引擎的指标命名：除 `SourceReceivedCount` 和 `SinkWriteCount` 外，还可能包含
`SinkCommitted*`、`*Bytes`、`*Qps`、`IntermediateQueueSize`、`FlushSignal*` 以及
`Table*` 等条目。
+
+</details>
+
+------------------------------------------------------------------------------------------
+
+### 获取运行中作业摘要
+
+<details>
+ <summary><code>GET</code>
<code><b>/hazelcast/rest/maps/running-jobs/summary</b></code> <code>(返回不包含 DAG
和指标的轻量运行中作业列表。)</code></summary>
+
+#### 参数
+
+无。
+
+#### 响应
+
+```json
+[
+ {
+ "jobId": "733584788375666689",
+ "jobName": "rest_api_test",
+ "jobStatus": "RUNNING",
+ "createTime": "2024-08-01 12:00:00"
+ }
+]
+```
+
+**说明:**
+- 这是 `/running-jobs` 的轻量版本，Web UI 会使用该接口：每个作业只返回 `jobId`、`jobName`、`jobStatus`
和 `createTime`，不解析 DAG 和指标。
+
</details>
------------------------------------------------------------------------------------------
@@ -305,6 +367,7 @@ network:
"jobName": "",
"jobStatus": "",
"createTime": "",
+ "startTime": "",
"jobDag": {
"jobId": "",
"envOptions": [],
@@ -346,7 +409,7 @@ network:
"TableSinkCommittedBytes": {},
"TableSinkCommittedBytesPerSeconds": {}
},
- "finishedTime": "",
+ "finishTime": "",
"errorMsg": null,
"envOptions": {
},
@@ -358,7 +421,7 @@ network:
`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` 字段总会返回.
`envOptions`, `pluginJarsUrls`, `isStartWithSavePoint` 字段在Job在RUNNING状态时会返回
-`finishedTime`, `errorMsg` 字段在Job结束时会返回，结束状态为不为RUNNING，可能为FINISHED，可能为CANCEL
+`finishTime`, `errorMsg` 字段在Job结束时会返回，结束状态为不为RUNNING，可能为FINISHED，可能为CANCEL
运行中的Job还会返回 `diagnostics` 字段（状态时间戳与各 Pipeline 的恢复次数），字段说明见 [REST API
V2](rest-api-v2.md)。该字段只在本接口返回，`/running-jobs` 不返回。
#### 指标字段说明
@@ -377,15 +440,16 @@ network:
| SinkCommittedQPS | Sink 已提交速率（行/秒） |
| SinkCommittedBytes | checkpoint 成功后的 Sink 已提交字节数 |
| SinkCommittedBytesPerSeconds | Sink 已提交速率（字节/秒） |
+| IntermediateQueueSize | 中间队列中当前积压的记录数 |
| TableSourceReceived* | 按表汇总的源指标，键格式 `TableSourceReceivedXXX#<表>` |
| TableSinkWrite* | 按表汇总的 Sink 写入尝试，键格式 `TableSinkWriteXXX#<表>` |
| TableSinkCommitted* | 按表汇总的 Sink 已提交指标，键格式 `TableSinkCommittedXXX#<表>` |
-当我们查询不到这个Job时，返回结果为：
+当我们查询不到这个Job时，返回结果只包含请求的 `jobId`：
```json
{
- "jobId" : ""
+ "jobId" : "733584788375666689"
}
```
@@ -414,6 +478,7 @@ network:
"jobName": "",
"jobStatus": "",
"createTime": "",
+ "startTime": "",
"jobDag": {
"jobId": "",
"envOptions": [],
@@ -430,10 +495,10 @@ network:
"pipelineEdges": {}
},
"metrics": {
- "sourceReceivedCount": "",
- "sinkWriteCount": ""
+ "SourceReceivedCount": "",
+ "SinkWriteCount": ""
},
- "finishedTime": "",
+ "finishTime": "",
"errorMsg": null,
"envOptions": {
},
@@ -445,13 +510,13 @@ network:
`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` 字段总会返回.
`envOptions`, `pluginJarsUrls`, `isStartWithSavePoint` 字段在Job在RUNNING状态时会返回
-`finishedTime`, `errorMsg` 字段在Job结束时会返回，结束状态为不为RUNNING，可能为FINISHED，可能为CANCEL
+`finishTime`, `errorMsg` 字段在Job结束时会返回，结束状态为不为RUNNING，可能为FINISHED，可能为CANCEL
-当我们查询不到这个Job时，返回结果为：
+当我们查询不到这个Job时，返回结果只包含请求的 `jobId`：
```json
{
- "jobId" : ""
+ "jobId" : "733584788375666689"
}
```
@@ -625,7 +690,7 @@ network:
```json
{
- "jobId": 733584788375666689,
+ "jobId": "733584788375666689",
"jobName": "rest_api_test"
}
```
@@ -765,7 +830,7 @@ network:
```json
{
-"jobId": 733584788375666689
+"jobId": "733584788375666689"
}
```
@@ -976,13 +1041,11 @@ network:
#### 请求参数
-#### 参数(在请求体中params字段中添加)
-
> | 参数名称 | 是否必传 | 参数类型 | 参数描述
> |
> |----------------------|----------|--------|-----------------------------------|
-> | jobId | optional | string | job id
|
+> | jobId | optional | string | job id，作为路径片段 `/logs/{jobId}`
追加 |
-当`jobId`为空时，返回所有节点的日志信息，否则返回指定`jobId`在所有节点的的日志列表。
+当`jobId`省略时，返回所有节点的日志信息，否则返回指定`jobId`在所有节点的的日志列表。
#### 响应
@@ -995,18 +1058,21 @@ network:
```json
[
{
- "node": "localhost:5801",
- "logLink":
"http://localhost:5801/hazelcast/rest/maps/logs/job-899485770241277953.log";,
+ "node": "localhost:8080",
+ "logLink": "http://localhost:8080/logs/job-899485770241277953.log";,
"logName": "job-899485770241277953.log"
},
{
- "node": "localhost:5801",
- "logLink":
"http://localhost:5801/hazelcast/rest/maps/logs/job-899470314109468673.log";,
+ "node": "localhost:8080",
+ "logLink": "http://localhost:8080/logs/job-899470314109468673.log";,
"logName": "job-899470314109468673.log"
}
]
```
+`logLink` 和 `node` 指向各节点的 Jetty HTTP 服务（默认端口 `8080`，见 `seatunnel.engine.http`），
+而不是 Hazelcast REST 端口。这些端点需要 `seatunnel.engine.http.enable-http: true`（默认为
`false`）。
+
当前支持的格式有`json`和`html`，默认为`html`。
#### 响应大小限制
diff --git a/docs/zh/engines/zeta/rest-api-v2.md
b/docs/zh/engines/zeta/rest-api-v2.md
index 20c22f5fe0..ef4f88c6f3 100644
--- a/docs/zh/engines/zeta/rest-api-v2.md
+++ b/docs/zh/engines/zeta/rest-api-v2.md
@@ -171,7 +171,7 @@ seatunnel:
"conditionRules": [],
"valueConstraints": [
{
- "expression": "'row.num' >= 1",
+ "expression": "'row.num' >= 0",
"conditionTree": {
"option": {
"key": "row.num",
@@ -181,7 +181,7 @@ seatunnel:
"fallbackKeys": [],
"optionValues": null
},
- "expectValue": 1,
+ "expectValue": 0,
"compareOperator": ">=",
"compareOption": null,
"conditionOperator": "GREATER_OR_EQUAL",
@@ -248,7 +248,7 @@ seatunnel:
"gitCommitAbbrev":"DeadD0d0",
"totalSlot":"0",
"unassignedSlot":"0",
- "works":"1",
+ "workers":"1",
"runningJobs":"0",
"pendingJobs":"0",
"finishedJobs":"0",
@@ -259,7 +259,7 @@ seatunnel:
**注意:**
- 当你使用`dynamic-slot`时, 返回结果中的`totalSlot`和`unassignedSlot`将始终为0. 设置为固定的slot值后,
将正确返回集群中总共的slot数量以及未分配的slot数量.
-- 当添加标签过滤后, `works`, `totalSlot`, `unassignedSlot`将返回满足条件的节点的相关指标.
注意`runningJobs`等job相关指标为集群级别结果, 无法根据标签进行过滤.
+- 当添加标签过滤后, `workers`, `totalSlot`, `unassignedSlot`将返回满足条件的节点的相关指标.
注意`runningJobs`等job相关指标为集群级别结果, 无法根据标签进行过滤.
</details>
@@ -367,13 +367,46 @@ seatunnel:
],
"isStartWithSavePoint": false,
"metrics": {
- "sourceReceivedCount": "",
- "sinkWriteCount": ""
+ "SourceReceivedCount": "",
+ "SinkWriteCount": ""
}
}
]
```
+**说明:**
+- `metrics` 对象的键遵循引擎的指标命名：除 `SourceReceivedCount` 和 `SinkWriteCount` 外，还可能包含
`SinkCommitted*`、`*Bytes`、`*Qps`、`IntermediateQueueSize`、`FlushSignal*` 以及
`Table*` 等条目。
+- 当传入 `page` 参数时，响应会被包装为 `{"data": [...], "total": n}`（与 `/finished-jobs`
相同）；不传分页参数时返回上面的裸数组。
+
+</details>
+
+------------------------------------------------------------------------------------------
+
+### 获取运行中作业摘要
+
+<details>
+ <summary><code>GET</code> <code><b>/running-jobs/summary</b></code>
<code>(返回不包含 DAG 和指标的轻量运行中作业列表。)</code></summary>
+
+#### 参数
+
+无。
+
+#### 响应
+
+```json
+[
+ {
+ "jobId": "733584788375666689",
+ "jobName": "rest_api_test",
+ "jobStatus": "RUNNING",
+ "createTime": "2024-08-01 12:00:00"
+ }
+]
+```
+
+**说明:**
+- 这是 `/running-jobs` 的轻量版本，Web UI 会使用该接口：每个作业只返回 `jobId`、`jobName`、`jobStatus`
和 `createTime`，不解析 DAG 和指标。
+
</details>
------------------------------------------------------------------------------------------
@@ -553,6 +586,7 @@ seatunnel:
"jobName": "",
"jobStatus": "",
"createTime": "",
+ "startTime": "",
"jobDag": {
"jobId": "",
"envOptions": [],
@@ -595,7 +629,7 @@ seatunnel:
"TableSinkCommittedBytes": {},
"TableSinkCommittedBytesPerSeconds": {}
},
- "finishedTime": "",
+ "finishTime": "",
"errorMsg": null,
"envOptions": {
},
@@ -633,7 +667,7 @@ seatunnel:
`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` 字段总会返回.
`envOptions`, `pluginJarsUrls`, `isStartWithSavePoint` 字段在Job在RUNNING状态时会返回
-`finishedTime`, `errorMsg` 字段在Job结束时会返回，结束状态为不为RUNNING，可能为FINISHED，可能为CANCEL
+`finishTime`, `errorMsg` 字段在Job结束时会返回，结束状态为不为RUNNING，可能为FINISHED，可能为CANCEL
`diagnostics`
字段在Job运行中且能从Master节点读取到诊断信息时返回。它属于辅助信息，读取失败时该字段会被省略，不会导致请求失败。该字段只在本接口返回，`/running-jobs`
不返回：为列表中的每个Job收集诊断信息会额外增加一次到Master的请求。
#### 指标字段说明
@@ -670,11 +704,11 @@ seatunnel:
| pipelines[].stateTimestamps | Pipeline 进入各状态的时间戳（毫秒）。发生恢复后，新一轮的时间戳会覆盖上一轮 |
| totalPipelineRestoreCount | Job 下所有 Pipeline 的 `restoreCount` 之和 |
-当我们查询不到这个Job时，返回结果为：
+当我们查询不到这个Job时，返回结果只包含请求的 `jobId`：
```json
{
- "jobId" : ""
+ "jobId" : "733584788375666689"
}
```
@@ -703,6 +737,7 @@ seatunnel:
"jobName": "",
"jobStatus": "",
"createTime": "",
+ "startTime": "",
"jobDag": {
"jobId": "",
"envOptions": [],
@@ -719,10 +754,29 @@ seatunnel:
"pipelineEdges": {}
},
"metrics": {
- "sourceReceivedCount": "",
- "sinkWriteCount": ""
+ "IntermediateQueueSize": "",
+ "SourceReceivedCount": "",
+ "SourceReceivedQPS": "",
+ "SourceReceivedBytes": "",
+ "SourceReceivedBytesPerSeconds": "",
+ "SinkWriteCount": "",
+ "SinkWriteQPS": "",
+ "SinkWriteBytes": "",
+ "SinkWriteBytesPerSeconds": "",
+ "SinkCommittedCount": "",
+ "SinkCommittedQPS": "",
+ "SinkCommittedBytes": "",
+ "SinkCommittedBytesPerSeconds": "",
+ "TableSourceReceivedCount": {},
+ "TableSourceReceivedBytes": {},
+ "TableSourceReceivedBytesPerSeconds": {},
+ "TableSourceReceivedQPS": {},
+ "TableSinkWriteCount": {},
+ "TableSinkWriteQPS": {},
+ "TableSinkWriteBytes": {},
+ "TableSinkWriteBytesPerSeconds": {}
},
- "finishedTime": "",
+ "finishTime": "",
"errorMsg": null,
"envOptions": {
},
@@ -734,13 +788,13 @@ seatunnel:
`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` 字段总会返回.
`envOptions`, `pluginJarsUrls`, `isStartWithSavePoint` 字段在Job在RUNNING状态时会返回
-`finishedTime`, `errorMsg` 字段在Job结束时会返回，结束状态为不为RUNNING，可能为FINISHED，可能为CANCEL
+`finishTime`, `errorMsg` 字段在Job结束时会返回，结束状态为不为RUNNING，可能为FINISHED，可能为CANCEL
-当我们查询不到这个Job时，返回结果为：
+当我们查询不到这个Job时，返回结果只包含请求的 `jobId`：
```json
{
- "jobId" : ""
+ "jobId" : "733584788375666689"
}
```
@@ -998,7 +1052,7 @@ INSERT INTO console_sink SELECT * FROM fake_source;
```json
{
- "jobId": 733584788375666689,
+ "jobId": "733584788375666689",
"jobName": "rest_api_test"
}
```
@@ -1046,7 +1100,7 @@ curl --location
'http://127.0.0.1:8080/submit-job/upload?restoreMode=CHECKPOINT&;
```json
{
- "jobId": 733584788375666689,
+ "jobId": "733584788375666689",
"jobName": "SeaTunnel_Job"
}
```
@@ -1185,7 +1239,7 @@ curl --location
'http://127.0.0.1:8080/submit-job/upload?restoreMode=CHECKPOINT&;
```json
{
-"jobId": 733584788375666689
+"jobId": "733584788375666689"
}
```
@@ -1396,13 +1450,11 @@ curl --location
'http://127.0.0.1:8080/submit-job/upload?restoreMode=CHECKPOINT&;
#### 请求参数
-#### 参数(在请求体中params字段中添加)
-
> | 参数名称 | 是否必传 | 参数类型 | 参数描述
> |
> |----------------------|----------|--------|-----------------------------------|
-> | jobId | optional | string | job id
|
+> | jobId | optional | string | job id，作为路径片段 `/logs/{jobId}`
追加 |
-当`jobId`为空时，返回所有节点的日志信息，否则返回指定`jobId`在所有节点的的日志列表。
+当`jobId`省略时，返回所有节点的日志信息，否则返回指定`jobId`在所有节点的的日志列表。
#### 响应
@@ -1475,8 +1527,8 @@ curl --location
'http://127.0.0.1:8080/submit-job/upload?restoreMode=CHECKPOINT&;
#### 例子
-获取当前节点的日志列表：`http://localhost:5801/log`
-获取日志文件内容：`http://localhost:5801/log/job-898380162133917698.log`
+获取当前节点的日志列表：`http://localhost:8080/log`
+获取日志文件内容：`http://localhost:8080/log/job-898380162133917698.log`
日志内容同样受 `seatunnel.engine.http.log-response-max-size-mb` 限制，规则与上面的全节点接口一致。
@@ -1822,10 +1874,21 @@ Checkpoint 信息字段：
"retentionMinutes": 3,
"latestBucketStartMs": 1700000000000
}
- ]
+ ],
+ "collector": {
+ "pollIntervalMs": 5000,
+ "fetchTimeoutMs": 3000,
+ "lastCollectStartMs": 1700000000000,
+ "lastCollectEndMs": 1700000001200,
+ "lastRawMetricsFetchCostMs": 120,
+ "lastRawMetricsBlobs": 2,
+ "collectFailureCount": 0
+ }
}
```
+`collector` 对象反映 Master 上 realtime 采集器的运行状态（轮询间隔、最近一次采集的时间点以及失败计数）。
+
</details>
<details>
diff --git a/docs/zh/engines/zeta/security.md b/docs/zh/engines/zeta/security.md
index e0729e7a37..4b0623beff 100644
--- a/docs/zh/engines/zeta/security.md
+++ b/docs/zh/engines/zeta/security.md
@@ -32,7 +32,7 @@ seatunnel:
| 参数名称 | 是否必填 | 参数描述 |
|--------|---------|--------|
-| `enable-http` | 否 | 是否开启 HTTP 服务，默认为 `true` |
+| `enable-http` | 否 | 是否开启 HTTP 服务，默认为 `false`（发行包中的 `config/seatunnel.yaml`
将其设置为 `true`） |
| `port` | 否 | HTTP 服务端口，默认为 `8080` |
| `enable-https` | 否 | 是否开启 HTTPS 服务，默认为 `false` |
| `https-port` | 否 | HTTPS 服务端口，默认为 `8443` |
diff --git a/docs/zh/engines/zeta/separated-cluster-deployment.md
b/docs/zh/engines/zeta/separated-cluster-deployment.md
index 6fce1b6868..2930dacf06 100644
--- a/docs/zh/engines/zeta/separated-cluster-deployment.md
+++ b/docs/zh/engines/zeta/separated-cluster-deployment.md
@@ -389,14 +389,16 @@ CoordinatorService 提供了每个作业从 LogicalDag 到 ExecutionDag，再到
**max-thread-num**
-同时可执行的最大作业数量
+CoordinatorService 线程池的最大线程数（即线程池的 `maximumPoolSize`），默认值为 `Integer.MAX_VALUE`。
Example
```yaml
-coordinator-service:
- core-thread-num: 30
- max-thread-num: 1000
+seatunnel:
+ engine:
+ coordinator-service:
+ core-thread-num: 30
+ max-thread-num: 1000
```
### 4.9 作业指标分区数量（此参数在 Worker 节点上无效）
diff --git a/docs/zh/engines/zeta/tcp.md b/docs/zh/engines/zeta/tcp.md
index 6d96f4fee9..887240dbb6 100644
--- a/docs/zh/engines/zeta/tcp.md
+++ b/docs/zh/engines/zeta/tcp.md
@@ -34,4 +34,4 @@ hazelcast:
`<members>192.168.1.0-7,192.168.1.21</members>`
-如果您没有为成员提供端口，Hazelcast 会自动尝试端口 `5701`, `5702` 等。
+如果您没有为成员提供端口，Hazelcast 会尝试 `network.port` 中配置的端口（SeaTunnel 默认 `hazelcast.yaml`
为 `5801`，worker 模板为 `5802`）。是否继续尝试后续端口由 `network.port.auto-increment`
控制，SeaTunnel 发行包中该项为 `false`。
diff --git a/docs/zh/engines/zeta/telemetry.md
b/docs/zh/engines/zeta/telemetry.md
index 4e2bd1d14e..25a56b430a 100644
--- a/docs/zh/engines/zeta/telemetry.md
+++ b/docs/zh/engines/zeta/telemetry.md
@@ -188,7 +188,7 @@ worker 发送 `RequestSlotOperation` 请求以预留 slot。这些指标用于
| MetricName | Type | Labels
| 描述 |
|------------|-------|---------------------------------------------------------------------------------------------------------|---------------------|
-| job_count | Gauge | **type**，作业的类型，包括："canceled" "cancelling" "created"
"failed" "failing" "finished" "running" "scheduled" | seatunnel 集群的所有作业计数 |
+| job_count | Gauge | **type**，作业的类型，包括："canceled" "cancelling" "created"
"failed" "failing" "finished" "pending" "running" "scheduled" | seatunnel
集群的所有作业计数 |
### JVM 指标
diff --git a/docs/zh/engines/zeta/user-command.md
b/docs/zh/engines/zeta/user-command.md
index 678ada66e6..fe40b1c2e5 100644
--- a/docs/zh/engines/zeta/user-command.md
+++ b/docs/zh/engines/zeta/user-command.md
@@ -24,6 +24,12 @@ Usage: seatunnel.sh [options]
-can, --cancel, --cancel-job Cancel job(s) by JobId
-f, --force-cancel, --force-cancel-job Force Cancel job(s) by jobId
--check Whether check config (default:
false)
+ --checkpoint-history Get checkpoint history by JobId
+ --checkpoint-history-limit Limit checkpoint history size
(default: 20)
+ --checkpoint-history-pipeline Filter checkpoint history by
pipeline id
+ --checkpoint-history-status Filter checkpoint history by
status:
+ COMPLETED,FAILED,CANCELED
+ --checkpoint-overview Get checkpoint overview by JobId
-cj, --close, --close-job Close client the task will also
be closed
(default: true)
-cn, --cluster The name of cluster
@@ -52,6 +58,7 @@ Usage: seatunnel.sh [options]
-s, --savepoint, --savepoint-job savepoint job by jobId
--sample-limit Maximum rows forwarded from each
source by sample dry-run mode (default: 10, max: 10000)
--sample-print-data Print sampled row values to
persistent logs (default: false)
+ --set-job-id Set custom job id for job
-i, --variable Variable substitution, such as -i
city=beijing, or -i
date=20190318.We use
',' as separator, when inside
"", ',' are
@@ -273,9 +280,9 @@ sh bin/seatunnel-cluster.sh -m -cn my_cluster
**输出示例：**
```
Member ID Address Role
Version
-a1b2c3d4-e5f6-7890-abcd-ef1234567890 192.168.1.100:5701 ACTIVE MASTER
5.3.0
-b2c3d4e5-f6g7-8901-bcde-f23456789012 192.168.1.101:5701 MASTER
5.3.0
-c3d4e5f6-g7h8-9012-cdef-345678901234 192.168.1.102:5701 WORKER
5.3.0
+a1b2c3d4-e5f6-7890-abcd-ef1234567890 192.168.1.100:5801 ACTIVE MASTER
5.3.0
+b2c3d4e5-f6g7-8901-bcde-f23456789012 192.168.1.101:5801 MASTER
5.3.0
+c3d4e5f6-g7h8-9012-cdef-345678901234 192.168.1.102:5801 WORKER
5.3.0
```
**注意**: 必须使用 `-cn` 参数指定集群名称。集群必须处于运行状态才能执行此命令。
diff --git a/docs/zh/faq.md b/docs/zh/faq.md
index f999c9bd02..e472a2ebb3 100644
--- a/docs/zh/faq.md
+++ b/docs/zh/faq.md
@@ -30,7 +30,7 @@ SeaTunnel 支持自定义数据清洗规则。可以在 `transform` 模块中配
SeaTunnel 支持增量数据同步。例如通过 CDC 连接器实现对数据库的增量同步，适用于需要实时捕获数据变更的场景。
## SeaTunnel 目前支持哪些数据源的 CDC ？
-目前支持 MongoDB CDC、MySQL CDC、Opengauss CDC、Oracle CDC、PostgreSQL CDC、Sql Server
CDC、TiDB CDC等，更多请查阅[Source](./connectors/source)。
+目前支持 MongoDB CDC、MySQL CDC、Opengauss CDC、Oracle CDC、PostgreSQL CDC、Sql Server
CDC、TiDB CDC、DB2 CDC、OceanBase CDC、Vitess
CDC等，更多请查阅[Source](./connectors/source)。
## SeaTunnel CDC 同步需要的权限如何开启？
这样就可以了。
@@ -40,9 +40,7 @@ SeaTunnel 支持增量数据同步。例如通过 CDC 连接器实现对数据
支持，是通过订阅 MySQL binlog 日志方式到同步服务器上解析 binlog 日志方式进行
## SeaTunnel 是否支持无主键表的 CDC 同步？
-不支持无主键表的 cdc 同步。原因如下：
-比如上游有 2 条一模一样的数据，然后上游删除或修改了一条，下游由于无法区分到底是哪条需要删除或修改，会出现这 2 条都被删除或修改的情况。
-没主键要类似去重的效果本身有点儿自相矛盾，就像辨别西游记里的真假悟空，到底哪个是真的
+支持，但有限制。没有唯一键时，下游无法区分两条完全相同的记录，更新和删除可能同时影响这两条记录。对于没有主键的表，基于 JDBC 的 CDC
连接器（MySQL、OpenGauss、PostgreSQL、Oracle、SQL
Server、DB2）会回退使用非空的唯一键列作为快照切分列；如果没有合适的键列，则整张表作为一个快照分片读取。也可以通过
`table-names-config.primaryKeys` 或 `snapshotSplitColumn`
显式指定切分列。另有两点限制：并行读取快照需要有这样的键列；开启 `exactly_once = true` 时必须存在主键或非空唯一键，否则作业在启动时报错。
## SeaTunnel 是否支持自动建表？
在同步任务启动之前，可以为目标端已有的表结构选择不同的处理方案。是通过 `schema_save_mode` 参数来控制的。
diff --git a/docs/zh/introduction/concepts/incompatible-changes.md
b/docs/zh/introduction/concepts/incompatible-changes.md
index bf78a745e4..f1de528596 100644
--- a/docs/zh/introduction/concepts/incompatible-changes.md
+++ b/docs/zh/introduction/concepts/incompatible-changes.md
@@ -115,6 +115,25 @@ Zeta SQL ARRAY 函数现在将 TINYINT 元素声明为 `ARRAY<TINYINT>`，与实
并使用 `schema_save_mode = "ERROR_WHEN_SCHEMA_NOT_EXIST"` 保留该表结构。不要对手动定义的目标表使用
`RECREATE_SCHEMA`。任意指定索引前缀长度可能拒绝前缀相同但完整值不同的源主键，因此无法保持原有主键语义。
+### Helm Chart：默认关闭 Zeta REST API v1
+
+- **行为变更：Kubernetes Helm chart 不再默认开启未鉴权的 Zeta REST API v1**
+ - **影响组件**：Helm chart
`deploy/kubernetes/seatunnel`（`conf/hazelcast-master.yaml`、
+ `conf/hazelcast-worker.yaml`、`values.yaml`）
+ - **描述**：此前 chart 将 `hazelcast.network.rest-api.enabled` 设为 `true`，在
Hazelcast 成员端口（5801）上
+ 无鉴权地暴露了已废弃的 Zeta REST API v1（包括
`submit-job`、`stop-job`、`encrypt-config`、日志和线程
+ dump）。现在该项为 `false`，与独立部署的 `config/hazelcast.yaml` 默认值和 v1 文档保持一致。默认的
+ Prometheus Pod 注解也从 `5801`（`/hazelcast/rest/instance/metrics`）改为指向 `8080`
上的 REST API v2 /
+ Jetty 监听（`/metrics`），两者返回相同的采样数据。
+ - **影响**：通过 5801 端口调用 REST API v1 的部署必须切换到 8080 端口的 REST API v2。直接抓取
+ `5801/hazelcast/rest/instance/metrics`（而非通过 Pod 注解）的 Prometheus 配置必须把目标改为
+ `8080/metrics`。通过 Hazelcast 客户端协议提交作业以及 8080 上的 REST API v2 不受影响。由于
ConfigMap 使用
+ `subPath` 挂载且 Deployment 没有配置校验和注解，运行中的 Pod 在重启前会继续使用旧设置，因此在
+ `helm upgrade` 之后请重启 master/worker Pod。
+ - **迁移指南**：使用 8080 端口上的 REST API v2（chart 文档中的接口）。如果确实需要 Zeta REST API
v1，请在自定义
+ ConfigMap（`existingConfigMap`）中设置 `rest-api.enabled: true`，并通过
`NetworkPolicy` 限制成员端口
+ （5801）的访问。升级后重启 Pod 以应用新配置。
+
### SQL ARRAY_MAX 和 ARRAY_MIN 精度
- **行为修正：整数和 DECIMAL 数组极值的精确比较**
diff --git a/docs/zh/introduction/concepts/schema-feature.md
b/docs/zh/introduction/concepts/schema-feature.md
index 1f533bfef8..27d0863806 100644
--- a/docs/zh/introduction/concepts/schema-feature.md
+++ b/docs/zh/introduction/concepts/schema-feature.md
@@ -155,7 +155,7 @@ columns = [
| timestamp_tz | `java.time.OffsetDateTime` | 存储带有 UTC
偏移量的日期和时间信息，包含本地日期时间和 UTC 偏移量。在处理多时区场景时，可以提供更精确的时间信息。
|
| row | `org.apache.seatunnel.api.table.type.SeaTunnelRowType` |
行类型，可以嵌套。
|
| map | `java.util.Map` | Map
是将键映射到值的对象。 键类型包括： `int` `string` `boolean` `tinyint` `smallint` `bigint`
`float` `double` `decimal` `date` `time` `timestamp` `null` , and the value
type includes `int` `string` `boolean` `tinyint` `smallint` `bigint` `float`
`double` `decimal` `date` `time` `timestamp` `null` `array` `map` `row`. |
-| array | `ValueType[]` |
数组是一种表示元素集合的数据类型。 元素类型包括： `int` `string` `boolean` `tinyint` `smallint`
`bigint` `float` `double`.
|
+| array | `ValueType[]` |
数组是一种表示元素集合的数据类型。 元素类型包括： `int` `string` `boolean` `tinyint` `smallint`
`bigint` `float` `double` `map` 以及嵌套的 `array`。
|
#### 如何声明支持的类型
@@ -165,9 +165,9 @@ SeaTunnel 提供了一种简单直接的方式来声明基本类型。基本类
声明复杂类型（例如 **decimal**、**array**、**map** 和 **row**）时，请注意具体注意事项。
- 声明decimal类型时，需要设置精度(precision)和小数位数(scale)，类型定义遵循“decimal(precision,
scale)”格式。 需要强调的是，十进制类型的声明必须用 `"` 括起来；不能像基本类型一样直接使用类型名称。例如，当声明精度为 10、小数位数为 2
的十进制字段时，您可以指定字段类型为`"decimal(10,2)"`。
-- 声明array类型时，需要指定元素类型，类型定义遵循 `array<T>` 格式，其中 `T`
代表元素类型。元素类型包括`int`,`string`,`boolean`,`tinyint`,`smallint`,`bigint`,`float` 和
`double`。与十进制类型声明类似，它也用 `"` 括起来。例如，在声明具有整数数组的字段时，将字段类型指定为 `"array<int>"`。
+- 声明array类型时，需要指定元素类型，类型定义遵循 `array<T>` 格式，其中 `T`
代表元素类型。元素类型包括`int`,`string`,`boolean`,`tinyint`,`smallint`,`bigint`,`float`,`double`、`map<K,V>`
以及嵌套的 `array<T>`。与十进制类型声明类似，它也用 `"` 括起来。例如，在声明具有整数数组的字段时，将字段类型指定为
`"array<int>"`；字符串到整数 map 的数组则为 `"array<map<string, int>>"`。
- 声明map类型时，需要指定键和值类型。map类型定义遵循`map<K,V>`格式，其中`K`表示键类型，`V`表示值类型。
`K`可以是任何基本类型和十进制类型，`V`可以是 SeaTunnel 支持的任何类型。 与之前的类型声明类似，map类型声明必须用双引号引起来。
例如，当声明一个map类型的字段时，键类型为字符串，值类型为整数，则可以将该字段声明为`"map<string, int>"`。
-- 声明row类型时，需要定义一个
[HOCON](https://github.com/lightbend/config/blob/main/HOCON.md) 对象来描述字段及其类型。
字段类型可以是 SeaTunnel 支持的任何类型。 例如，当声明包含整数字段“a”和字符串字段“b”的行类型时，可以将其声明为“{a = int, b =
string}”。 将定义作为字符串括在 `"` 中也是可以接受的，因此 `"{a = int, b = string}"` 相当于 `{a = int, c
= string}`。由于 HOCON 与 JSON 兼容， `"{\"a\":\"int\", \"b\":\"string\"}"` 等价于 `"{a =
int, b = string}"`。
+- 声明row类型时，需要定义一个
[HOCON](https://github.com/lightbend/config/blob/main/HOCON.md) 对象来描述字段及其类型。
字段类型可以是 SeaTunnel 支持的任何类型。 例如，当声明包含整数字段“a”和字符串字段“b”的行类型时，可以将其声明为“{a = int, b =
string}”。 将定义作为字符串括在 `"` 中也是可以接受的，因此 `"{a = int, b = string}"` 相当于 `{a = int, b
= string}`。由于 HOCON 与 JSON 兼容， `"{\"a\":\"int\", \"b\":\"string\"}"` 等价于 `"{a =
int, b = string}"`。
以下是复杂类型声明的示例：
@@ -193,19 +193,19 @@ schema {
### 主键（PrimaryKey）
-主键是用于定义模式中主键的配置，它包含name、columns字段。
+主键是用于定义模式中主键的配置，它包含name、columnNames字段。
```
primaryKey {
name = id
- columns = [id]
+ columnNames = [id]
}
```
-| 字段 | 是否必须 | 默认值 | 描述 |
-|:--------|:-----|:----|---------|
-| name | 是 | - | 主键名称 |
-| columns | 是 | - | 主键中的列列表 |
+| 字段 | 是否必须 | 默认值 | 描述 |
+|:------------|:-----|:----|---------|
+| name | 是 | - | 主键名称 |
+| columnNames | 是 | - | 主键中的列列表 |
### 约束键（constraintKeys）
diff --git a/docs/zh/transforms/data-validator.md
b/docs/zh/transforms/data-validator.md
index cdc221e0e1..9937ee2eda 100644
--- a/docs/zh/transforms/data-validator.md
+++ b/docs/zh/transforms/data-validator.md
@@ -12,7 +12,7 @@ DataValidator 转换插件会根据配置规则校验字段值，并按照指定
|-----------------|--------|----------|--------|
| row_error_handle_way| enum | 否 | FAIL |
| row_error_handle_way.error_table | string | 否 | |
-| field_rules | array | 否 | |
+| field_rules | array | 是（单表模式） | |
### row_error_handle_way [enum]
diff --git a/docs/zh/transforms/dynamic-compile.md
b/docs/zh/transforms/dynamic-compile.md
index 7f5ca99d13..34adc73930 100644
--- a/docs/zh/transforms/dynamic-compile.md
+++ b/docs/zh/transforms/dynamic-compile.md
@@ -42,7 +42,7 @@ SOURCE_CODE,ABSOLUTE_PATH
### absolute_path [string]
-服务器上Java或Groovy文件的绝对路径
+服务器上Java、Groovy或Scala源码文件的绝对路径
### source_code [string]
源代码
diff --git a/docs/zh/transforms/filter.md b/docs/zh/transforms/filter.md
index 32140bf173..156861e8aa 100644
--- a/docs/zh/transforms/filter.md
+++ b/docs/zh/transforms/filter.md
@@ -27,7 +27,7 @@ Filter 转换用于决定表中的哪些字段会传递给下一步。使用 `in
| exclude_fields | Array<String> | 否 | - | 需要删除的字段，其余字段将被保留。与
`include_fields` 互斥。 |
| table_match_regex | String | 否 | `.*` | 转换级别规则所适用的表路径的正则表达式。
|
| table_transform | Array<Map> | 否 | - | 针对单表的规则，参见
[多表转换](transform-multi-table.md)。 |
-| rule_match_mode | String | 否 | - | 当多个 `table_transform` 条目的
`table_path` 相同时，决定它们的匹配方式。 |
+| rule_match_mode | String | 否 | `FIRST_MATCH` | 当多个
`table_transform` 条目的 `table_path` 相同时，决定它们的匹配方式。 |
| common options | - | 否 | - | 转换插件的通用参数，参见 [Transform
Plugin](common-options/common-options.md)。 |
:::caution
diff --git a/docs/zh/transforms/multi-table-transform-and-join-boundary.md
b/docs/zh/transforms/multi-table-transform-and-join-boundary.md
index cac6e33281..987a72d87e 100644
--- a/docs/zh/transforms/multi-table-transform-and-join-boundary.md
+++ b/docs/zh/transforms/multi-table-transform-and-join-boundary.md
@@ -50,7 +50,7 @@ flowchart LR
|---|---|---|
| 按表字段重命名 / 映射 | ✅ 支持 | 使用带 `plugin_input` 和 `table_match_regex` 的
`FieldMapper` |
| 按表列过滤 | ✅ 支持 | 使用带 `plugin_input` 和 `table_match_regex` 的 `Filter` |
-| 按表类型转换 | ✅ 支持 | 使用 `FieldMapper` 的 `define_sink_type` 选项 |
+| 按表类型转换 | ✅ 支持 | 使用 `DefineSinkType` Transform 的 `columns` 选项（可用
`table_transform` 按表配置） |
| 单表 SQL Transform | ✅ 支持 | 使用带 `plugin_input` 的 `SQL` Transform；必要时再用
`table_match_regex` 限定表范围 |
| `TableMerge`——将多表合并为一张 | ✅ 支持 | 各表需具有兼容的 Schema |
| `TableRename`——重命名流中的表 | ✅ 支持 | 适合按表名路由到 Sink |
@@ -78,7 +78,7 @@ Schema。通常用于将多表路由至同一个 Sink。
"plugin_name": "TableMerge",
"plugin_input": ["orders_2023", "orders_2024"],
"plugin_output": "all_orders",
- "merge_by_field": true
+ "table": "all_orders"
}
```
@@ -176,13 +176,17 @@ PostgreSQL-CDC 流进行 JOIN）。
{
"plugin_name": "Kafka",
"plugin_input": ["cdc_stream"],
- "topic": "products",
- "table_match_regex": "mydb\\.products"
+ "topic": "products"
}
]
}
```
+带 `table_match_regex` 的 Transform 只会改写匹配到的表；其输入流中的其他表会原样透传，而 Sink 会接收其
+`plugin_input` 所订阅流中的全部表——Sink 无法按表名过滤。因此在上面的示例中，每个 Kafka Sink 仍然会
+收到其输入流中的全部三张表。如果要让某个 Sink 只接收一张表，请在它前面加一个 `TableFilter` Transform
+（例如 `database_pattern = "mydb"`、`table_pattern = "products"`）。
+
---
## 6. 公共字段示例（共享 Schema）
@@ -195,6 +199,7 @@ PostgreSQL-CDC 流进行 JOIN）。
"plugin_name": "TableMerge",
"plugin_input": ["cdc_stream"],
"plugin_output": "all_events",
+ "table": "all_events",
"table_match_regex": "mydb\\.(orders|payments|refunds)"
},
{
@@ -269,8 +274,10 @@ Transform。若各表 Schema 不同，则必须拆成多个 Transform 块，通
**Q: 未被任何 Transform 匹配的表会怎样？**
-未匹配的表会继续在流水线中流动，可被使用正确 `plugin_input` 和 `table_match_regex`
-的 Sink 捕获。
+未匹配的表会保持原样继续在流水线中流动。Sink 会接收其 `plugin_input` 所订阅流中的所有表——
+Sink 没有 `table_match_regex` 选项。如果需要在 Sink 前选择或路由特定表，请使用带
+`table_match_regex`/`table_transform` 的 Transform（例如
`TableFilter`、`TableRename` 或
+`TableMerge`）。
**Q: 能否在不停机的情况下向现有 CDC 流水线新增表？**
diff --git a/docs/zh/transforms/recipes.md b/docs/zh/transforms/recipes.md
index ed691da0b5..dab10665f2 100644
--- a/docs/zh/transforms/recipes.md
+++ b/docs/zh/transforms/recipes.md
@@ -137,7 +137,7 @@ Transform V2 的 SPI 设计上同时支持：
- map（1 行输入 -> 1 行输出）
- flatMap（1 行输入 -> N 行输出）
-但目前文档层面并没有一个“内置的 explode（把数组字段拆成多行）” transform。如果你的 JSON 里有 `items: [...]`
这类数组字段，并希望把每个 item 展开成独立行，通常有以下选择：
+如果要把 `items: [...]` 这类数组字段展开成“每个元素一行”，可以使用 `Sql` Transform 的 `LATERAL VIEW
EXPLODE`（或保留空数组的 `OUTER EXPLODE`）；参见 [SQL 函数](sql-functions.md)。其他选择：
- 在上游预处理（例如 Flink/Spark SQL，或由上游生产端直接输出“一行一条目标记录”的 JSON）。
- 写一个自定义 Transform V2（flatMap）插件来实现 1:N 拆分。
diff --git a/docs/zh/transforms/transform-multi-table.md
b/docs/zh/transforms/transform-multi-table.md
index c54226cab7..679c0d6c8b 100644
--- a/docs/zh/transforms/transform-multi-table.md
+++ b/docs/zh/transforms/transform-multi-table.md
@@ -21,7 +21,7 @@ SeaTunnel 的 Multi-Table Transform 支持在上游插件输出多个表时，
| table_match_regex | String | No | .* |
表名的正则表达式，通过正则表达式来匹配需要进行转换的表，默认匹配所有的表。注意这个表名是上游的真正表名，不是`plugin_output`。
|
| table_transform | List | No | - |
可以通过table_transform列表来指定部分表的规则，当在table_transform中配置某个表的转换规则后，外层针对当前表的规则不会生效，以table_transform中的为准
|
| table_transform.table_path | String | Yes | - |
当在table_transform中配置某个表的转换规则后，需要使用table_path字段指定表名，表名需要包含`databaseName[.schemaName].tableName`，并且按完整表路径精确匹配，不能为空。
|
-| rule_match_mode | String | No | - | 控制多个
`table_transform` 配置使用相同精确 `table_path` 时的执行方式，支持 `FIRST_MATCH` 和 `ALL_MATCH`。 |
+| rule_match_mode | String | No | `FIRST_MATCH` | 控制多个
`table_transform` 配置使用相同精确 `table_path` 时的执行方式，支持 `FIRST_MATCH` 和 `ALL_MATCH`。 |
## 匹配逻辑
