This is an automated email from the ASF dual-hosted git repository.
davidzollo pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/seatunnel.git
The following commit(s) were added to refs/heads/dev by this push:
new 068d5a414b [Docs][Connector-V2] Improve MySQL Oracle and SQL Server
docs (#11367)
068d5a414b is described below
commit 068d5a414bf40ddc1ab70db0503f20ec34cebe4b
Author: Daniel Carter <[email protected]>
AuthorDate: Tue Jul 7 17:50:39 2026 +0800
[Docs][Connector-V2] Improve MySQL Oracle and SQL Server docs (#11367)
---
docs/en/connectors/sink/Mysql.md | 12 +++++++-----
docs/en/connectors/sink/Oracle.md | 12 +++++++-----
docs/en/connectors/sink/SqlServer.md | 16 ++++++++++++----
docs/en/connectors/source/Mysql.md | 13 ++++++++-----
docs/en/connectors/source/Oracle.md | 12 ++++++++----
docs/en/connectors/source/SqlServer.md | 12 ++++++++----
docs/zh/connectors/sink/Mysql.md | 15 +++++++++------
docs/zh/connectors/sink/Oracle.md | 13 +++++++------
docs/zh/connectors/sink/SqlServer.md | 16 ++++++++++++----
docs/zh/connectors/source/Mysql.md | 15 +++++++++------
docs/zh/connectors/source/Oracle.md | 14 +++++++++++---
docs/zh/connectors/source/SqlServer.md | 12 ++++++++----
12 files changed, 106 insertions(+), 56 deletions(-)
diff --git a/docs/en/connectors/sink/Mysql.md b/docs/en/connectors/sink/Mysql.md
index 4207b48934..ccf3c5c216 100644
--- a/docs/en/connectors/sink/Mysql.md
+++ b/docs/en/connectors/sink/Mysql.md
@@ -34,16 +34,16 @@ semantics (using XA transaction guarantee).
- [x] [exactly-once](../../introduction/concepts/connector-v2-features.md)
- [x] [cdc](../../introduction/concepts/connector-v2-features.md)
- [x] [support multiple table
write](../../introduction/concepts/connector-v2-features.md)
+- [x] [timer flush](../../introduction/concepts/connector-v2-features.md)
> Use `Xa transactions` to ensure `exactly-once`. So only support
> `exactly-once` for the database which is
-> support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
-- [ ] [timer flush](../../introduction/concepts/connector-v2-features.md)
+> support `Xa transactions`. You can set `is_exactly_once=true` and
`max_retries=0` to enable it.
## Supported DataSource Info
| Datasource | Supported Versions |
Driver | Url |
Maven |
|------------|----------------------------------------------------------|--------------------------|---------------------------------------|---------------------------------------------------------------------------|
-| Mysql | Different dependency version has different driver class. |
com.mysql.cj.jdbc.Driver | jdbc:mysql://localhost:3306:3306/test |
[Download](https://mvnrepository.com/artifact/mysql/mysql-connector-java) |
+| Mysql | Different dependency version has different driver class. |
com.mysql.cj.jdbc.Driver | jdbc:mysql://localhost:3306/test |
[Download](https://mvnrepository.com/artifact/mysql/mysql-connector-java) |
## Data Type Mapping
@@ -69,7 +69,7 @@ semantics (using XA transaction guarantee).
| Name | Type | Required |
Default |
Description
|
|-------------------------------------------|---------|----------|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| url | String | Yes | -
| The URL of the JDBC connection. Refer to a case:
jdbc:mysql://localhost:3306:3306/test
|
+| url | String | Yes | -
| The URL of the JDBC connection. Refer to a case:
jdbc:mysql://localhost:3306/test
|
| driver | String | Yes | -
| The jdbc class name used to connect to the remote data
source,<br/> if you use MySQL the value is `com.mysql.cj.jdbc.Driver`.
|
| username | String | No | -
| Connection instance user name
|
| password | String | No | -
| Connection instance password
|
@@ -79,7 +79,8 @@ semantics (using XA transaction guarantee).
| primary_keys | Array | No | -
| This option is used to support operations such as `insert`,
`delete`, and `update` when automatically generate sql.
|
| connection_check_timeout_sec | Int | No | 30
| The time in seconds to wait for the database operation used
to validate the connection to complete.
|
| max_retries | Int | No | 0
| The number of retries to submit failed (executeBatch)
|
-| batch_size | Int | No | 1000
| For batch writing, when the number of buffered records
reaches the number of `batch_size` or the time reaches
`checkpoint.interval`<br/>, the data will be flushed into the database
|
+| batch_size | Int | No | 1000
| For batch writing, when the number of buffered records
reaches `batch_size`, the data will be flushed into the database. If
`batch_interval_ms` is greater than 0, elapsed time can also trigger a flush.
|
+| batch_interval_ms | Long | No | 0
| Write-triggered flush interval in milliseconds. `0` disables
time-based flushing. When greater than 0, the writer checks elapsed time on
each record and flushes synchronously when the interval is reached.
|
| is_exactly_once | Boolean | No | false
| Whether to enable exactly-once semantics, which will use Xa
transactions. If on, you need to<br/>set `xa_data_source_class_name`.
|
| generate_sink_sql | Boolean | No | false
| Generate sql statements based on the database table you want
to write to
|
| xa_data_source_class_name | String | No | -
| The xa data source class name of the database Driver, for
example, mysql is `com.mysql.cj.jdbc.MysqlXADataSource`, and<br/>please refer
to appendix for other data sources
|
@@ -93,6 +94,7 @@ semantics (using XA transaction guarantee).
| data_save_mode | Enum | No | APPEND_DATA
| Before the synchronous task is turned on, different
processing schemes are selected for data existing data on the target side.
|
| custom_sql | String | No | -
| When data_save_mode selects CUSTOM_PROCESSING, you should
fill in the CUSTOM_SQL parameter. This parameter usually fills in a SQL that
can be executed. SQL will be executed before synchronization tasks.
|
| enable_upsert | Boolean | No | true
| Enable upsert by primary_keys exist, If the task only has
`insert`, setting this parameter to `false` can speed up data import
|
+| multi_table_sink_replica | Int | No | 1
| The number of sink writer replicas used when writing
multiple tables.
|
### Tips
diff --git a/docs/en/connectors/sink/Oracle.md
b/docs/en/connectors/sink/Oracle.md
index 0fbdbd33f8..b142dfdc87 100644
--- a/docs/en/connectors/sink/Oracle.md
+++ b/docs/en/connectors/sink/Oracle.md
@@ -29,10 +29,11 @@ semantics (using XA transaction guarantee).
- [x] [exactly-once](../../introduction/concepts/connector-v2-features.md)
- [x] [cdc](../../introduction/concepts/connector-v2-features.md)
+- [x] [support multiple table
write](../../introduction/concepts/connector-v2-features.md)
+- [x] [timer flush](../../introduction/concepts/connector-v2-features.md)
> Use `Xa transactions` to ensure `exactly-once`. So only support
> `exactly-once` for the database which is
-> support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
-- [ ] [timer flush](../../introduction/concepts/connector-v2-features.md)
+> support `Xa transactions`. You can set `is_exactly_once=true` and
`max_retries=0` to enable it.
## Supported DataSource Info
@@ -77,8 +78,8 @@ semantics (using XA transaction guarantee).
| primary_keys | Array | No | -
| This option is used to support operations such as `insert`,
`delete`, and `update` when automatically generate sql.
|
| connection_check_timeout_sec | Int | No | 30
| The time in seconds to wait for the database operation used
to validate the connection to complete.
|
| max_retries | Int | No | 0
| The number of retries to submit failed (executeBatch)
|
-| batch_size | Int | No | 1000
| For batch writing, when the number of buffered records
reaches the number of `batch_size` or the time reaches
`batch_interval_ms`<br/>, the data will be flushed into the database
|
-| batch_interval_ms | Int | No | 1000
| For batch writing, when the number of buffers reaches the
number of `batch_size` or the time reaches `batch_interval_ms`, the data will
be flushed into the database
|
+| batch_size | Int | No | 1000
| For batch writing, when the number of buffered records
reaches `batch_size`, the data will be flushed into the database. If
`batch_interval_ms` is greater than 0, elapsed time can also trigger a flush.
|
+| batch_interval_ms | Long | No | 0
| Write-triggered flush interval in milliseconds. `0` disables
time-based flushing. When greater than 0, the writer checks elapsed time on
each record and flushes synchronously when the interval is reached.
|
| is_exactly_once | Boolean | No | false
| Whether to enable exactly-once semantics, which will use Xa
transactions. If on, you need to<br/>set `xa_data_source_class_name`.
|
| generate_sink_sql | Boolean | No | false
| Generate sql statements based on the database table you want
to write to.
|
| xa_data_source_class_name | String | No | -
| The xa data source class name of the database Driver, for
example, Oracle is `oracle.jdbc.xa.client.OracleXADataSource`, and<br/>please
refer to appendix for other data sources
|
@@ -91,6 +92,7 @@ semantics (using XA transaction guarantee).
| data_save_mode | Enum | No | APPEND_DATA
| Before the synchronous task is turned on, different
processing schemes are selected for data existing data on the target side.
|
| custom_sql | String | No | -
| When data_save_mode selects CUSTOM_PROCESSING, you should
fill in the CUSTOM_SQL parameter. This parameter usually fills in a SQL that
can be executed. SQL will be executed before synchronization tasks.
|
| enable_upsert | Boolean | No | true
| Enable upsert by primary_keys exist, If the task has no key
duplicate data, setting this parameter to `false` can speed up data import
|
+| multi_table_sink_replica | Int | No | 1
| The number of sink writer replicas used when writing
multiple tables.
|
### Tips
@@ -209,4 +211,4 @@ sink {
## Changelog
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
diff --git a/docs/en/connectors/sink/SqlServer.md
b/docs/en/connectors/sink/SqlServer.md
index d967a343c8..7caa49168d 100644
--- a/docs/en/connectors/sink/SqlServer.md
+++ b/docs/en/connectors/sink/SqlServer.md
@@ -33,10 +33,11 @@ semantics (using XA transaction guarantee).
- [x] [exactly-once](../../introduction/concepts/connector-v2-features.md)
- [x] [cdc](../../introduction/concepts/connector-v2-features.md)
+- [x] [support multiple table
write](../../introduction/concepts/connector-v2-features.md)
+- [x] [timer flush](../../introduction/concepts/connector-v2-features.md)
> Use `Xa transactions` to ensure `exactly-once`. So only support
> `exactly-once` for the database which is
-> support `Xa transactions`. You can set `is_exactly_once=true` to enable it.
-- [ ] [timer flush](../../introduction/concepts/connector-v2-features.md)
+> support `Xa transactions`. You can set `is_exactly_once=true` and
`max_retries=0` to enable it.
## Supported DataSource Info
@@ -80,15 +81,21 @@ semantics (using XA transaction guarantee).
| primary_keys | Array | No | - |
This option is used to support operations such as `insert`, `delete`, and
`update` when automatically generate sql.
|
| connection_check_timeout_sec | Int | No | 30 |
The time in seconds to wait for the database operation used to validate the
connection to complete.
|
| max_retries | Int | No | 0 |
The number of retries to submit failed (executeBatch)
|
-| batch_size | Int | No | 1000 |
For batch writing, when the number of buffered records reaches the number of
`batch_size` or the time reaches `checkpoint.interval`<br/>, the data will be
flushed into the database
|
+| batch_size | Int | No | 1000 |
For batch writing, when the number of buffered records reaches `batch_size`,
the data will be flushed into the database. If `batch_interval_ms` is greater
than 0, elapsed time can also trigger a flush.
|
+| batch_interval_ms | Long | No | 0 |
Write-triggered flush interval in milliseconds. `0` disables time-based
flushing. When greater than 0, the writer checks elapsed time on each record
and flushes synchronously when the interval is reached.
|
| is_exactly_once | Boolean | No | false |
Whether to enable exactly-once semantics, which will use Xa transactions. If
on, you need to<br/>set `xa_data_source_class_name`.
|
| generate_sink_sql | Boolean | No | false |
Generate sql statements based on the database table you want to write to
|
| xa_data_source_class_name | String | No | - |
The xa data source class name of the database Driver, for example, SqlServer is
`com.microsoft.sqlserver.jdbc.SQLServerXADataSource`, and<br/>please refer to
appendix for other data sources |
| max_commit_attempts | Int | No | 3 |
The number of retries for transaction commit failures
|
| transaction_timeout_sec | Int | No | -1 |
The timeout after the transaction is opened, the default is -1 (never timeout).
Note that setting the timeout may affect<br/>exactly-once semantics
|
| auto_commit | Boolean | No | true |
Automatic transaction commit is enabled by default
|
+| properties | Map | No | - |
Additional JDBC connection parameters. When the same parameter exists in both
`properties` and the URL, priority depends on the SQL Server JDBC driver.
|
| common-options | | no | - |
Sink plugin common parameters, please refer to [Sink Common
Options](../common-options/sink-common-options.md) for details
|
+| schema_save_mode | Enum | No |
CREATE_SCHEMA_WHEN_NOT_EXIST | Before the synchronization task starts, controls
how the target table schema is handled.
|
+| data_save_mode | Enum | No | APPEND_DATA
| Before the synchronization task starts, controls how existing target table
data is handled.
|
+| custom_sql | String | No | - |
When `data_save_mode` is `CUSTOM_PROCESSING`, fill this option with SQL that
can be executed before synchronization starts.
|
| enable_upsert | Boolean | No | true |
Enable upsert by primary_keys exist, If the task has no key duplicate data,
setting this parameter to `false` can speed up data import
|
+| multi_table_sink_replica | Int | No | 1 |
The number of sink writer replicas used when writing multiple tables.
|
## tips
@@ -172,6 +179,7 @@ Jdbc {
url = "jdbc:sqlserver://localhost:1433;databaseName=column_type_test"
username = SA
password = "Y.sa123456"
+ max_retries = 0
query = "insert into full_types_jdbc_sink( id, val_char, val_varchar,
val_text, val_nchar, val_nvarchar, val_ntext, val_decimal, val_numeric,
val_float, val_real, val_smallmoney, val_money, val_bit, val_tinyint,
val_smallint, val_int, val_bigint, val_date, val_time, val_datetime2,
val_datetime, val_smalldatetime ) values( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ? )"
is_exactly_once = "true"
@@ -184,4 +192,4 @@ Jdbc {
## Changelog
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
diff --git a/docs/en/connectors/source/Mysql.md
b/docs/en/connectors/source/Mysql.md
index 575da1a528..16c2e7b88f 100644
--- a/docs/en/connectors/source/Mysql.md
+++ b/docs/en/connectors/source/Mysql.md
@@ -44,7 +44,7 @@ Read external data source data through JDBC.
| Datasource | Supported versions |
Driver | Url |
Maven |
|------------|----------------------------------------------------------|--------------------------|---------------------------------------|---------------------------------------------------------------------------|
-| Mysql | Different dependency version has different driver class. |
com.mysql.cj.jdbc.Driver | jdbc:mysql://localhost:3306:3306/test |
[Download](https://mvnrepository.com/artifact/mysql/mysql-connector-java) |
+| Mysql | Different dependency version has different driver class. |
com.mysql.cj.jdbc.Driver | jdbc:mysql://localhost:3306/test |
[Download](https://mvnrepository.com/artifact/mysql/mysql-connector-java) |
## Data Type Mapping
@@ -71,11 +71,11 @@ Read external data source data through JDBC.
| Name | Type | Required | Default
| Description
[...]
|--------------------------------------------|------------|----------|-----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[...]
-| url | String | Yes | -
| The URL of the JDBC connection. Refer to a case:
jdbc:mysql://localhost:3306:3306/test
[...]
+| url | String | Yes | -
| The URL of the JDBC connection. Refer to a case:
jdbc:mysql://localhost:3306/test
[...]
| driver | String | Yes | -
| The jdbc class name used to connect to the remote data source,<br/>
if you use MySQL the value is `com.mysql.cj.jdbc.Driver`.
[...]
| username | String | No | -
| Connection instance user name
[...]
| password | String | No | -
| Connection instance password
[...]
-| query | String | Yes | -
| Query statement
[...]
+| query | String | No | -
| Query statement. Required when neither `table_path` nor `table_list`
is configured.
[...]
| connection_check_timeout_sec | Int | No | 30
| The time in seconds to wait for the database operation used to
validate the connection to complete
[...]
| partition_column | String | No | -
| The column name for parallelism's partition, only support numeric
type,Only support numeric type primary key, and only can config one column.
[...]
| partition_lower_bound | BigDecimal | No | -
| The partition_column min value for scan, if not set SeaTunnel will
query database get min value.
[...]
@@ -85,12 +85,15 @@ Read external data source data through JDBC.
| properties | Map | No | -
| Additional connection configuration parameters,when properties and
URL have the same parameters, the priority is determined by the <br/>specific
implementation of the driver. For example, in MySQL, properties take precedence
over the URL.
[...]
| use_regex | Boolean | No | false
| Control regular expression matching for table_path. When set to
`true`, the table_path will be treated as a regular expression pattern. When
set to `false` or not specified, the table_path will be treated as an exact
path (no regex matching).
[...]
| table_path | String | No | -
| The path to the full path of table, you can use this configuration
instead of `query`. <br/>example: <br/>"testdb.table1"
[...]
-| table_list | Array | No | -
| The list of tables to be read, you can use this configuration instead
of `table_path` example: ```[{ table_path = "testdb.table1"}, {table_path =
"testdb.table2", query = "select * id, name from testdb.table2"}]```
[...]
+| table_list | Array | No | -
| The list of tables to be read, you can use this configuration instead
of `table_path` example: ```[{ table_path = "testdb.table1"}, {table_path =
"testdb.table2", query = "select id, name from testdb.table2"}]```
[...]
| where_condition | String | No | -
| Common row filter conditions for all tables/queries, must start with
`where`. for example `where id > 100`
[...]
| split.size | Int | No | 8096
| The split size (number of rows) of table, captured tables are split
into multiple splits when read of table.
[...]
| split.even-distribution.factor.lower-bound | Double | No | 0.05
| The lower bound of the chunk key distribution factor. This factor is
used to determine whether the table data is evenly distributed. If the
distribution factor is calculated to be greater than or equal to this lower
bound (i.e., (MAX(id) - MIN(id) + 1) / row count), the table chunks would be
optimized for even distribution. Otherwise, if the distribution factor is less,
the table will be considered a [...]
| split.even-distribution.factor.upper-bound | Double | No | 100
| The upper bound of the chunk key distribution factor. This factor is
used to determine whether the table data is evenly distributed. If the
distribution factor is calculated to be less than or equal to this upper bound
(i.e., (MAX(id) - MIN(id) + 1) / row count), the table chunks would be
optimized for even distribution. Otherwise, if the distribution factor is
greater, the table will be considered a [...]
-| split.sample-sharding.threshold | Int | No | 10000
| This configuration specifies the threshold of estimated shard count
to trigger the sample sharding strategy. When the distribution factor is
outside the bounds specified by
`chunk-key.even-distribution.factor.upper-bound` and
`chunk-key.even-distribution.factor.lower-bound`, and the estimated shard count
(calculated as approximate row count / chunk size) exceeds this threshold, the
sample sharding st [...]
+| split.sample-sharding.threshold | Int | No | 1000
| This configuration specifies the threshold of estimated shard count
to trigger the sample sharding strategy. When the distribution factor is
outside the bounds specified by
`chunk-key.even-distribution.factor.upper-bound` and
`chunk-key.even-distribution.factor.lower-bound`, and the estimated shard count
(calculated as approximate row count / chunk size) exceeds this threshold, the
sample sharding st [...]
+| split.allow-sampling | Boolean | No | true
| Whether to allow sampling-based sharding for uneven split keys. When
set to `false`, SeaTunnel falls back to iterative uneven chunk splitting.
[...]
+| use_select_count | Boolean | No | false
| Whether to use `select count(*)` to estimate table row count before
splitting.
[...]
+| skip_analyze | Boolean | No | false
| Whether to skip table row-count analysis before splitting.
[...]
| split.inverse-sampling.rate | Int | No | 1000
| The inverse of the sampling rate used in the sample sharding
strategy. For example, if this value is set to 1000, it means a 1/1000 sampling
rate is applied during the sampling process. This option provides flexibility
in controlling the granularity of the sampling, thus affecting the final number
of shards. It's especially useful when dealing with very large datasets where a
lower sampling rate is p [...]
| int_type_narrowing | Boolean | No | true
| Int type narrowing, if true, the tinyint(1) type will be narrowed to
the boolean type if without loss of precision. Support for MySQL at now. Please
refer to `int_type_narrowing` below
[...]
| common-options | | No | -
| Source plugin common parameters, please refer to [Source Common
Options](../common-options/source-common-options.md) for details
[...]
diff --git a/docs/en/connectors/source/Oracle.md
b/docs/en/connectors/source/Oracle.md
index b26accc384..05c236ee65 100644
--- a/docs/en/connectors/source/Oracle.md
+++ b/docs/en/connectors/source/Oracle.md
@@ -22,6 +22,7 @@ Read external data source data through JDBC.
- [x] [column projection](../../introduction/concepts/connector-v2-features.md)
- [x] [parallelism](../../introduction/concepts/connector-v2-features.md)
- [x] [support user-defined
split](../../introduction/concepts/connector-v2-features.md)
+- [x] [support multiple table
reading](../../introduction/concepts/connector-v2-features.md)
> supports query SQL and can achieve projection effect.
@@ -68,7 +69,7 @@ Read external data source data through JDBC.
| driver | String | Yes | - | The
jdbc class name used to connect to the remote data source,<br/> if you use
Oracle the value is `oracle.jdbc.OracleDriver`.
|
| username | String | No | - |
Connection instance user name
|
| password | String | No | - |
Connection instance password
|
-| query | String | Yes | - |
Query statement
|
+| query | String | No | - |
Query statement. Required when neither `table_path` nor `table_list` is
configured.
|
| connection_check_timeout_sec | Int | No | 30 | The
time in seconds to wait for the database operation used to validate the
connection to complete
|
| partition_column | String | No | - | The
column name for parallelism's partition, only support numeric type,Only support
numeric type primary key, and only can config one column.
|
| partition_lower_bound | BigDecimal | No | - | The
partition_column min value for scan, if not set SeaTunnel will query database
get min value.
|
@@ -78,12 +79,15 @@ Read external data source data through JDBC.
| properties | Map | No | - |
Additional connection configuration parameters,when properties and URL have the
same parameters, the priority is determined by the <br/>specific implementation
of the driver. For example, in Oracle, properties take precedence over the URL.
|
| use_regex | Boolean | No | false |
Control regular expression matching for table_path. When set to `true`, the
table_path will be treated as a regular expression pattern. When set to `false`
or not specified, the table_path will be treated as an exact path (no regex
matching). |
| table_path | String | No | -
| The path to the full path of table, you can use this configuration
instead of `query`. <br/>example: <br/>"test_schema.table1"
[...]
-| table_list | Array | No | -
| The list of tables to be read, you can use this configuration instead
of `table_path` example: ```[{ table_path = "testdb.table1"}, {table_path =
"testdb.table2", query = "select * id, name from testdb.table2"}]```
[...]
+| table_list | Array | No | -
| The list of tables to be read, you can use this configuration instead
of `table_path` example: ```[{ table_path = "testdb.table1"}, {table_path =
"testdb.table2", query = "select id, name from testdb.table2"}]```
[...]
| where_condition | String | No | -
| Common row filter conditions for all tables/queries, must start with
`where`. for example `where id > 100`
[...]
| split.size | Int | No | 8096
| The split size (number of rows) of table, captured tables are split
into multiple splits when read of table.
[...]
| split.even-distribution.factor.lower-bound | Double | No | 0.05
| The lower bound of the chunk key distribution factor. This factor is
used to determine whether the table data is evenly distributed. If the
distribution factor is calculated to be greater than or equal to this lower
bound (i.e., (MAX(id) - MIN(id) + 1) / row count), the table chunks would be
optimized for even distribution. Otherwise, if the distribution factor is less,
the table will be considered a [...]
| split.even-distribution.factor.upper-bound | Double | No | 100
| The upper bound of the chunk key distribution factor. This factor is
used to determine whether the table data is evenly distributed. If the
distribution factor is calculated to be less than or equal to this upper bound
(i.e., (MAX(id) - MIN(id) + 1) / row count), the table chunks would be
optimized for even distribution. Otherwise, if the distribution factor is
greater, the table will be considered a [...]
-| split.sample-sharding.threshold | Int | No | 10000
| This configuration specifies the threshold of estimated shard count
to trigger the sample sharding strategy. When the distribution factor is
outside the bounds specified by
`chunk-key.even-distribution.factor.upper-bound` and
`chunk-key.even-distribution.factor.lower-bound`, and the estimated shard count
(calculated as approximate row count / chunk size) exceeds this threshold, the
sample sharding st [...]
+| split.sample-sharding.threshold | Int | No | 1000
| This configuration specifies the threshold of estimated shard count
to trigger the sample sharding strategy. When the distribution factor is
outside the bounds specified by
`chunk-key.even-distribution.factor.upper-bound` and
`chunk-key.even-distribution.factor.lower-bound`, and the estimated shard count
(calculated as approximate row count / chunk size) exceeds this threshold, the
sample sharding st [...]
+| split.allow-sampling | Boolean | No | true
| Whether to allow sampling-based sharding for uneven split keys. When
set to `false`, SeaTunnel falls back to iterative uneven chunk splitting.
[...]
+| use_select_count | Boolean | No | false
| Whether to use `select count(*)` to estimate table row count before
splitting.
[...]
+| skip_analyze | Boolean | No | false
| Whether to skip table row-count analysis before splitting.
[...]
| split.inverse-sampling.rate | Int | No | 1000
| The inverse of the sampling rate used in the sample sharding
strategy. For example, if this value is set to 1000, it means a 1/1000 sampling
rate is applied during the sampling process. This option provides flexibility
in controlling the granularity of the sampling, thus affecting the final number
of shards. It's especially useful when dealing with very large datasets where a
lower sampling rate is p [...]
| decimal_type_narrowing | Boolean | No | true
| Decimal type narrowing, if true, the decimal type will be narrowed to
the int or long type if without loss of precision. Only support for Oracle at
now. Please refer to `decimal_type_narrowing` below
[...]
| common-options | | No | -
| Source plugin common parameters, please refer to [Source Common
Options](../common-options/source-common-options.md) for details
[...]
@@ -329,4 +333,4 @@ sink {
## Changelog
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
diff --git a/docs/en/connectors/source/SqlServer.md
b/docs/en/connectors/source/SqlServer.md
index 92944e0d18..3ea567eb65 100644
--- a/docs/en/connectors/source/SqlServer.md
+++ b/docs/en/connectors/source/SqlServer.md
@@ -32,6 +32,7 @@ import ChangeLog from '../changelog/connector-jdbc.md';
- [x] [column projection](../../introduction/concepts/connector-v2-features.md)
- [x] [parallelism](../../introduction/concepts/connector-v2-features.md)
- [x] [support user-defined
split](../../introduction/concepts/connector-v2-features.md)
+- [x] [support multiple table
reading](../../introduction/concepts/connector-v2-features.md)
> supports query SQL and can achieve projection effect.
@@ -75,7 +76,7 @@ Read external data source data through JDBC.
| driver | String | Yes | -
| The jdbc class name used to connect to the remote data source,<br/> if
you use SQLserver the value is `com.microsoft.sqlserver.jdbc.SQLServerDriver`.
[...]
| username | String | No | -
| Connection instance user name
[...]
| password | String | No | -
| Connection instance password
[...]
-| query | String | Yes | -
| Query statement
[...]
+| query | String | No | -
| Query statement. Required when neither `table_path` nor `table_list` is
configured.
[...]
| connection_check_timeout_sec | Int | No | 30
| The time in seconds to wait for the database operation used to validate
the connection to complete
[...]
| partition_column | String | No | -
| The column name for parallelism's partition, only support numeric type.
[...]
| partition_lower_bound | Long | No | -
| The partition_column min value for scan, if not set SeaTunnel will query
database get min value.
[...]
@@ -85,12 +86,15 @@ Read external data source data through JDBC.
| properties | Map | No | -
| Additional connection configuration parameters,when properties and URL
have the same parameters, the priority is determined by the <br/>specific
implementation of the driver. For example, in MySQL, properties take precedence
over the URL.
[...]
| use_regex | Boolean| No | false
| Control regular expression matching for table_path. When set to `true`,
the table_path will be treated as a regular expression pattern. When set to
`false` or not specified, the table_path will be treated as an exact path (no
regex matching).
[...]
| table_path | String | No | -
| The path to the full path of table, you can use this configuration
instead of `query`. <br/>example: <br/>"testdb.test_schema.table1"
[...]
-| table_list | Array | No | -
| The list of tables to be read, you can use this configuration instead of
`table_path` example: ```[{ table_path = "testdb.table1"}, {table_path =
"testdb.table2", query = "select * id, name from testdb.table2"}]```
[...]
+| table_list | Array | No | -
| The list of tables to be read, you can use this configuration instead of
`table_path` example: ```[{ table_path = "testdb.table1"}, {table_path =
"testdb.table2", query = "select id, name from testdb.table2"}]```
[...]
| where_condition | String | No | -
| Common row filter conditions for all tables/queries, must start with
`where`. for example `where id > 100`
[...]
| split.size | Int | No | 8096
| The split size (number of rows) of table, captured tables are split into
multiple splits when read of table.
[...]
| split.even-distribution.factor.lower-bound | Double | No | 0.05
| The lower bound of the chunk key distribution factor. This factor is used
to determine whether the table data is evenly distributed. If the distribution
factor is calculated to be greater than or equal to this lower bound (i.e.,
(MAX(id) - MIN(id) + 1) / row count), the table chunks would be optimized for
even distribution. Otherwise, if the distribution factor is less, the table
will be considered as un [...]
| split.even-distribution.factor.upper-bound | Double | No | 100
| The upper bound of the chunk key distribution factor. This factor is used
to determine whether the table data is evenly distributed. If the distribution
factor is calculated to be less than or equal to this upper bound (i.e.,
(MAX(id) - MIN(id) + 1) / row count), the table chunks would be optimized for
even distribution. Otherwise, if the distribution factor is greater, the table
will be considered as un [...]
-| split.sample-sharding.threshold | Int | No | 10000
| This configuration specifies the threshold of estimated shard count to
trigger the sample sharding strategy. When the distribution factor is outside
the bounds specified by `chunk-key.even-distribution.factor.upper-bound` and
`chunk-key.even-distribution.factor.lower-bound`, and the estimated shard count
(calculated as approximate row count / chunk size) exceeds this threshold, the
sample sharding strate [...]
+| split.sample-sharding.threshold | Int | No | 1000
| This configuration specifies the threshold of estimated shard count to
trigger the sample sharding strategy. When the distribution factor is outside
the bounds specified by `chunk-key.even-distribution.factor.upper-bound` and
`chunk-key.even-distribution.factor.lower-bound`, and the estimated shard count
(calculated as approximate row count / chunk size) exceeds this threshold, the
sample sharding strate [...]
+| split.allow-sampling | Boolean| No | true
| Whether to allow sampling-based sharding for uneven split keys. When set
to `false`, SeaTunnel falls back to iterative uneven chunk splitting.
[...]
+| use_select_count | Boolean| No | false
| Whether to use `select count(*)` to estimate table row count before
splitting.
[...]
+| skip_analyze | Boolean| No | false
| Whether to skip table row-count analysis before splitting.
[...]
| split.inverse-sampling.rate | Int | No | 1000
| The inverse of the sampling rate used in the sample sharding strategy.
For example, if this value is set to 1000, it means a 1/1000 sampling rate is
applied during the sampling process. This option provides flexibility in
controlling the granularity of the sampling, thus affecting the final number of
shards. It's especially useful when dealing with very large datasets where a
lower sampling rate is prefe [...]
| common-options | | No | -
| Source plugin common parameters, please refer to [Source Common
Options](../common-options/source-common-options.md) for details
[...]
@@ -269,4 +273,4 @@ sink {
## Changelog
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
diff --git a/docs/zh/connectors/sink/Mysql.md b/docs/zh/connectors/sink/Mysql.md
index 3a6d1e2e4e..75b7648d6c 100644
--- a/docs/zh/connectors/sink/Mysql.md
+++ b/docs/zh/connectors/sink/Mysql.md
@@ -33,15 +33,16 @@ import ChangeLog from '../changelog/connector-jdbc.md';
- [x] [精确一次](../../introduction/concepts/connector-v2-features.md)
- [x] [cdc](../../introduction/concepts/connector-v2-features.md)
-- [x] [x] [支持多表写入](../../introduction/concepts/connector-v2-features.md)
->使用“Xa事务”来确保“精确一次”。因此，数据库只支持“精确一次”，即
->支持“Xa事务”。您可以设置`is_exactly_once=true `来启用它。
+- [x] [支持多表写入](../../introduction/concepts/connector-v2-features.md)
+- [x] [定时刷新](../../introduction/concepts/connector-v2-features.md)
+
+> 使用 `XA 事务` 来保证 `精确一次`。因此仅支持支持 `XA 事务` 的数据库。可以通过设置 `is_exactly_once=true` 和
`max_retries=0` 来启用。
## 支持的数据源信息
| 数据源 | 支持的版本 | 驱动器 |
网址 | Maven下载链接
|
|-----|---------------------------------------------------------|--------------------------|---------------------------------------|---------------------------------------------------------------------|
-| Mysql | 不同的依赖版本具有不同的驱动程序类。 | com.mysql.cj.jdbc.Driver |
jdbc:mysql://localhost:3306:3306/test |
[下载](https://mvnrepository.com/artifact/mysql/mysql-connector-java) |
+| Mysql | 不同的依赖版本具有不同的驱动程序类。 | com.mysql.cj.jdbc.Driver |
jdbc:mysql://localhost:3306/test |
[下载](https://mvnrepository.com/artifact/mysql/mysql-connector-java) |
## 数据类型映射
@@ -68,7 +69,7 @@ import ChangeLog from '../changelog/connector-jdbc.md';
| 名称 | 类型 | 是否必填 | 默认值 |
描述
|
|------------------------------|---------|----------|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| url | String | 是 | -
| JDBC 连接的 URL。参见示例: <br/>`jdbc:mysql://localhost:3306:3306/test`。
|
+| url | String | 是 | -
| JDBC 连接的 URL。参见示例: <br/>`jdbc:mysql://localhost:3306/test`。
|
| driver | String | 是 | -
| 用于连接远程数据源的 JDBC 类名，<br/>如果使用 MySQL，值为 `com.mysql.cj.jdbc.Driver`。
|
| username | String | 否 | -
| 连接实例用户名。
|
| password | String | 否 | -
| 连接实例密码。
|
@@ -78,7 +79,8 @@ import ChangeLog from '../changelog/connector-jdbc.md';
| primary_keys | Array | 否 | -
| 此选项用于支持以下操作，例如 `insert`, `delete`, 和 `update` 当自动生成sql.
|
| connection_check_timeout_sec | Int | 否 | 30
| 等待用于验证连接的数据库操作完成的时间（秒）。
|
| max_retries | Int | 否 | 0
| 提交失败的重试次数（executeBatch）
|
-| batch_size | Int | 否 | 1000
|
对于批量写入，当缓冲记录的数量达到“batch_size”的数量或时间达到“checkpoint.interval”<br/>时，数据将被刷新到数据库中
|
+| batch_size | Int | 否 | 1000
| 对于批量写入，当缓冲记录数达到 `batch_size` 时，数据会刷新到数据库。如果 `batch_interval_ms` 大于
0，经过指定时间也会触发刷新。 |
+| batch_interval_ms | Long | 否 | 0
| 写入触发的定时刷新间隔，单位毫秒。`0` 表示关闭定时刷新；大于 0 时，写入器会在每条记录写入时检查间隔，达到间隔后同步刷新。
|
| is_exactly_once | Boolean | 否 | false
| 是否启用精确一次语义，这将使用Xa事务。如果启用，则需要<br/>设置`xa_data_source_class_name`。
|
| generate_sink_sql | Boolean | 否 | false
| 根据要写入的数据库表生成sql语句
|
| xa_data_source_class_name | String | 否 | -
|
数据库Driver的xa数据源类名，例如mysql是`com.mysql.cj.jdbc。MysqlXADataSource，和<br/>请参阅附录了解其他数据源
|
@@ -92,6 +94,7 @@ import ChangeLog from '../changelog/connector-jdbc.md';
| data_save_mode | Enum | 否 | APPEND_DATA
| 在启动同步任务之前，对目标端的现有数据选择不同的处理方案。
|
| custom_sql | String | 否 | -
|
当data_save_mode选择CUSTOM_PROCESSING时，您应该填写CUSTOM_SQL参数。此参数通常填充可以执行的SQL。SQL将在同步任务之前执行。
|
| enable_upsert | Boolean | 否 | true
| 通过primary_keys存在启用upstart，如果任务只有“插入”，将此参数设置为“false”可以加快数据导入
|
+| multi_table_sink_replica | Int | 否 | 1
| 多表写入时使用的 Sink Writer 副本数量。
|
### 提示
diff --git a/docs/zh/connectors/sink/Oracle.md
b/docs/zh/connectors/sink/Oracle.md
index 20b054713c..f515866202 100644
--- a/docs/zh/connectors/sink/Oracle.md
+++ b/docs/zh/connectors/sink/Oracle.md
@@ -29,10 +29,10 @@ import ChangeLog from '../changelog/connector-jdbc.md';
- [x] [精确一次](../../introduction/concepts/connector-v2-features.md)
- [x] [cdc](../../introduction/concepts/connector-v2-features.md)
+- [x] [支持多表写入](../../introduction/concepts/connector-v2-features.md)
+- [x] [定时刷新](../../introduction/concepts/connector-v2-features.md)
->使用“Xa事务”来确保“精确一次”。因此，数据库只支持“精确一次”，即
->支持“Xa事务”。您可以设置`is_exactly_once=true `来启用它。
-- [ ] [timer flush](../../introduction/concepts/connector-v2-features.md)
+> 使用 `XA 事务` 来保证 `精确一次`。因此仅支持支持 `XA 事务` 的数据库。可以通过设置 `is_exactly_once=true` 和
`max_retries=0` 来启用。
## 支持的数据源信息
@@ -77,8 +77,8 @@ import ChangeLog from '../changelog/connector-jdbc.md';
| primary_keys | Array | 否 | -
| 此选项用于支持以下操作，例如 `insert`, `delete`, 和 `update` 当自动生成sql.
|
| connection_check_timeout_sec | Int | 否 | 30
| 等待用于验证连接的数据库操作完成的时间（秒）。
|
| max_retries | Int | 否 | 0
| 提交失败的重试次数（executeBatch）
|
-| batch_size | Int | 否 | 1000
|
对于批量写入，当缓冲记录的数量达到“batch_size”的数量或时间达到“checkpoint.interval”<br/>时，数据将被刷新到数据库中。
|
-| batch_interval_ms | Int | 否 | 1000
| 对于批写入，当缓冲区的数量达到“batch_size”的数量或时间达到“batch-interval_ms”时，数据将被刷新到数据库中。
|
+| batch_size | Int | 否 | 1000
| 对于批量写入，当缓冲记录数达到 `batch_size` 时，数据会刷新到数据库。如果 `batch_interval_ms` 大于
0，经过指定时间也会触发刷新。
|
+| batch_interval_ms | Long | 否 | 0
| 写入触发的定时刷新间隔，单位毫秒。`0` 表示关闭定时刷新；大于 0 时，写入器会在每条记录写入时检查间隔，达到间隔后同步刷新。
|
| is_exactly_once | Boolean | 否 | false
| 是否启用精确一次语义，这将使用Xa事务。如果启用，则需要<br/>设置`xa_data_source_class_name`。
|
| generate_sink_sql | Boolean | 否 | false
| 根据要写入的数据库表生成sql语句
|
| xa_data_source_class_name | String | 否 | -
|
数据库Driver的xa数据源类名，例如Oracle，是`Oracle.jdbc.xa.client。OracleXADataSource和<br/>请参阅附录了解其他数据源
|
@@ -91,6 +91,7 @@ import ChangeLog from '../changelog/connector-jdbc.md';
| data_save_mode | Enum | 否 | APPEND_DATA
| 在启动同步任务之前，对目标端的现有数据选择不同的处理方案。
|
| custom_sql | String | 否 | -
|
当data_save_mode选择CUSTOM_PROCESSING时，您应该填写CUSTOM_SQL参数。此参数通常填充可以执行的SQL。SQL将在同步任务之前执行。
|
| enable_upsert | Boolean | 否 | true
| 通过primary_keys存在启用upstart，如果任务只有“插入”，将此参数设置为“false”可以加快数据导入
|
+| multi_table_sink_replica | Int | 否 | 1
| 多表写入时使用的 Sink Writer 副本数量。
|
### 提示
@@ -209,4 +210,4 @@ sink {
## 变更日志
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
diff --git a/docs/zh/connectors/sink/SqlServer.md
b/docs/zh/connectors/sink/SqlServer.md
index 45b119b05c..e3cda0c70b 100644
--- a/docs/zh/connectors/sink/SqlServer.md
+++ b/docs/zh/connectors/sink/SqlServer.md
@@ -32,9 +32,10 @@ import ChangeLog from '../changelog/connector-jdbc.md';
- [x] [精确一次](../../introduction/concepts/connector-v2-features.md)
- [x] [cdc](../../introduction/concepts/connector-v2-features.md)
+- [x] [支持多表写入](../../introduction/concepts/connector-v2-features.md)
+- [x] [定时刷新](../../introduction/concepts/connector-v2-features.md)
-> 使用 `Xa 事务` 来保证 `精确一次`。因此仅支持支持 `Xa 事务` 的数据库。可以通过设置 `is_exactly_once=true` 来启用。
-- [ ] [timer flush](../../introduction/concepts/connector-v2-features.md)
+> 使用 `XA 事务` 来保证 `精确一次`。因此仅支持支持 `XA 事务` 的数据库。可以通过设置 `is_exactly_once=true` 和
`max_retries=0` 来启用。
## 支持的数据源信息
@@ -78,15 +79,21 @@ import ChangeLog from '../changelog/connector-jdbc.md';
| primary_keys | Array | 否 | - | 此选项用于在自动生成 SQL
时支持 `insert`、`delete` 和 `update` 等操作。
|
| connection_check_timeout_sec | Int | 否 | 30 |
用于验证连接完成的数据库操作的等待时间（秒）。
|
| max_retries | Int | 否 | 0 |
提交失败（executeBatch）的重试次数。
|
-| batch_size | Int | 否 | 1000 | 对于批量写入，当缓冲的记录数达到
`batch_size` 或时间达到 `checkpoint.interval` 时，数据将被刷新到数据库中。
|
+| batch_size | Int | 否 | 1000 | 对于批量写入，当缓冲记录数达到
`batch_size` 时，数据会刷新到数据库。如果 `batch_interval_ms` 大于 0，经过指定时间也会触发刷新。
|
+| batch_interval_ms | Long | 否 | 0 |
写入触发的定时刷新间隔，单位毫秒。`0` 表示关闭定时刷新；大于 0 时，写入器会在每条记录写入时检查间隔，达到间隔后同步刷新。
|
| is_exactly_once | Boolean | 否 | false | 是否启用精确一次语义，将使用
Xa 事务。如果启用，需要设置 `xa_data_source_class_name`。
|
| generate_sink_sql | Boolean | 否 | false | 根据要写入的数据库表生成
SQL 语句。
|
| xa_data_source_class_name | String | 否 | - | 数据库驱动的 XA
数据源类名，例如 SQL Server 为
`com.microsoft.sqlserver.jdbc.SQLServerXADataSource`，其他数据源请参考附录。
|
| max_commit_attempts | Int | 否 | 3 | 事务提交失败的重试次数。
|
| transaction_timeout_sec | Int | 否 | -1 | 事务打开后的超时时间，默认为
-1（永不超时）。注意：设置超时可能会影响精确一次语义。
|
| auto_commit | Boolean | 否 | true | 默认启用自动事务提交。
|
+| properties | Map | 否 | - | 额外的 JDBC
连接参数。当 `properties` 和 URL 中存在相同参数时，优先级由 SQL Server JDBC 驱动决定。
|
| common-options | | 否 | - | 接收器插件通用参数，详情请参考
[Sink Common Options](../common-options/sink-common-options.md)。
|
+| schema_save_mode | Enum | 否 |
CREATE_SCHEMA_WHEN_NOT_EXIST | 同步任务启动前，控制目标表结构的处理方式。
|
+| data_save_mode | Enum | 否 | APPEND_DATA |
同步任务启动前，控制目标表已有数据的处理方式。
|
+| custom_sql | String | 否 | - | 当
`data_save_mode` 为 `CUSTOM_PROCESSING` 时，填写同步任务启动前需要执行的 SQL。
|
| enable_upsert | Boolean | 否 | true | 通过主键存在启用
upsert。如果任务中没有键重复数据，将此参数设置为 `false` 可以加快数据导入速度。
|
+| multi_table_sink_replica | Int | 否 | 1 | 多表写入时使用的 Sink
Writer 副本数量。
|
## 提示
@@ -168,6 +175,7 @@ Jdbc {
url = "jdbc:sqlserver://localhost:1433;databaseName=column_type_test"
username = SA
password = "Y.sa123456"
+ max_retries = 0
query = "insert into full_types_jdbc_sink( id, val_char, val_varchar,
val_text, val_nchar, val_nvarchar, val_ntext, val_decimal, val_numeric,
val_float, val_real, val_smallmoney, val_money, val_bit, val_tinyint,
val_smallint, val_int, val_bigint, val_date, val_time, val_datetime2,
val_datetime, val_smalldatetime ) values( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ? )"
is_exactly_once = "true"
xa_data_source_class_name =
"com.microsoft.sqlserver.jdbc.SQLServerXADataSource"
@@ -179,4 +187,4 @@ Jdbc {
## 变更日志
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
diff --git a/docs/zh/connectors/source/Mysql.md
b/docs/zh/connectors/source/Mysql.md
index d5a683715b..fa416247b3 100644
--- a/docs/zh/connectors/source/Mysql.md
+++ b/docs/zh/connectors/source/Mysql.md
@@ -44,7 +44,7 @@ import ChangeLog from '../changelog/connector-jdbc.md';
| 数据源 | 支持的版本 | 驱动器 |
网址 | Maven下载链接
|
|-----|---------------------------------------------------------|--------------------------|---------------------------------------|---------------------------------------------------------------------|
-| Mysql | 不同的依赖版本具有不同的驱动程序类。 | com.mysql.cj.jdbc.Driver |
jdbc:mysql://localhost:3306:3306/test |
[下载](https://mvnrepository.com/artifact/mysql/mysql-connector-java) |
+| Mysql | 不同的依赖版本具有不同的驱动程序类。 | com.mysql.cj.jdbc.Driver |
jdbc:mysql://localhost:3306/test |
[下载](https://mvnrepository.com/artifact/mysql/mysql-connector-java) |
## 数据类型映射
@@ -71,11 +71,11 @@ import ChangeLog from '../changelog/connector-jdbc.md';
| 名称 | 类型 | 是否必填 | 默认值 | 描述
|
|--------------------------------------------|------------|------|-------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| url | String | 是 | - |
JDBC 连接的 URL。参见示例: <br/>`jdbc:mysql://localhost:3306:3306/test`。
|
+| url | String | 是 | - |
JDBC 连接的 URL。参见示例: <br/>`jdbc:mysql://localhost:3306/test`。
|
| driver | String | 是 | - |
用于连接远程数据源的 JDBC 类名，<br/>如果使用 MySQL，值为 `com.mysql.cj.jdbc.Driver`。
|
| username | String | 否 | - |
连接实例用户名。
|
| password | String | 否 | - |
连接实例密码。
|
-| query | String | 是 | - |
查询语句。
|
+| query | String | 否 | - |
查询语句。当未配置 `table_path` 和 `table_list` 时必填。
|
| connection_check_timeout_sec | Int | 否 | 30 |
验证数据库连接所使用的操作完成的等待时间（秒）。
|
| partition_column | String | 否 | - |
用于并行度分区的列名，仅支持数字类型，仅支持数字类型的主键，并且只能配置一列。
|
| partition_lower_bound | BigDecimal | 否 | - | 扫描时
`partition_column` 的最小值，如果未设置，`SeaTunnel` 将查询数据库以获取最小值。
|
@@ -85,12 +85,15 @@ import ChangeLog from '../changelog/connector-jdbc.md';
| properties | Map | 否 | - |
额外的连接配置参数，当属性和 URL 中有相同的参数时，优先级由驱动程序的具体实现决定。<br/>例如，在 MySQL 中，属性优先于 URL。
|
| use_regex | Boolean | 否 | false |
控制表路径的正则表达式匹配。当设置为true时，table_path 将被视为正则表达式模式。当设置为false或未指定时，table_path
将被视为精确路径（不进行正则匹配）。
|
| table_path | String | 否 | - |
表的完整路径，您可以使用此配置代替 `query`。<br/>示例：<br/>"testdb.table1"
|
-| table_list | Array | 否 | - |
要读取的表的列表，您可以使用此配置代替 `table_path`，示例如下： ```[{ table_path = "testdb.table1"},
{table_path = "testdb.table2", query = "select * id, name from
testdb.table2"}]``` |
+| table_list | Array | 否 | - |
要读取的表的列表，您可以使用此配置代替 `table_path`，示例如下： ```[{ table_path = "testdb.table1"},
{table_path = "testdb.table2", query = "select id, name from
testdb.table2"}]``` |
| where_condition | String | 否 | - |
所有表/查询的通用行过滤条件，必须以 `where` 开头。例如 `where id > 100`。
|
| split.size | Int | 否 | 8096 |
表的分割大小（行数），当读取表时，捕获的表会被分割成多个分片。
|
| split.even-distribution.factor.lower-bound | Double | 否 | 0.05 |
分片键分布因子的下限。该因子用于判断表数据的分布是否均匀。如果计算得到的分布因子大于或等于该下限（即，(MAX(id) - MIN(id) + 1) /
行数），则会对表的分片进行优化，以确保数据的均匀分布。反之，如果分布因子较低，则表数据将被视为分布不均匀。如果估算的分片数量超过
`sample-sharding.threshold` 所指定的值，则会采用基于采样的分片策略。默认值为 0.05。 |
| split.even-distribution.factor.upper-bound | Double | 否 | 100 |
分片键分布因子的上限。该因子用于判断表数据的分布是否均匀。如果计算得到的分布因子小于或等于该上限（即，(MAX(id) - MIN(id) + 1) /
行数），则会对表的分片进行优化，以确保数据的均匀分布。反之，如果分布因子较大，则表数据将被视为分布不均匀，并且如果估算的分片数量超过
`sample-sharding.threshold` 所指定的值，则会采用基于采样的分片策略。默认值为 100.0。 |
-| split.sample-sharding.threshold | Int | 否 | 10000 |
此配置指定了触发样本分片策略的估算分片数阈值。当分布因子超出由
`chunk-key.even-distribution.factor.upper-bound` 和
`chunk-key.even-distribution.factor.lower-bound` 指定的范围，并且估算的分片数量（计算方法为大致行数 /
分片大小）超过此阈值时，将使用样本分片策略。此配置有助于更高效地处理大型数据集。默认值为 1000 个分片。 |
+| split.sample-sharding.threshold | Int | 否 | 1000 |
此配置指定了触发样本分片策略的估算分片数阈值。当分布因子超出由
`chunk-key.even-distribution.factor.upper-bound` 和
`chunk-key.even-distribution.factor.lower-bound` 指定的范围，并且估算的分片数量（计算方法为大致行数 /
分片大小）超过此阈值时，将使用样本分片策略。此配置有助于更高效地处理大型数据集。默认值为 1000 个分片。 |
+| split.allow-sampling | Boolean | 否 | true |
是否允许对分布不均匀的分片键使用采样分片策略。设置为 `false` 时，SeaTunnel 会退回到迭代式不均匀分片。
|
+| use_select_count | Boolean | 否 | false |
是否使用 `select count(*)` 在分片前估算表行数。
|
+| skip_analyze | Boolean | 否 | false |
是否跳过分片前的表行数分析。
|
| split.inverse-sampling.rate | Int | 否 | 1000 |
样本分片策略中使用的采样率的倒数。例如，如果该值设置为 1000，则表示在采样过程中应用 1/1000
的采样率。此选项提供了灵活性，可以控制采样的粒度，从而影响最终的分片数量。特别适用于处理非常大的数据集，在这种情况下通常会选择较低的采样率。默认值为
1000。
|
| int_type_narrowing | Boolean | 否 | true |
Int类型收窄，如果为 true，则 tinyint(1) 类型将被收窄为 boolean 类型（如果没有精度损失）。目前仅支持 MySQL。
|
| common-options | | 否 | - |
源插件的常见参数，请参阅 [源常见参数](../common-options/source-common-options.md) 了解详细信息。
|
@@ -338,4 +341,4 @@ sink {
## 变更日志
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
diff --git a/docs/zh/connectors/source/Oracle.md
b/docs/zh/connectors/source/Oracle.md
index 5af3ce9984..530d81da69 100644
--- a/docs/zh/connectors/source/Oracle.md
+++ b/docs/zh/connectors/source/Oracle.md
@@ -22,6 +22,7 @@ import ChangeLog from '../changelog/connector-jdbc.md';
- [x] [列投影](../../introduction/concepts/connector-v2-features.md)
- [x] [并行性](../../introduction/concepts/connector-v2-features.md)
- [x] [支持用户自定义split](../../introduction/concepts/connector-v2-features.md)
+- [x] [支持多表读取](../../introduction/concepts/connector-v2-features.md)
> 支持查询 SQL 并可以实现投影效果。
@@ -68,7 +69,7 @@ import ChangeLog from '../changelog/connector-jdbc.md';
| driver | String | 是 | - | 用于连接到远程数据源的 jdbc 类名，如果您使用 Oracle，值为
`oracle.jdbc.OracleDriver`。 |
| username | String | 否 | - | 连接实例用户名 |
| password | String | 否 | - | 连接实例密码 |
-| query | String | 是 | - | 查询语句 |
+| query | String | 否 | - | 查询语句。当未配置 `table_path` 和 `table_list` 时必填。 |
| connection_check_timeout_sec | Int | 否 | 30 | 等待用于验证连接的数据库操作完成的时间（秒） |
| partition_column | String | 否 | - | 用于并行性分割的列名，仅支持数值类型，仅支持数值类型主键，只能配置一列。 |
| partition_lower_bound | BigDecimal | 否 | - | partition_column
的最小值用于扫描，如果未设置，SeaTunnel 将查询数据库获取最小值。 |
@@ -78,12 +79,19 @@ import ChangeLog from '../changelog/connector-jdbc.md';
| properties | Map | 否 | - | 其他连接配置参数，当 properties 和 URL
具有相同参数时，优先级由驱动程序的具体实现确定。例如，在 Oracle 中，properties 优先于 URL。 |
| use_regex | Boolean | 否 | false | 控制 table_path 的正则表达式匹配。设置为 `true`
时，table_path 将被视为正则表达式模式。设置为 `false` 或未指定时，table_path 将被视为精确路径（无正则表达式匹配）。 |
| table_path | String | 否 | - | 表的完整路径，您可以使用此配置代替
`query`。<br/>示例：<br/>"test_schema.table1" |
-| table_list | Array | 否 | - | 要读取的表列表，您可以使用此配置代替 `table_path`。 |
+| table_list | Array | 否 | - | 要读取的表列表，您可以使用此配置代替 `table_path`。示例：```[{
table_path = "SCHEMA1.TABLE1"}, {table_path = "SCHEMA1.TABLE2", query = "select
ID, NAME from SCHEMA1.TABLE2"}]``` |
| where_condition | String | 否 | - | 所有表/查询的通用行过滤条件，必须以 `where` 开头。 |
| split.size | Int | 否 | 8096 | 一个分割中有多少行。 |
+| split.even-distribution.factor.lower-bound | Double | 否 | 0.05 |
分片键分布因子的下限，用于判断数据是否均匀分布。 |
+| split.even-distribution.factor.upper-bound | Double | 否 | 100 |
分片键分布因子的上限，用于判断数据是否均匀分布。 |
+| split.sample-sharding.threshold | Int | 否 | 1000 | 触发采样分片策略的估算分片数阈值。 |
+| split.inverse-sampling.rate | Int | 否 | 1000 | 采样分片策略使用的采样率倒数，例如 `1000` 表示
1/1000 采样。 |
+| split.allow-sampling | Boolean | 否 | true | 是否允许对分布不均匀的分片键使用采样分片策略。设置为
`false` 时，SeaTunnel 会退回到迭代式不均匀分片。 |
+| use_select_count | Boolean | 否 | false | 是否使用 `select count(*)` 在分片前估算表行数。 |
+| skip_analyze | Boolean | 否 | false | 是否跳过分片前的表行数分析。 |
+| decimal_type_narrowing | Boolean | 否 | true | Decimal 类型收窄。为 `true`
时，如果没有精度损失，会把 Oracle Decimal 类型收窄为 Int 或 Long。 |
| common-options | | 否 | - | 源插件通用参数，请参考
[源通用选项](../common-options/source-common-options.md) 详见。 |
## 变更日志
<ChangeLog />
-
diff --git a/docs/zh/connectors/source/SqlServer.md
b/docs/zh/connectors/source/SqlServer.md
index ce82fb10e2..e7c781b95d 100644
--- a/docs/zh/connectors/source/SqlServer.md
+++ b/docs/zh/connectors/source/SqlServer.md
@@ -32,6 +32,7 @@ import ChangeLog from '../changelog/connector-jdbc.md';
- [x] [列投影](../../introduction/concepts/connector-v2-features.md)
- [x] [并行度](../../introduction/concepts/connector-v2-features.md)
- [x] [支持用户定义分割](../../introduction/concepts/connector-v2-features.md)
+- [x] [支持多表读取](../../introduction/concepts/connector-v2-features.md)
> 支持查询 SQL 并可以实现投影效果。
@@ -75,7 +76,7 @@ import ChangeLog from '../changelog/connector-jdbc.md';
| driver | String | 是 | -
| 用于连接远程数据源的 jdbc 类名，<br/>如果使用 SQLserver，值为
`com.microsoft.sqlserver.jdbc.SQLServerDriver`。
|
| username | String | 否 | -
| 连接实例的用户名
|
| password | String | 否 | -
| 连接实例的密码
|
-| query | String | 是 | -
| 查询语句
|
+| query | String | 否 | -
| 查询语句。当未配置 `table_path` 和 `table_list` 时必填。
|
| connection_check_timeout_sec | Int | 否 | 30
| 等待用于验证连接的数据库操作完成的时间（秒）
|
| partition_column | String | 否 | -
| 用于并行度分区的列名，仅支持数值类型。
|
| partition_lower_bound | Long | 否 | -
| partition_column 扫描的最小值，如果未设置，SeaTunnel 将查询数据库获取最小值。
|
@@ -85,12 +86,15 @@ import ChangeLog from '../changelog/connector-jdbc.md';
| properties | Map | 否 | -
| 额外的连接配置参数，当 properties 和 URL 具有相同参数时，优先级由<br/>驱动的具体实现决定。例如，在 MySQL
中，properties 优先于 URL。
|
| use_regex | Boolean | 否 | false
| 控制 table_path 的正则表达式匹配。当设置为 `true` 时，table_path 将被视为正则表达式模式。当设置为 `false`
或未指定时，table_path 将被视为精确路径（不进行正则匹配）。
|
| table_path | String | 否 | -
| 表的完整路径，您可以使用此配置代替 `query`。<br/>示例：<br/>"testdb.test_schema.table1"
|
-| table_list | Array | 否 | -
| 要读取的表列表，您可以使用此配置代替 `table_path`。示例：```[{ table_path = "testdb.table1"},
{table_path = "testdb.table2", query = "select * id, name from
testdb.table2"}]```
|
+| table_list | Array | 否 | -
| 要读取的表列表，您可以使用此配置代替 `table_path`。示例：```[{ table_path = "testdb.table1"},
{table_path = "testdb.table2", query = "select id, name from
testdb.table2"}]```
|
| where_condition | String | 否 | -
| 所有表/查询的通用行过滤条件，必须以 `where` 开头。例如 `where id > 100`
|
| split.size | Int | 否 | 8096
| 表的分割大小（行数），读取表时，捕获的表会被分割为多个分割。
|
| split.even-distribution.factor.lower-bound | Double | 否 | 0.05
| 分块键分布因子的下界。此因子用于确定表数据是否均匀分布。如果计算的分布因子大于或等于此下界（即，(MAX(id) - MIN(id) + 1) /
行数），表分块将被优化以实现均匀分布。否则，如果分布因子较小，如果估计的分片数超过 `sample-sharding.threshold`
指定的值，表将被视为不均匀分布并使用基于采样的分片策略。默认值为 0.05。 |
| split.even-distribution.factor.upper-bound | Double | 否 | 100
| 分块键分布因子的上界。此因子用于确定表数据是否均匀分布。如果计算的分布因子小于或等于此上界（即，(MAX(id) - MIN(id) + 1) /
行数），表分块将被优化以实现均匀分布。否则，如果分布因子较大，如果估计的分片数超过 `sample-sharding.threshold`
指定的值，表将被视为不均匀分布并使用基于采样的分片策略。默认值为 100.0。 |
-| split.sample-sharding.threshold | Int | 否 | 10000
| 此配置指定了触发采样分片策略的估计分片数阈值。当分布因子超出
`chunk-key.even-distribution.factor.upper-bound` 和
`chunk-key.even-distribution.factor.lower-bound` 指定的范围，并且估计的分片数（计算为近似行数 /
分块大小）超过此阈值时，将使用采样分片策略。这可以帮助更有效地处理大型数据集。默认值为 1000 分片。 |
+| split.sample-sharding.threshold | Int | 否 | 1000
| 此配置指定了触发采样分片策略的估计分片数阈值。当分布因子超出
`chunk-key.even-distribution.factor.upper-bound` 和
`chunk-key.even-distribution.factor.lower-bound` 指定的范围，并且估计的分片数（计算为近似行数 /
分块大小）超过此阈值时，将使用采样分片策略。这可以帮助更有效地处理大型数据集。默认值为 1000 分片。 |
+| split.allow-sampling | Boolean | 否 | true
| 是否允许对分布不均匀的分片键使用采样分片策略。设置为 `false` 时，SeaTunnel 会退回到迭代式不均匀分片。
|
+| use_select_count | Boolean | 否 | false
| 是否使用 `select count(*)` 在分片前估算表行数。
|
+| skip_analyze | Boolean | 否 | false
| 是否跳过分片前的表行数分析。
|
| split.inverse-sampling.rate | Int | 否 | 1000
| 采样分片策略中使用的采样率的倒数。例如，如果此值设置为 1000，则意味着在采样过程中应用 1/1000
的采样率。此选项提供了控制采样粒度的灵活性，从而影响最终的分片数量。对于非常大的数据集，首选较低的采样率时，此选项特别有用。默认值为 1000。
|
| common-options | | 否 | -
| 源插件通用参数，请参考 [源通用选项](../common-options/source-common-options.md) 获取详细信息
|
@@ -269,4 +273,4 @@ sink {
## 变更日志
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
