jinkachy commented on code in PR #9002:
URL: https://github.com/apache/seatunnel/pull/9002#discussion_r2007546492
##########
docs/en/connector-v2/source/Jdbc.md:
##########
@@ -68,6 +68,8 @@ supports query SQL and can achieve projection effect.
| split.sample-sharding.threshold | Int | No | 1000
| This configuration specifies the threshold of estimated shard count to
trigger the sample sharding strategy. When the distribution factor is outside
the bounds specified by `chunk-key.even-distribution.factor.upper-bound` and
`chunk-key.even-distribution.factor.lower-bound`, and the estimated shard count
(calculated as approximate row count / chunk size) exceeds this threshold, the
sample sharding strategy will be used. This can help to handle large datasets
more efficiently. The default value is 1000 shards.
|
| split.inverse-sampling.rate | Int | No | 1000
| The inverse of the sampling rate used in the sample sharding strategy.
For example, if this value is set to 1000, it means a 1/1000 sampling rate is
applied during the sampling process. This option provides flexibility in
controlling the granularity of the sampling, thus affecting the final number of
shards. It's especially useful when dealing with very large datasets where a
lower sampling rate is preferred. The default value is 1000.
|
| common-options | | No | -
| Source plugin common parameters, please refer to [Source Common
Options](../source-common-options.md) for details.
|
+| string_split_mode | String | No | -
| When set to "charsetBased", enables charset-based string splitting
algorithm. The algorithm assumes characters of partition_column are within
ASCII range 32-126, which covers most character-based splitting scenarios.
|
Review Comment:
> 1. which ones value I can set?
> 2. which database supported this feature? Dose all?
@Hisoka-X This feature provides two parameters:
1. string_split_mode=charset_based
2. string_split_mode_collate=xxx
The string_split_mode=charset_based is supported by all databases, while
string_split_mode_collate only needs to be specified when a collation different
from the database default is explicitly required. The specific values for
string_split_mode_collate depend on the database being used:
- MySQL supports: utf8_general_ci, utf8mb4_0900_ai_ci, etc.
- PostgreSQL supports: en_US.UTF-8, fr_FR.UTF-8, etc.
- SQL Server supports: Latin1_General_BIN, etc.
- Oracle supports: SPANISH, BINARY_CI, etc.
I have tested that MySQL, Oracle, PostgreSQL, GreenPlum, SQL Server,
ClickHouse, Vertica, and TDSQL all support specifying collations, so most
databases should support collation config. In most cases, only setting
string_split_mode=charset_based is sufficient, as users rarely need to specify
a collation different from the database default.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
