wuhainan opened a new issue, #12884: URL: https://github.com/apache/seatunnel/issues/12884
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened ### Problem description When synchronizing MySQL to Doris with SeaTunnel 3.0.0, a MySQL `TIMESTAMP` column is written to Doris with an 8-hour difference, even though the MySQL-CDC source explicitly sets: ```hocon server-time-zone = "Asia/Shanghai" ``` For the same record (`id = 359144`), the observed values are: | Location | `updated_at` | |---|---| | MySQL, session time zone `+08:00` | `2026-06-26 00:08:32` | | MySQL, session time zone `+00:00` | `2026-06-25 16:08:32` | | Doris | `2026-06-25 16:08:32` | The value stored in Doris matches the UTC representation, whereas our existing pipeline expects the Asia/Shanghai wall-clock value. The same scenario did not show this difference with SeaTunnel 2.3.13. ### Actual environment and observations - Source column: MySQL `TIMESTAMP`. - Target column: Doris `DATETIME`. - Doris table: `UNIQUE KEY(id)`, merge-on-write enabled, no Sequence column. - Source startup mode: `initial`. - No checkpoint or savepoint was restored. - Doris save mode: `APPEND_DATA`. - Re-running the initial snapshot still produces the shifted value. - No exception is reported. The relevant source columns are: ```sql created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP ``` The MySQL time zone settings are: ```text @@global.time_zone = SYSTEM @@session.time_zone = SYSTEM @@system_time_zone = CST ``` To check the actual timestamp behavior independently of the `CST` abbreviation, we queried the same record using explicit session time zones: ```sql SET @original_time_zone = @@session.time_zone; SET SESSION time_zone = '+08:00'; SELECT id, created_at, updated_at FROM quote_data.t_contract WHERE id = 359144; ``` Result: ```text id created_at updated_at 359144 2026-06-24 09:07:04 2026-06-26 00:08:32 ``` Then: ```sql SET SESSION time_zone = '+00:00'; SELECT id, created_at, updated_at FROM quote_data.t_contract WHERE id = 359144; SET SESSION time_zone = @original_time_zone; ``` Result: ```text id created_at updated_at 359144 2026-06-24 09:07:04 2026-06-25 16:08:32 ``` As expected, the MySQL `DATETIME` value remains unchanged, while the `TIMESTAMP` value changes with the session time zone. The Doris `updated_at` value matches the second result. ### Source investigation Comparing the local `2.3.13` tag with the local `v3.0.0` source checkout shows a change in the conversion path. #### SeaTunnel 2.3.13 In `MySqlTypeConverter`, both MySQL `DATETIME` and `TIMESTAMP` map to `LOCAL_DATE_TIME_TYPE`: ```java case MYSQL_DATETIME: case MYSQL_TIMESTAMP: builder.dataType(LocalTimeType.LOCAL_DATE_TIME_TYPE); ``` For a Debezium timestamp string, the conversion path uses the configured server time zone: ```java Instant instant = Instant.parse(str); return LocalDateTime.ofInstant(instant, serverTimeZone); ``` With `server-time-zone = "Asia/Shanghai"`, the conversion is: ```text 2026-06-25T16:08:32Z -> 2026-06-26T00:08:32 -> Doris DATETIME: 2026-06-26 00:08:32 ``` #### Examined SeaTunnel 3.0.0 source MySQL `TIMESTAMP` now maps to `OFFSET_DATE_TIME_TYPE` / `TIMESTAMP_TZ`: ```java case MYSQL_TIMESTAMP: builder.dataType(LocalTimeType.OFFSET_DATE_TIME_TYPE); ``` The CDC conversion in `SeaTunnelRowDebeziumDeserializationConverters` normalizes the timestamp to UTC: ```java OffsetDateTime.parse(str) .withOffsetSameInstant(ZoneOffset.UTC); ``` Normalizing to UTC preserves the instant and is not itself the problem. However, the Doris `SeaTunnelRowSerializer` creates the JSON serializer with: ```java new JsonSerializationSchema(this.seaTunnelRowType, true); ``` This enables the following `TIMESTAMP_TZ` serialization branch in `RowToJsonConverters`: ```java ISO_LOCAL_DATE_TIME.format( ((OffsetDateTime) value).toLocalDateTime() ) ``` `toLocalDateTime()` removes the offset without converting the value to the intended target time zone. Consequently, the conversion becomes: ```text 2026-06-25T16:08:32Z -> "2026-06-25T16:08:32" -> Doris DATETIME: 2026-06-25 16:08:32 ``` The JSON sent to Doris no longer contains an offset, so the original time zone information is already lost before ingestion. The relevant type-mapping and serialization changes appear in: https://github.com/apache/seatunnel/pull/10724 ### Local source-level verification The relevant CDC conversion and Doris serialization sources were compiled and exercised in isolation. The observed output was: ```text CDC value: 2026-06-25T16:08:32Z Doris JSON before explicit time zone conversion: {"id":359144,"created_at":"2026-06-24T09:07:04","updated_at":"2026-06-25T16:08:32"} Doris JSON after explicit time zone conversion: {"id":359144,"created_at":"2026-06-24T09:07:04","updated_at":"2026-06-26T00:08:32"} ``` The SQL expression used for the conversion was: ```sql updated_at AT TIME ZONE 'Asia/Shanghai' AS updated_at ``` The check confirmed that this conversion preserves the instant and leaves the `created_at` value unchanged. Verification scope: the source checkout is from a company-maintained fork, locally tagged `v3.0.0` at commit `5056e3e1766690a2fa01934b9349c645d8086b93`. This source-level check is not an end-to-end reproduction using a pristine official 3.0.0 distribution. The exact deployed connector JARs still need to be matched against the examined sources. ### Suggested minimal reproduction The following reduced reproduction is proposed for independent verification. The production-table observations above are confirmed; this reduced end-to-end case has not yet been executed. 1. Create a MySQL source table and insert a known timestamp using an explicit session time zone: ```sql CREATE TABLE cdc_timezone_test ( id BIGINT NOT NULL PRIMARY KEY, created_at DATETIME, updated_at TIMESTAMP ); SET SESSION time_zone = '+08:00'; INSERT INTO cdc_timezone_test VALUES ( 1, '2026-06-24 09:07:04', '2026-06-26 00:08:32' ); ``` 2. Create the corresponding Doris target table: ```sql CREATE TABLE cdc_timezone_test ( id BIGINT, created_at DATETIME, updated_at DATETIME ) UNIQUE KEY(id) DISTRIBUTED BY HASH(id) BUCKETS 1 PROPERTIES ( "replication_num" = "1", "enable_unique_key_merge_on_write" = "true" ); ``` 3. Run a MySQL-CDC → Doris job using the configuration in the “SeaTunnel Config” field, with: ```hocon startup.mode = "initial" server-time-zone = "Asia/Shanghai" ``` Start a new job without restoring a checkpoint or savepoint. 4. After the snapshot data has been committed, query Doris: ```sql SELECT id, created_at, updated_at FROM cdc_timezone_test WHERE id = 1; ``` The expected value for compatibility with the existing Asia/Shanghai pipeline is: ```text updated_at = 2026-06-26 00:08:32 ``` The suspected regression would produce: ```text updated_at = 2026-06-25 16:08:32 ``` 5. Repeat with SeaTunnel 2.3.13 using equivalent configuration and a clean target table to compare the results. ### Expected behavior / clarification requested Could maintainers confirm whether this change in MySQL-CDC → Doris timestamp behavior is intentional? For a Doris `DATETIME` target, the pipeline needs a way to select the target time zone before dropping the offset. Possible approaches include an explicit sink-side time zone conversion or a documented SQL Transform requirement. If the intended behavior is to store the UTC wall-clock value after the `TIMESTAMP_TZ` mapping change, please document this migration requirement for existing MySQL-CDC → Doris jobs. Keeping the existing `server-time-zone = "Asia/Shanghai"` configuration currently does not preserve the previous output in the examined source path. ### SeaTunnel Version 3.0.0 ### SeaTunnel Config ```conf env { parallelism = 1 job.mode = "STREAMING" job.name = "mysql2doris_quote_data" checkpoint.interval = 30000 checkpoint.timeout = 600000 checkpoint.mode = "EXACTLY_ONCE" restart-strategy = "failure-rate" restart-strategy.failure-rate.max-failures-per-interval = 10 restart-strategy.failure-rate.failure-rate-interval = "300 s" restart-strategy.failure-rate.delay = "10 s" jobmanager.memory.process.size = "1024m" taskmanager.memory.process.size = "2048m" taskmanager.numberOfTaskSlots = 1 } source { MySQL-CDC { hostname = "<MYSQL_HOST>" port = 3307 url = "jdbc:mysql://<MYSQL_HOST>:3307/" username = "<MYSQL_USERNAME>" password = "<MYSQL_PASSWORD>" table-names-config = [ { table = "quote_data.t_contract" primaryKeys = ["id"] }, { table = "quote_data.t_contract_snapshot" primaryKeys = ["id"] }, { table = "quote_data.t_financial" primaryKeys = ["id"] }, { table = "quote_data.t_package" primaryKeys = ["id"] }, { table = "quote_data.t_package_snapshot" primaryKeys = ["id"] } ] database-names = ["quote_data"] table-names = [ "quote_data.t_contract", "quote_data.t_contract_snapshot", "quote_data.t_financial", "quote_data.t_package", "quote_data.t_package_snapshot" ] server-id = 5493 startup.mode = "initial" server-time-zone = "Asia/Shanghai" schema-changes.enabled = true plugin_output = "src_tidb_0" } } sink { Doris { fenodes = "<DORIS_FE_HOST>:8030" username = "<DORIS_USERNAME>" password = "<DORIS_PASSWORD>" database = "quote_data_test" table = "${table_name}" plugin_input = ["src_tidb_0"] batch_interval_ms = 5000 data_save_mode = "APPEND_DATA" schema_save_mode = "CREATE_SCHEMA_WHEN_NOT_EXIST" sink.enable-2pc = false sink.enable-delete = true sink.label-prefix = "datat-doris-3-v4_tidb_0" doris.config { format = "json" read_json_by_line = true strict_mode = false } } } ``` ### Running Command ```shell The job was submitted through the platform using the SeaTunnel configuration provided above. Startup mode: initial Checkpoint/savepoint restoration: disabled ``` ### Error Exception ```log No exception or stack trace is reported. The job continues running, but the synchronized TIMESTAMP value is 8 hours behind the expected Asia/Shanghai wall-clock value. MySQL (+08:00): 2026-06-26 00:08:32 Doris: 2026-06-25 16:08:32 ``` ### Zeta or Flink or Spark Version Apache Flink 1.20 ### Java or Scala Version jdk-17.0.18 ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
