ziyanTOP opened a new issue, #666:
URL: https://github.com/apache/doris-flink-connector/issues/666
## Problem Summary
When using `mongodb-sync-database` to sync MongoDB data to Doris, BSON Date
fields are always converted using a hardcoded timezone (`Asia/Shanghai` in
`MongoDateConverter`). There is no way for users to configure the target
timezone, and even if one were added, the timezone configuration would not take
effect on TaskManagers due to the use of a static variable that resets after
serializer deserialization.
## Root Causes
1. **Hardcoded timezone**: `MongoDateConverter` hardcodes `Asia/Shanghai`,
making it impossible for users in other regions to control conversion.
2. **Debezium serializes BSON Date as ISO 8601 strings**: The Change Stream
JSON represents dates as strings like `\"2024-06-04T08:03:37Z\"` instead of
`{\"$date\": ...}` objects. The existing `formatSpecialFieldData()` only
handles the object form, so these strings bypass conversion entirely.
3. **Static variable not serializable to TaskManager**:
`MongoDateConverter.timeZone` is a static variable set on the Flink client
side. After serializer serialization to TaskManagers, it resets to default
`\"UTC\"`.
## Proposed Changes
1. Add new `date.timezone` ConfigOption to `MongoDBDatabaseSync` (default
`UTC`).
2. Add ISO 8601 string parsing support in
`MongoJsonDebeziumSchemaChange.formatSpecialFieldData()`.
3. Pass timezone through `JsonDebeziumChangeContext` (Serializable) to
ensure it reaches TaskManagers.
4. Remove hardcoded `Asia/Shanghai` from `MongoDateConverter`.
## Checklist
1. Does it affect the original behavior: Yes (adds timezone conversion)
2. Has unit tests been added: No
3. Has document been added or modified: No Need
4. Does it need to update dependencies: No
5. Are there any changes that cannot be rolled back: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
