doyong365 commented on PR #10724:
URL: https://github.com/apache/seatunnel/pull/10724#issuecomment-4476475841
@davidzollo Thank you for the thorough review! Here is a summary of how each
blocking point was addressed:
1. Clarify and verify the Iceberg upgrade path for existing tables
The IcebergTypeMapper already maps withZone() → TIMESTAMP_TZ and
withoutZone() → TIMESTAMP. To make this explicit and verifiable, two
upgrade-path regression tests were added to IcebergTypeMapperTest:
- upgradePath_legacyWithZoneColumnIsReadAsTimestampTz() — asserts that
existing Iceberg tables written by the old SeaTunnel (which incorrectly used
withZone() for TIMESTAMP) are now read back as TIMESTAMP_TZ
- upgradePath_newTimestampWrittenAsWithoutZoneIsReadAsTimestamp() —
asserts that new tables written with withoutZone() are read back as TIMESTAMP
The upgrade impact and migration guidance are documented in
incompatible-changes.md.
2. Unify the downgrade contract for TIMESTAMP_TZ
The contract is now clearly defined in two tiers and documented in
incompatible-changes.md:
- DB-typed sinks without native timezone support (Text, Doris, StarRocks,
Xugu): timezone offset is dropped and the wall-clock value is stored. Text
previously converted to UTC before dropping the offset, which was inconsistent;
it now uses toLocalDateTime() to match Doris and StarRocks.
- JSON-based messaging sinks (Kafka, Pulsar, RocketMQ, etc.): the full ISO
8601 offset is preserved (e.g., "2024-01-01T03:00:00+09:00"). Changing the JSON
default to wall-clock would be a large breaking change affecting all messaging
connectors, so the two-tier contract is intentional and explicitly documented.
3. Make the Xugu TIMESTAMP_TZ path honest
XuguJdbcRowConverter now has an explicit log-once WARN (via AtomicBoolean)
on the first write, with a strong inline comment explaining that this is a
known Xugu JDBC driver limitation (bug [E19138]) and not silent data loss. The
limitation is also documented in incompatible-changes.md.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
