liujiayi771 opened a new pull request, #11074: URL: https://github.com/apache/incubator-gluten/pull/11074
## What changes are proposed in this pull request? Spark’s TimestampType is timezone-aware: it internally stores timestamps in UTC (by converting input values to UTC based on the session time zone or just read UTC timestamp from parquet file) and represents an absolute point in time. This semantics aligns with Substrait’s timestamp_tz type, which also denotes a timezone-aware timestamp that can be unambiguously mapped to a moment on the timeline. To maintain semantic consistency between Spark and Substrait, this PR maps Spark’s TimestampType to Substrait’s timestamp_tz. https://substrait.io/types/type_classes This approach is consistent with other projects—for example, Apache Iceberg also maps Spark’s TimestampType to TIMESTAMP WITH TIME ZONE and Spark’s TimestampNTZ to TIMESTAMP WITHOUT TIME ZONE. Note: For future support of Spark’s TimestampNTZType (timezone-naive timestamps), we should map it to Substrait’s timestamp type instead. ## How was this patch tested? The existing tests already cover this change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
