MaxGekk opened a new pull request, #56544: URL: https://github.com/apache/spark/pull/56544
### What changes were proposed in this pull request? This PR adds support for `CAST` between same-family nanosecond-precision timestamp types of different precision: - `TIMESTAMP_NTZ(p1)` <-> `TIMESTAMP_NTZ(p2)` - `TIMESTAMP_LTZ(q1)` <-> `TIMESTAMP_LTZ(q2)` where the precisions are in `[7, 9]`. Both `TimestampNTZNanosType` and `TimestampLTZNanosType` share the same physical value `TimestampNanosVal(epochMicros, nanosWithinMicro)` (with `nanosWithinMicro` in `[0, 999]`). A cross-precision cast only re-floors the sub-microsecond part of the value; `epochMicros` and the time zone are untouched: - **Widening** (target precision >= source precision): lossless; the value is unchanged. - **Narrowing** (target precision < source precision): floors `nanosWithinMicro` toward the past to the target precision step (drops the lowest `9 - precision` sub-microsecond digits), consistent with the existing nanos -> micros narrowing rule. The store-assignment / up-cast contract follows the established micros <-> nanos precedent (SPARK-57293): - Widening (lossless) is allowed as an ANSI store assignment but is not an up-cast. - Narrowing (lossy) is explicit-CAST only (rejected by both up-cast and store assignment). - Equal precision is the identity cast. Concretely, this adds the `canCast` / `canAnsiCast` / `canANSIStoreAssign` rules, the interpreted and codegen eval paths in `Cast`, and a `DateTimeUtils.truncateTimestampNanosToPrecision` helper reused by both paths. ### Why are the changes needed? The nanosecond-capable timestamp types `TIMESTAMP_NTZ(p)` and `TIMESTAMP_LTZ(p)` (p in `[7, 9]`) are gated behind `spark.sql.timestampNanosTypes.enabled`. Casts between these types and their microsecond counterparts (`TIMESTAMP_NTZ` / `TIMESTAMP`), as well as to/from `DATE` and `STRING`, are already supported. However, casting between two nanosecond timestamps of the same family but different precision was not allowed: such a cast was absent from `Cast.canCast` / `Cast.canAnsiCast` and failed type checking. ### Does this PR introduce _any_ user-facing change? Yes, but only behind the preview flag `spark.sql.timestampNanosTypes.enabled`. With the flag enabled, `CAST(... AS TIMESTAMP_NTZ(p))` / `CAST(... AS TIMESTAMP_LTZ(p))` from another nanosecond timestamp of the same family is now supported instead of failing type checking. ### How was this patch tested? Added unit tests in `CastSuiteBase` covering: - all NTZ/LTZ precision pairs (widening is lossless, narrowing floors the sub-microsecond digits), including pre-epoch (negative-epoch) values and nulls; - the store-assignment / up-cast contract for cross-precision pairs; - null-cast coverage for cross-precision pairs. Ran `catalyst/testOnly *CastSuite *CastWithAnsiOnSuite *CastWithAnsiOffSuite` (355 tests passed) and `scalastyle` on `sql-api` and `catalyst`. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Cursor -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
