MaxGekk opened a new pull request, #56375: URL: https://github.com/apache/spark/pull/56375
### What changes were proposed in this pull request? This PR adds explicit `CAST` support in both directions between `DATE` and the nanosecond-precision timestamp types `TIMESTAMP_NTZ(p)` / `TIMESTAMP_LTZ(p)` (precision `p` in `[7, 9]`, represented by `TimestampNTZNanosType` / `TimestampLTZNanosType`): - `DATE -> TIMESTAMP_NTZ(p)` - `DATE -> TIMESTAMP_LTZ(p)` - `TIMESTAMP_NTZ(p) -> DATE` - `TIMESTAMP_LTZ(p) -> DATE` Semantics are kept at parity with the existing microsecond `DATE <-> TIMESTAMP` casts: - `DATE -> TIMESTAMP_LTZ(p)`: midnight of the date in the session time zone, sub-microsecond part = 0. - `DATE -> TIMESTAMP_NTZ(p)`: midnight UTC, sub-microsecond part = 0. - `TIMESTAMP_LTZ(p) -> DATE`: calendar date in the session time zone; the time-of-day and sub-microsecond digits are dropped. - `TIMESTAMP_NTZ(p) -> DATE`: calendar date on the UTC wall-clock grid; the time-of-day and sub-microsecond digits are dropped. All changes are confined to `Cast.scala`: - `canCast` / `canAnsiCast` allow all four conversions. - `needsTimeZone` marks only the LTZ directions as zone-sensitive; the NTZ directions use a fixed UTC grid (mirroring micro `TIMESTAMP_NTZ <-> DATE`, which is intentionally zone-independent). - `canANSIStoreAssign` returns `false` for all four `DATE <-> nanos` combinations, so an explicit `CAST` is required and there is no silent store assignment in either direction. This avoids silently dropping sub-microsecond digits and time-of-day, consistent with the nanos -> micros narrowing rule from SPARK-57293. - `forceNullable` returns `false` for all four (parity with micro `Timestamp <-> Date`). - The interpreted and codegen cast methods (`castToDate`, `castToTimestampLTZNanos`, `castToTimestampNTZNanos`, and their `*Code` counterparts) implement the conversions. No implicit up-cast / type-coercion changes (`UpCastRule` and `findWiderDateTimeType` are unchanged). ### Why are the changes needed? The nanosecond-precision timestamp types `TIMESTAMP_NTZ(p)` and `TIMESTAMP_LTZ(p)` currently have no `CAST` support to or from `DATE`: the Catalyst `Cast` expression only implements `DATE <-> TIMESTAMP` and `DATE <-> TIMESTAMP_NTZ` for the microsecond GA types, plus the micros <-> nanos casts added in SPARK-57293. This is a gap in the nanosecond timestamp feature (SPARK-56822); users should be able to convert between `DATE` and the nanosecond timestamp types just as they can with the microsecond types. ### Does this PR introduce _any_ user-facing change? Yes. Casting between `DATE` and `TIMESTAMP_NTZ(p)` / `TIMESTAMP_LTZ(p)` previously failed type checking; it is now a supported explicit `CAST` in both directions. There is no change to released Spark versions, since the nanosecond timestamp types are unreleased (under SPARK-56822 on master). ### How was this patch tested? Added tests in `CastSuiteBase` (run under ANSI-off, ANSI-on, and `try_cast` via the `CastWithAnsiOffSuite`, `CastWithAnsiOnSuite`, and `TryCastSuite` subclasses): - Null casts for both directions at MIN/MAX precision. - `canANSIStoreAssign` assertions that all four `DATE <-> nanos` combinations are blocked. - "cast between date and timestamp_ntz with nanosecond precision": `DATE -> NTZ(p)` yields midnight UTC with nanos = 0; `NTZ(p) -> DATE` drops the time-of-day and sub-microsecond digits; round trip; nulls both ways. - "cast between date and timestamp_ltz with nanosecond precision": same in the session zone, plus a zone-sensitivity case (a western zone maps the date to a later midnight instant) to exercise `needsTimeZone`. ``` build/sbt 'catalyst/testOnly org.apache.spark.sql.catalyst.expressions.CastWithAnsiOffSuite org.apache.spark.sql.catalyst.expressions.CastWithAnsiOnSuite org.apache.spark.sql.catalyst.expressions.TryCastSuite' ``` `catalyst/scalastyle` and `catalyst/Test/scalastyle` both pass. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Cursor (Claude Opus 4.8) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
