MaxGekk opened a new pull request, #56677: URL: https://github.com/apache/spark/pull/56677
### What changes were proposed in this pull request? This PR adds bidirectional casts between the `TIME(p)` data type (`p` in `[0, 9]`) and `TIMESTAMP_NTZ(q)` (`q` in `[6, 9]`, where `q=6` is the microsecond `TimestampNTZType` and `q` in `[7, 9]` is the nanosecond `TimestampNTZNanosType`). Semantics follow the SQL standard (section 6.13 `<cast specification>`): - `CAST(TIMESTAMP_NTZ(q) AS TIME(p))` (rule 15.d): extracts the hour/minute/second and fractional fields and truncates to precision `p`. Deterministic and time-zone independent. - `CAST(TIME(p) AS TIMESTAMP_NTZ(q))` (rule 17.c): the date fields come from `CURRENT_DATE` and the time fields from the value. Since `CURRENT_DATE` is stable within a query, the cast is stabilized via the existing `ComputeCurrentTime` optimizer rule, so it shares the same date literal as `current_date()`. Fractional precision handling is pure truncation (floor toward the precision step; no rounding). Both directions always succeed, so no new nullability or error condition is introduced. Implementation notes: - New rule-table entries in `Cast.canCast` / `Cast.canAnsiCast` for the four pairs, restricted to the NTZ family (`TIMESTAMP_LTZ` is intentionally not a valid counterpart). `canTryCast` inherits these for atomic types. - Interpreted and codegen paths for both directions. - `TIME -> TIMESTAMP_NTZ` advertises the `CURRENT_LIKE` tree pattern and `needsTimeZone`, and `ComputeCurrentTime` rewrites it into a date+time builder (`makeTimestampNTZ` / new internal `MakeTimestampNTZNanos`) anchored on the query current date. The `Cast` eval/codegen fallback (using `currentDate(zoneId)`) covers direct expression evaluation. - New helpers: `SparkDateTimeUtils.timestampNTZToNanosOfDay` / `timestampNTZNanosToNanosOfDay` and `DateTimeUtils.makeTimestampNTZNanos`. Out of scope: Structured Streaming batch-timestamp parity for `TIME -> TIMESTAMP_NTZ` (the cast uses the optimizer-instant current date rather than the micro-batch timestamp). ### Why are the changes needed? Spark already supports `TIME` casts to/from `STRING`, integral, and `DECIMAL` types, but there was no way to convert between `TIME` and `TIMESTAMP_NTZ`. These conversions are required by the SQL standard and are a common user need (attaching a time-of-day to a timestamp, or extracting the time-of-day from a timestamp). This is a sub-task of SPARK-56822 (timestamps with nanosecond precision). ### Does this PR introduce _any_ user-facing change? Yes. Casting between `TIME(p)` and `TIMESTAMP_NTZ(q)` is now allowed (previously it failed analysis with a cast type-mismatch). Examples: ```sql -- extract the time-of-day SELECT CAST(TIMESTAMP_NTZ'2020-05-17 12:34:56.789012' AS TIME(6)); -- 12:34:56.789012 -- attach the current date SELECT CAST(TIME'12:34:56.789012345' AS TIMESTAMP_NTZ(9)); -- <current_date> 12:34:56.789012345 ``` This is a new feature on an unreleased branch; there is no behavior change relative to a released version. ### How was this patch tested? - New unit tests in `CastSuiteBase` (run under ANSI on and off): allowed-pair / `needsTimeZone` matrix, `TIMESTAMP_NTZ(q) -> TIME(p)` values across all precisions (including pre-epoch and sub-microsecond truncation), interpreted-vs-codegen consistency, and date-independent round trips. - New test in `ComputeCurrentTimeSuite` asserting the forward cast is rewritten with a query-stable current-date literal consistent with `current_date()`. - New unit tests in `DateTimeUtilsSuite` for `makeTimestampNTZNanos` and the time-of-day extraction helpers. - New deterministic cases in `cast.sql` (and the imported `nonansi/cast.sql`) with regenerated golden files. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Cursor (Claude Opus 4.8) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
