stevomitric opened a new pull request, #59782: URL: https://github.com/apache/spark/pull/59782
### What changes were proposed in this pull request? `date_add` / `date_sub` (and the `dateadd` alias) take a DATE start and rely on the `DateTimeOperations` coercion rule to cast a timestamp start to DATE. That rule matched only the microsecond timestamp types, so `TIMESTAMP_NTZ(p)` / `TIMESTAMP_LTZ(p)` (p in [7, 9]) failed with `DATATYPE_MISMATCH.UNEXPECTED_INPUT_TYPE` in both ANSI modes. This PR makes the `DateAdd` / `DateSub` cases of `DateTimeOperationsTypeCoercion` and `AnsiDateTimeOperationsTypeCoercion` also match `AnyTimestampNanoTypeExpression`, the same pair of extractors `AnsiGetDateFieldOperationsTypeCoercion` uses for the date field functions (SPARK-57469). The nanosecond start is cast to DATE exactly like a microsecond one: the sub-day digits never reach a DATE result, and `TIMESTAMP_LTZ(p)` resolves the date in the session time zone. ### Why are the changes needed? Part of SPARK-56822. `date_add(TIMESTAMP_NTZ '2020-01-01 00:00:00.123456789', 1)` failed while the microsecond equivalent returns `2020-01-02` and the sibling date-valued functions (`add_months`, `last_day`, `next_day`, `datediff`, `to_date`, `trunc`) already accept nanosecond timestamps. That inconsistency is what the SPIP's parity criterion rules out. With this change every DATE-input function agrees between microsecond and nanosecond timestamps in both ANSI modes. ### Does this PR introduce _any_ user-facing change? Yes, only for the nanosecond timestamp types, which are behind the preview flag `spark.sql.timestampNanosTypes.enabled`: `date_add` / `date_sub` / `dateadd` now accept them instead of failing. Nothing changes for DATE, string or microsecond timestamp inputs. ### How was this patch tested? - `TypeCoercionSuite` / `AnsiTypeCoercionSuite` "Datetime operations": `DateAdd` / `DateSub` over NTZ and LTZ nanos get a `Cast(_, DateType)` start in both rule sets. - New `TimestampNanosFunctionsSuiteBase` tests (ANSI on and off): NTZ/LTZ at p=7..9 match the microsecond path, including LTZ under two session time zones, plus the Column API and NULLs; new golden cases in `timestamp-ntz-nanos.sql` / `timestamp-ltz-nanos.sql` for both ANSI modes. - Existing `DateExpressionsSuite` and the `date.sql` / `timestamp.sql` / `timestamp-ntz.sql` golden files pass unchanged. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Isaac -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
