MaxGekk opened a new pull request, #56593: URL: https://github.com/apache/spark/pull/56593
### What changes were proposed in this pull request? This PR allows `unix_timestamp` / `to_unix_timestamp` to accept the nanosecond-precision timestamp types `TIMESTAMP_LTZ(p)` / `TIMESTAMP_NTZ(p)` (`p in [7, 9]`, i.e. `AnyTimestampNanoType`) in their timestamp-argument form. The result stays whole-second `BIGINT`; the sub-second digits are dropped. Concretely: - Extends the `UnixTime` base (`UnixTimestamp` / `ToUnixTimestamp`) input typing to accept `AnyTimestampNanoType` alongside the existing string / date / microsecond-timestamp types. - Reads `epochMicros` from `TimestampNanosVal` in the timestamp branch of both the interpreted (`eval`) and codegen (`doGenCode`) paths, dividing by `MICROS_PER_SECOND` exactly like the existing microsecond/NTZ path (plain integer division, truncation toward zero), so nanos and micro types behave identically. - Adds catalyst unit tests (interpreted + codegen), Scala/Java Column API end-to-end tests, and SQL golden-file coverage for `TIMESTAMP_NTZ(p)` / `TIMESTAMP_LTZ(p)`. The string-parsing overload `unix_timestamp(str, fmt)` producing nanosecond precision is out of scope and tracked separately under Parsing/Formatting. ### Why are the changes needed? Part of the [SPARK-56822](https://issues.apache.org/jira/browse/SPARK-56822) umbrella (timestamps with nanosecond precision). `unix_timestamp` already accepts microsecond timestamp families but rejected the new nanosecond-precision timestamp types, leaving valid conversions unsupported. ### Does this PR introduce _any_ user-facing change? Yes. `unix_timestamp(timeExp)` / `to_unix_timestamp(timeExp)` now accept `TIMESTAMP_LTZ(p)` / `TIMESTAMP_NTZ(p)` and return the whole-second `BIGINT`. This is a change only within the unreleased nanosecond-timestamp preview; existing microsecond / date / string behavior is unchanged. Example: ```sql SELECT unix_timestamp(TIMESTAMP_LTZ '2008-12-25 15:30:00.123456789'); -- 1230219000 ``` ### How was this patch tested? - `build/sbt 'catalyst/testOnly org.apache.spark.sql.catalyst.expressions.DateExpressionsSuite'` - `build/sbt 'sql/testOnly org.apache.spark.sql.TimestampNanosFunctionsAnsiOnSuite org.apache.spark.sql.TimestampNanosFunctionsAnsiOffSuite'` - `SPARK_GENERATE_GOLDEN_FILES=1 build/sbt 'sql/testOnly org.apache.spark.sql.SQLQueryTestSuite -- -z "nanos"'` ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Cursor -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
