[
https://issues.apache.org/jira/browse/SPARK-57528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk reassigned SPARK-57528:
--------------------------------
Assignee: Max Gekk
> Support nanosecond-precision timestamps in the `unix_timestamp` function
> ------------------------------------------------------------------------
>
> Key: SPARK-57528
> URL: https://issues.apache.org/jira/browse/SPARK-57528
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Max Gekk
> Assignee: Max Gekk
> Priority: Major
> Labels: pull-request-available
>
> h3. Background
> Part of the SPARK-56822 umbrella. {{unix_timestamp(timestamp)}} (and the
> no-arg form)
> returns the number of whole seconds since the Unix epoch as {{BIGINT}}. It
> currently accepts
> {{TimestampType}}, {{DateType}} and strings, but not the new
> nanosecond-precision timestamp
> types.
> h3. Proposal
> Allow {{unix_timestamp}} to accept nanosecond-precision timestamps
> ({{TIMESTAMP_LTZ(p)}} / {{TIMESTAMP_NTZ(p)}}, i.e. {{AnyTimestampNanoType}})
> in its
> timestamp-argument form. The result stays {{BIGINT}} seconds; sub-second
> digits are
> truncated, consistent with current behavior.
> * Value: {{Math.floorDiv(epochMicros, MICROS_PER_SECOND)}}, read from
> {{TimestampNanosVal}}.
> * No overflow concern (result is seconds-granular).
> {code:sql}
> SELECT unix_timestamp(TIMESTAMP_LTZ '2008-12-25 15:30:00.123456789');
> -- 1230219000
> {code}
> h3. Implementation notes
> * Extend the input type check of the {{UnixTime}} base ({{UnixTimestamp}} /
> {{ToUnixTimestamp}})
> to include {{AnyTimestampNanoType}}.
> * In the timestamp branch, read {{epochMicros}} from {{TimestampNanosVal}}
> instead of a plain
> {{Long}}.
> * Interpreted + codegen paths.
> h3. Out of scope
> * The string-parsing overload {{unix_timestamp(str, fmt)}} producing
> nanosecond precision -
> tracked separately under Parsing/Formatting (P1).
> h3. Acceptance criteria
> * {{unix_timestamp}} accepts {{TIMESTAMP_LTZ(p)}} / {{TIMESTAMP_NTZ(p)}} and
> returns the
> correct whole-second {{BIGINT}}.
> * Existing micros / date / string behavior unchanged.
> * Unit tests + golden files added.
> Parent: SPARK-56822
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
