[
https://issues.apache.org/jira/browse/SPARK-57827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-57827:
-----------------------------
Shepherd: Max Gekk
> Support nanosecond precision in make_timestamp, make_timestamp_ntz and
> make_timestamp_ltz
> -----------------------------------------------------------------------------------------
>
> Key: SPARK-57827
> URL: https://issues.apache.org/jira/browse/SPARK-57827
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Max Gekk
> Priority: Major
>
> This sub-task is part of the umbrella SPARK-56822 (timestamps with nanosecond
> precision).
> h2. Problem
> The public builders produce {{MakeTimestamp}} / {{MakeTimestampNTZ}} /
> {{MakeTimestampLTZ}} / {{MakeTimestampFromDateTime}} with microsecond output;
> the {{sec}} argument is {{DecimalType(16, 6)}} (datetimeExpressions.scala
> ~L3403-3512 / ~L3017-3036 / ~L3722-3734). Nanosecond-capable expressions
> {{MakeTimestampNTZNanos}} / {{MakeTimestampLTZNanos}} exist but are internal
> (used only by {{ComputeCurrentTime}}) and unregistered.
> h2. Goal
> Allow constructing nanosecond-precision timestamps from parts, either by
> widening {{sec}} to accept up to 9 fractional digits or by exposing the
> nanosecond make-timestamp expressions, and preserve TIME(9) sub-microsecond
> digits in {{make_timestamp(date, time)}}.
> h2. Scope
> Widen {{sec}} precision / add nanosecond codepaths in {{MakeTimestamp*}};
> register the nanosecond variants or extend the existing builders; preserve
> the NTZ/LTZ family.
> h2. Acceptance criteria
> * {{make_timestamp(..., 5.123456789)}} and {{make_timestamp(date, time(9))}}
> produce a nanosecond timestamp with the correct remainder.
> h2. Testing
> {{DateExpressionsSuite}}; nanos golden files.
> h2. Dependencies
> None - independent. The internal nanosecond make-timestamp expressions it
> exposes are reused by the current_timestamp(p)/localtimestamp(p) sub-task.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
