[
https://issues.apache.org/jira/browse/SPARK-57850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-57850:
-----------------------------
Shepherd: Max Gekk
> Return the correct fractional-second scale from EXTRACT(SECOND FROM TIME(p))
> for p > 6
> --------------------------------------------------------------------------------------
>
> Key: SPARK-57850
> URL: https://issues.apache.org/jira/browse/SPARK-57850
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Max Gekk
> Priority: Major
>
> This sub-task is part of the umbrella SPARK-57550 (extend support for the
> TIME data type).
> h2. Problem
> {{SecondsOfTimeWithFraction}} (timeExpressions.scala ~L364-377;
> DateTimeUtils.scala ~L173-178) always returns {{DECIMAL(8,6)}}, truncating
> sub-microsecond digits for {{TIME(7..9)}}. ANSI requires the {{EXTRACT(SECOND
> ...)}} result scale to be at least the source fractional-seconds precision
> (std ~L15080-15082). This is now relevant since TIME precision goes to 9
> (SPARK-57551).
> h2. Goal
> Make the {{EXTRACT(SECOND FROM TIME(p))}} / {{date_part}} result decimal
> scale track the TIME precision (e.g. {{DECIMAL(11,9)}} for p in [7,9]) so
> nanoseconds are preserved. Mirror the nanosecond-timestamp
> {{SecondWithFractionNanos}} behavior (SPARK-57340).
> h2. Scope
> Make {{SecondsOfTimeWithFraction}} (and its {{DateTimeUtils}} helper) return
> a decimal whose scale is derived from the TIME precision.
> h2. Acceptance criteria
> * {{EXTRACT(SECOND FROM TIME(9) '..:..:05.123456789')}} returns
> {{5.123456789}}; p<=6 unchanged.
> h2. Testing
> {{TimeExpressionsSuite}}; {{extract}} / {{date_part}} golden files.
> h2. Dependencies
> Builds on the resolved precision->9 (SPARK-57551).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
