[
https://issues.apache.org/jira/browse/SPARK-57323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk reassigned SPARK-57323:
--------------------------------
Assignee: Max Gekk
> Support casting between DATE and nanosecond-precision timestamp types
> TIMESTAMP_NTZ(p) and TIMESTAMP_LTZ(p)
> -----------------------------------------------------------------------------------------------------------
>
> Key: SPARK-57323
> URL: https://issues.apache.org/jira/browse/SPARK-57323
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Max Gekk
> Assignee: Max Gekk
> Priority: Major
> Labels: pull-request-available
> Attachments: SPARK-57323-plan.md
>
>
> The nanosecond-precision timestamp types {{TIMESTAMP_NTZ(p)}} and
> {{TIMESTAMP_LTZ(p)}} (precision {{p}} in [7, 9], represented by
> {{TimestampNTZNanosType}} / {{TimestampLTZNanosType}}) currently have no CAST
> support to or from {{DATE}}. The Catalyst {{Cast}} expression only implements
> {{DATE}} <-> {{TIMESTAMP}} and {{DATE}} <-> {{TIMESTAMP_NTZ}} for the
> microsecond GA types, plus the micros <-> nanos casts added in SPARK-57293.
> This task adds explicit CAST support in both directions:
> * {{DATE}} -> {{TIMESTAMP_NTZ(p)}}
> * {{DATE}} -> {{TIMESTAMP_LTZ(p)}}
> * {{TIMESTAMP_NTZ(p)}} -> {{DATE}}
> * {{TIMESTAMP_LTZ(p)}} -> {{DATE}}
> h2. Semantics
> Parity with the existing microsecond {{DATE}} <-> {{TIMESTAMP}} casts:
> * {{DATE}} -> {{TIMESTAMP_LTZ(p)}}: midnight of the date in the session time
> zone, sub-microsecond part = 0.
> * {{DATE}} -> {{TIMESTAMP_NTZ(p)}}: midnight UTC, sub-microsecond part = 0.
> * {{TIMESTAMP_LTZ(p)}} -> {{DATE}}: calendar date in the session time zone;
> the time-of-day and sub-microsecond digits are dropped.
> * {{TIMESTAMP_NTZ(p)}} -> {{DATE}}: calendar date on the UTC wall-clock grid;
> the time-of-day and sub-microsecond digits are dropped.
> h2. Scope
> * Explicit CAST only. {{canCast}} / {{canAnsiCast}} allow all four
> conversions.
> * {{canANSIStoreAssign}} blocks {{DATE}} <-> nanos in both directions (no
> silent store assignment); an explicit CAST is required. This avoids silently
> dropping sub-microsecond digits and time-of-day, consistent with the nanos ->
> micros narrowing rule from SPARK-57293.
> * No implicit up-cast / type-coercion changes ({{UpCastRule}} and
> {{findWiderDateTimeType}} are unchanged).
> Implementation is confined to {{Cast.scala}} (eligibility rules,
> {{needsTimeZone}}, {{forceNullable}}, interpreted and codegen cast methods)
> with tests in {{CastSuiteBase}}, covering both ANSI-off and ANSI-on modes,
> time-zone sensitivity for the LTZ directions, and null handling.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
