[
https://issues.apache.org/jira/browse/SPARK-57461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-57461:
-----------------------------
Affects Version/s: 4.3.0
(was: 5.0.0)
> Cast between nanosecond-precision timestamp types with different fractional
> precision
> -------------------------------------------------------------------------------------
>
> Key: SPARK-57461
> URL: https://issues.apache.org/jira/browse/SPARK-57461
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Max Gekk
> Priority: Major
>
> Umbrella: SPARK-56822 (Timestamps with nanosecond precision).
> Support casting a nanosecond-precision timestamp to the same family with a
> different fractional precision, i.e. TIMESTAMP_NTZ(p1) -> TIMESTAMP_NTZ(p2)
> and TIMESTAMP_LTZ(p1) -> TIMESTAMP_LTZ(p2) for p1 != p2 (p in 7-9).
> Today Cast.scala has no canCast / canAnsiCast rule for an already-nanosecond
> source to a nanosecond target with a different precision, and the runtime
> helpers castToTimestampNTZNanos / castToTimestampLTZNanos only handle String
> / microsecond / DATE sources - their precision parameter is used only for
> string parsing. As a result CAST(ts7 AS TIMESTAMP_NTZ(9)) (and narrowing
> CAST(ts9 AS TIMESTAMP_NTZ(7))) fails the type check.
> Scope:
> - Add canCast / canAnsiCast rules for nanos(p1) -> nanos(p2) within the same
> family (NTZ->NTZ, LTZ->LTZ).
> - Implement interpreted and codegen runtime: widening keeps the value;
> narrowing truncates (or rounds per FSP rules, aligned with microsecond cast
> behavior) the sub-microsecond nanos component to the target precision.
> - Decide store-assignment / up-cast behavior (coordinate with SPARK-57303):
> widening p1<p2 as up-cast vs narrowing requiring explicit cast.
> - Tests in CastSuite / AnsiCastSuite covering all 7-9 combinations, NTZ and
> LTZ, ANSI and non-ANSI, including boundary nanos values.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
