[
https://issues.apache.org/jira/browse/SPARK-57841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-57841:
-----------------------------
Shepherd: Max Gekk
> Add cross-precision and nested test coverage for nanosecond-precision
> timestamps
> --------------------------------------------------------------------------------
>
> Key: SPARK-57841
> URL: https://issues.apache.org/jira/browse/SPARK-57841
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Max Gekk
> Priority: Major
>
> This sub-task is part of the umbrella SPARK-56822 (timestamps with nanosecond
> precision).
> h2. Problem
> The cross-precision widening logic is implemented (SPARK-57454), but several
> operations lack dedicated nanosecond e2e tests: join on mixed-precision keys,
> {{INTERSECT}} / {{EXCEPT}}, {{BETWEEN}}, {{if()}} / {{nvl()}} / {{ifnull()}},
> {{InSubquery}}; nested {{explode(array<ts_nanos>)}}, {{element_at}},
> struct-field extraction, and map-with-nanosecond-key lookup (note
> {{GetMapValue}}'s hash fast-path lists only microsecond timestamps); and
> {{show()}} / {{collect()}} rendering at p=7/8/9.
> h2. Goal
> Lock in behavior with tests, fixing any latent issue surfaced (e.g. the
> map-key hash fast-path).
> h2. Scope
> Add e2e tests to the widening SQL suites and golden files; add nested-type
> tests; add show/collect precision assertions.
> h2. Acceptance criteria
> * Tests cover the listed operations at nanosecond precision and pass.
> h2. Testing
> {{TimestampNanosWideningSuite}}, {{timestamp-*-nanos.sql}},
> collection/complex-type suites.
> h2. Dependencies
> Do AFTER SPARK-57811 (string <-> nanosecond coercion) for the
> string-comparison cases; otherwise exercises the resolved widening and the
> function sub-tasks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
