Max Gekk created SPARK-57834:
--------------------------------
Summary: Support nanosecond-precision timestamp endpoints in the
sequence function
Key: SPARK-57834
URL: https://issues.apache.org/jira/browse/SPARK-57834
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.3.0
Reporter: Max Gekk
This sub-task is part of the umbrella SPARK-56822 (timestamps with nanosecond
precision).
h2. Problem
{{Sequence}} (collectionOperations.scala ~L3322-3378) allows only
{{TimestampType | TimestampNTZType}} endpoints and steps through {{Long}}
micros, so nanosecond start/stop raise an analysis error.
h2. Goal
Generate an array of nanosecond timestamps from nanosecond start/stop with an
interval step, preserving precision/family and the {{nanosWithinMicro}}
remainder across steps.
h2. Scope
Extend {{Sequence.checkInputDataTypes}} and the implementation to accept
{{AnyTimestampNanoType}}; add a {{TimestampNanosVal}} stepping path (day-time
interval step, consistent with the SPARK-57501 arithmetic).
h2. Acceptance criteria
* {{sequence(ts_nanos_start, ts_nanos_stop, interval)}} yields correctly-spaced
nanosecond timestamps of the input type.
h2. Testing
{{CollectionExpressionsSuite}}; nanos golden files.
h2. Dependencies
None hard - steps via resolved SPARK-57501 arithmetic; coordinate with the
overflow/range-hardening sub-task for step overflow.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
