vboo123 opened a new pull request, #57101:
URL: https://github.com/apache/spark/pull/57101
### What changes were proposed in this pull request?
Extend `sequence(start, stop[, step])` to accept TIME endpoints with a
day-time interval step. Previously, `Sequence.checkInputDataTypes` in
collectionOperations.scala only allowed integral, DATE, TIMESTAMP, and
TIMESTAMP_NTZ endpoints. TIME fell through to the catch-all rejection branch,
so any call like `sequence(TIME '08:00:00', TIME '10:00:00', INTERVAL '30'
MINUTE)` raised an analysis error.
Changes:
1. `Sequence.checkInputDataTypes` now recognizes TIME endpoints paired with
an optional day-time interval step.
2. A new `TimeSequenceImpl` generates the array via a plain integer walk in
nanoseconds.
3. The `SEQUENCE_WRONG_INPUT_TYPES` error message is extended to mention
TIME.
TIME is stored internally as nanoseconds-of-day while a day-time interval is
stored as microseconds, so `TimeSequenceImpl` converts the step from
microseconds to nanoseconds and walks the range, truncating each element to the
start endpoint's declared precision. It does not reuse the DATE/TIMESTAMP
temporal machinery (no calendar, DST, or time-zone arithmetic) because a TIME
sequence stays within the single-day domain [0, 24:00) by construction.
When no step is given the default is INTERVAL '1' SECOND. This is open to
reviewer feedback since DATE defaults to 1 day and TIMESTAMP defaults to 1
month, but TIME has no obvious natural default.
SPARK-57853 (defines TIME +/- INTERVAL out-of-range semantics, open PR
#57044) does not conflict with this implementation because all produced values
are within-day by construction.
### Why are the changes needed?
The TIME type was added by SPIP SPARK-51162 (shipped in Spark 4.1.0) and
works in the engine, but `sequence()` rejected TIME endpoints at analysis time
because the type check had no branch for it. This is a sub-task of the TIME
umbrella SPARK-57550.
### Does this PR introduce any user-facing change?
Yes. Users can now generate TIME arrays with `sequence`:
```sql
SELECT sequence(TIME '08:00:00', TIME '10:00:00', INTERVAL '30' MINUTE);
-- [08:00:00, 08:30:00, 09:00:00, 09:30:00, 10:00:00]
```
### How was this patch tested?
Added test("SPARK-57852: Sequence of times") to CollectionExpressionsSuite
covering null arguments, ascending 30-minute step, descending negative step,
stop not exactly reachable, single element (start == stop), default 1-second
step, sub-second step preserving precision, and a negative case asserting a
year-month interval step is rejected at analysis time. All 62 tests in
CollectionExpressionsSuite pass locally.
### Was this patch authored or co-authored using generative AI tooling?
Yes. Generated using Kiro (Claude Opus 4.8)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
