stevomitric opened a new pull request, #56592:
URL: https://github.com/apache/spark/pull/56592
### What changes were proposed in this pull request?
This PR adds end-to-end, Catalyst-level, and golden-file test coverage for
the `MIN` and `MAX` aggregates (and the closely related `min_by` / `max_by` /
`greatest` / `least`) over the nanosecond-precision timestamp types
`TIMESTAMP_NTZ(p)` / `TIMESTAMP_LTZ(p)` (`p ∈ [7, 9]`), part of the nanosecond
timestamp preview (SPARK-56822).
No production code is changed. `Max`/`Min` are type-agnostic
`DeclarativeAggregate`s whose only type gate is
`TypeUtils.checkForOrderingExpr`, which already returns `TypeCheckSuccess` for
the nanosecond types because they are orderable `AtomicType`s `Comparable` were
wired in SPARK-57103, the physical value is UnsafeRow-mutable (SPARK-56981),
and `CodeGenerator.genComp` already has a dedicated `case _:
AnyTimestampNanoType => compareTo` arm. Since `dataType = child.dataType`, the
result preserves the input precision. This mirrors the TimeType precedent
(SPARK-52626 / SPARK-52660), where becoming orderable made `MIN`/`MAX` work and
the remaining work was test coverage.
Added tests:
- `TimestampNanosFunctionsSuiteBase` (runs under ANSI on and off):
precision/type preservation; sub-microsecond ordering on both the codegen and
interpreted paths (two values sharing the same `epochMicros` but differing
within the microsecond); all-NULL and empty input → NULL; `GROUP BY` a
nanosecond key whose sub-microsecond remainder must keep distinct keys from
collapsing; `min_by`/`max_by`/`greatest`/`least` over same-precision columns;
and equivalence with the microsecond path when the sub-microsecond digits are
zero.
- `ExpressionTypeCheckingSuite`: asserts `Max`/`Min` `checkInputDataTypes()`
succeeds for `TimestampNTZNanosType` / `TimestampLTZNanosType` and that the
result type preserves the precision.
- `timestamp-ntz-nanos.sql` / `timestamp-ltz-nanos.sql`: `MAX`/`MIN` and
`GROUP BY` queries, with regenerated `results/` and `analyzer-results/` golden
files.
Mixed-precision inputs (e.g. `greatest(ts_p7, ts_p9)`, a `UNION` of two
precisions, or comparing an aggregate against a micros literal) route through
`findWiderDateTimeType`, which has no nanosecond arm yet, so they are
intentionally **out of scope** here and tracked by SPARK-57454; every column in
these tests is strictly same-precision.
### Why are the changes needed?
`MIN`/`MAX` over nanosecond timestamps are a core, expected operation, but
the nanosecond preview had no aggregate test coverage.
### Does this PR introduce any user-facing change?
No. Tests only;
### How was this patch tested?
New tests in this PR.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Cursor
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
