marcuslin123 opened a new pull request, #57338: URL: https://github.com/apache/spark/pull/57338
### What changes were proposed in this pull request? Add `TimeType` support to the `avg`/`mean` aggregate function. Previously `AVG()` only accepted numeric and ANSI interval types and rejected TIME. Since TIME is internally a `Long` number of nanoseconds since midnight, it can be averaged like the interval types. ```sql SELECT avg(t) FROM VALUES (TIME'10:00:00'), (TIME'14:00:00') AS tab(t); -- 12:00:00 ``` The changes: - `Average.scala`: accept `AnyTimeType` in `inputTypes`; map a TIME input to a TIME output of the same precision in `resultType`; accumulate the sum as `LongType`; and add a TIME branch to `evaluateExpression` that divides the summed nanoseconds by the count and re-tags the result as `TimeType(precision)`. - To accumulate the sum, the child value is converted to the sum's type via `PreciseTimestampConversion` rather than a regular cast. A regular `CAST(time AS BIGINT)` is lossy for this purpose — it truncates to whole seconds — so it would corrupt the average. `PreciseTimestampConversion` reinterprets the underlying `Long` nanoseconds without changing the value. - New `TimeDivide` expression (in `intervalExpressions.scala`, modeled on `DivideDTInterval`) that divides a `Long`-nanosecond sum by the count with HALF_UP rounding and returns a `TimeType(precision)`. `DivideDTInterval` could not be reused because its output type is hardcoded to `DayTimeIntervalType`, and there is no `CAST(BIGINT AS TIME)` to re-wrap the result. - New `TypeUtils.checkForAnsiIntervalOrNumericOrTimeType` helper. `Sum`'s existing `checkForAnsiIntervalOrNumericType` is intentionally left unchanged, since `Sum` does not (yet) support TIME. The output preserves the input precision (e.g. `avg` over a `TIME(3)` column yields `TIME(3)`). This mirrors how `min`/`max`/`percentile` already support TIME. ### Why are the changes needed? TIME is orderable and quantifiable, and `min`, `max`, `percentile`, `approx_percentile`, etc. already support it. `avg` was an inconsistent gap: to average a TIME column, users had to cast to a numeric type, average, and cast back. Native support lets them write `avg(time_col)` directly. ### Does this PR introduce _any_ user-facing change? Yes. `avg`/`mean` now accepts TIME columns and returns a TIME average (same precision). Previously it raised a `DATATYPE_MISMATCH.UNEXPECTED_INPUT_TYPE` error. ### How was this patch tested? Added a test in `DataFrameAggregateSuite` covering: basic average, sub-second rounding, NULL handling, all-null/empty input (returns NULL), precision preservation (`TIME(3)`), grouped average, and the SQL `avg` surface over TIME literals. Updated the expected error message in `ExpressionTypeCheckingSuite` (avg's required-type list now includes TIME). Full `DataFrameAggregateSuite` (168 tests) passes with no regressions, including the existing interval-avg test. ### Was this patch authored or co-authored using generative AI tooling? Generative AI tooling (Claude Code) was used as an assistive tool for implementation guidance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
