benbellick opened a new pull request, #26835: URL: https://github.com/apache/datafusion/pull/26835
## Which issue does this PR close? Related to [#25345](https://github.com/apache/datafusion/issues/25345) and [#25344](https://github.com/apache/datafusion/issues/25344). This implements only the narrow `date_bin` case below. ## Rationale for this change Extend the aggregate-locality optimization from [#26449](https://github.com/apache/datafusion/pull/26449) to a small audited subset of `date_bin`, following the NULL-to-error fix in [#26607](https://github.com/apache/datafusion/pull/26607). When bins do not cross timestamp range splits, aggregation can reuse the input partitions instead of hash-repartitioning. ## What changes are included in this PR? Admit only two-argument calls over timezone-less nanosecond timestamps, with a non-null literal `IntervalMonthDayNano` stride whose months and days are zero and whose nanoseconds are positive. Explicit origins, other timestamp precisions, timezones, and other interval representations remain denied. For example, suppose `events` is range-partitioned on `event_time`, a timezone-less nanosecond timestamp column: ```sql SELECT date_bin(INTERVAL '1 hour', event_time) AS hour_bin, COUNT(*) AS event_count FROM events GROUP BY hour_bin; ``` If the ascending input partitions split at `2024-01-01 01:00:00`, timestamps immediately before the split bin to `00:00`, while the timestamp at the split bins to `01:00`. The groups stay partition-local, so aggregation can skip the hash shuffle. If the split is instead at `2024-01-01 01:30:00`, timestamps on both sides can bin to `01:00`. The shuffle remains. Every split must separate bins. Add a value-aware audit hook so admission can inspect the stride literal. Its default delegates to the existing type-only hook, leaving existing opt-ins unchanged. Concrete split checks still reject crossing bins and boundary evaluation errors. ## What is the testing strategy for this PR? Unit tests cover hook forwarding, admitted and rejected arguments, NULL preservation, negative timestamps, ascending and descending splits, and overflow at a split's adjacent timestamp. SQLLogicTests check plans and results for aligned bins, crossing bins, and explicit origins. ## Are there any user-facing changes? Eligible `date_bin` aggregations can avoid a hash shuffle. Function results are unchanged. --- Note: This PR was developed with AI assistance. All changes have been reviewed, and I take full responsibility for this contribution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
