geoffreyclaude opened a new pull request, #26370: URL: https://github.com/apache/datafusion/pull/26370
## Which issue does this PR close? Closes #26369. Follow-up to #25150. ## Rationale for this change Fully ordered multi-column `GROUP BY` still hashes temporal, decimal, and fixed-size binary keys. These types can use the existing adjacent-run grouping path because Arrow's adjacent equality agrees with the grouping builders. ## What changes are included in this PR? - Enable timestamps with all units and timezone metadata, dates, valid time units, durations, intervals, Decimal32/64/128/256, and every valid fixed-size binary width. - Preserve type metadata and the existing selection of single-column specializations. - Add SQL benchmarks in the first commit, then the type expansion and regression tests in the second. The benchmark loader materializes ordered input before timing and retains its verified ordering metadata. ## What is the testing strategy for this PR? Differential tests compare ordered and hash grouping across nulls, units/timezones, decimal metadata, sliced batches, and group emission. A dedicated test covers zero-width binary. SQL regression tests cover single-stage and partial/final aggregation, mixed sort directions, null ordering, and aggregate filters. Local validation passed: the extended workspace suite (12,672 Rust tests and 535 SQL logic files), 107 benchmark harness tests, all 16 SQL benchmark cases, formatting, documentation, license, TOML, and spelling checks. Clippy passes for all targets/features with `clippy::mutable_key_type` allowed. The unmodified baseline fails the strict command at three existing sites in `datafusion/optimizer/src/push_down_filter.rs`. <details> <summary>SQL benchmark results</summary> Apple M1 Max, `release-nonlto`, 8,388,608 rows, one input partition. Medians of 20 measured iterations per case across two alternating runs of each binary; the first iteration of each run is excluded. Input generation and sorting happen before timing. Both versions passed the result and physical-plan checks. | Query | Before (ms) | After (ms) | Speedup | | --- | ---: | ---: | ---: | | Int64 control | 28.41 | 28.18 | 1.01x | | Utf8View control | 33.42 | 33.22 | 1.01x | | Timestamp (ns) | 133.09 | 28.35 | 4.69x | | Timestamp (µs, UTC) | 132.68 | 28.40 | 4.67x | | Date32 | 130.85 | 27.83 | 4.70x | | Time32 (ms) | 130.82 | 27.78 | 4.71x | | Time64 (ns) | 132.29 | 28.32 | 4.67x | | Duration (ns) | 132.72 | 28.34 | 4.68x | | Interval (day/time) | 132.76 | 29.53 | 4.50x | | Interval (month/day/ns) | 156.41 | 34.15 | 4.58x | | Decimal128 | 133.51 | 31.04 | 4.30x | | Decimal256 | 138.64 | 34.33 | 4.04x | | FixedSizeBinary(16) | 152.15 | 44.13 | 3.45x | | FixedSizeBinary(7) | 169.92 | 65.49 | 2.59x | | Timestamp, singleton groups | 265.87 | 81.66 | 3.26x | | Timestamp, 512 rows/group | 119.33 | 25.66 | 4.65x | ```shell cargo run --profile release-nonlto --bin benchmark_runner -- ordered_group_types --rows 8388608 --iterations 11 ``` Run from `benchmarks` on each commit. </details> ## Are there any user-facing changes? Already-sorted multi-column aggregations can use lower-overhead key storage for these types. Query results stay the same. Developed with AI assistance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
