NGA-TRAN commented on issue #23569: URL: https://github.com/apache/datafusion/issues/23569#issuecomment-5347049735
## Test coverage for this case https://github.com/apache/datafusion/pull/24500 (draft) adds `metrics_range_sorted`: range-partitioned on `timestamp` into two 60-minute file groups, sorted on `(key, timestamp)`, and pins today's plan for: ```sql SELECT key, date_bin(INTERVAL '60 seconds', timestamp) AS time_bin, sum(value) FROM metrics_range_sorted WHERE service = 'a' GROUP BY key, time_bin ``` The hour split is aligned to 60-second bins, so `(key, time_bin)` is partition-disjoint, but the planner still inserts Partial → hash `RepartitionExec` → Final (`ordering_mode=Sorted` is already recognized). That PR is the current-plan coverage this issue should improve. A follow-up can skip the shuffle when `Range([timestamp])` subset-satisfies grouping by a monotonic `date_bin` whose bins do not straddle split points. Related: #24438 (streaming when partitions are unsorted but non-overlapping). This issue is the sorted / range-on-timestamp case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
