SubhamSinghal opened a new pull request, #23869:
URL: https://github.com/apache/datafusion/pull/23869
## Which issue does this PR close?
- Related to #6899. Completes the ROW_NUMBER + RANK + DENSE_RANK trio
requested by the issue (ROW_NUMBER shipped in #21479; RANK is in flight as PR
A).
## Rationale for this change
Naive plan for `Filter(dr <= K) → BoundedWindowAggExec(DENSE_RANK) →
SortExec(pk, ob) → input` sorts the full input even though only rows at the K
distinct-smallest ob values per partition are needed. DENSE_RANK is stable
under tail-pruning, so the rewrite preserves ranks for every retained row.
## What changes are included in this PR?
## Are these changes tested?
Yes
**h2o bench** — Q24–Q29 added; top-2 on 10M-row `large`, 3-iter avg vs
`enable_window_topn=false`:
| # | Cardinality | OFF (ms) | ON (ms) | Δ |
|-----|------------------|---------:|--------:|:-------------|
| Q24 | ~100 parts | 308.19 | 114.45 | 2.20× faster |
| Q25 | ~1K, low ties | 261.24 | 113.85 | 2.13× faster |
| Q26 | ~1K, heavy ties | 296.54 | 124.42 | 2.09× faster |
| Q27 | ~10K, low ties | 250.83 | 131.85 | 1.70× faster |
| Q28 | ~10K, heavy ties | 281.00 | 149.00 | 1.76× faster |
| Q29 | ~100K parts | 264.32 | 304.37 | 1.40× slower |
## Are there any user-facing changes?
No breaking changes. Rule fires on DENSE_RANK when
`datafusion.optimizer.enable_window_topn = true`; flag stays
`false` by default
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
