jayzhan211 opened a new pull request, #25535: URL: https://github.com/apache/datafusion/pull/25535
## Which issue does this PR close? - N/A — no issue filed; found while reading the code. ## Rationale for this change For a `RANK()` window Top-N (`PartitionedTopKExec: fn=rank`), the `output_rows` and `output_batches` metrics shown by `EXPLAIN ANALYZE` are too high whenever rows tie at the K-th value. `PartitionedTopKRank::emit` calls `record_output` on every tie batch before pushing it into the `BatchCoalescer`, and then calls `record_output` again on each completed batch coming out of that coalescer — which already contains those tie rows. Heap rows are only counted at the second site, so each tie row is counted twice and each tie batch adds a phantom output batch. Example: K = 2, one partition with values `5, 5, 10, 5`. Three rows are emitted in one batch, but the metrics report `output_rows=4`, `output_batches=2`. `PartitionedTopK` (`ROW_NUMBER`) and `PartitionedTopKDenseRank` only record at the coalescer output and are not affected. ## What changes are included in this PR? Remove the extra `record_output` call on tie batches so every emitted row is counted once, at the coalescer output, as in the other two operators. Query results are unchanged; only the metrics are corrected. ## What is the testing strategy for this PR? New unit test `test_partitioned_topk_rank_output_rows_counts_ties_once`, which emits heap rows plus a boundary tie and asserts `output_rows` / `output_batches` equal what the stream actually produced. It fails on `main` with `(2, 4)` vs `(1, 3)`. ## Are there any user-facing changes? `output_rows` / `output_batches` reported for `PartitionedTopKExec` with `fn=rank` are now accurate when ties are present. No API changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
