shinzoxD opened a new pull request, #24495: URL: https://github.com/apache/datafusion/pull/24495
## Which issue does this PR close? - Closes #24470 ## Rationale for this change `PartitionedTopKExec` already collected `ExecutionPlanMetricsSet` internally, but `ExecutionPlan::metrics()` was unimplemented so it returned `None`. `EXPLAIN ANALYZE` therefore showed no metrics, and `output_rows` / `output_batches` could not be checked against what the operator actually emits. ## What changes are included in this PR? - Implement `PartitionedTopKExec::metrics()` so it returns the existing metrics set, matching `SortExec` / window operators. - Record baseline output metrics (`output_rows`, `output_batches`, `output_bytes`) on the coalesced batches sent to the consumer, not on the pre-coalesce per-partition heap batches. `elapsed_compute` was already tracked via `TopKMetrics`. This does not address #24468 (`SortExec` TopK `output_batches`). ## Are these changes tested? Added the issue's reproduction as `partitioned_topk_output_batches_metric_matches_emitted_batches`: it asserts `metrics()` is `Some` and that `output_batches` equals the number of batches actually emitted (5 coalesced batches of 10 rows). Also ran: - `cargo test -p datafusion-physical-plan --lib topk` (82 passed) - `cargo test -p datafusion --test core_integration partitioned_topk_output_batches_metric_matches_emitted_batches` (passed) - `cargo fmt --all` - `cargo clippy -p datafusion-physical-plan --all-targets --all-features -- -D warnings` - `cargo clippy -p datafusion --test core_integration -- -D warnings` ## Are there any user-facing changes? `EXPLAIN ANALYZE` now reports metrics for `PartitionedTopKExec` (`output_rows`, `output_batches`, `elapsed_compute`, etc.). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
