asolimando opened a new pull request, #23051:
URL: https://github.com/apache/datafusion/pull/23051
## Which issue does this PR close?
- Closes #22958.
## Rationale for this change
Follow-up to #21815. Per @2010YOUY01's suggestion there, this decouples
statistics
traversal/caching from each operator's computation: operators now express
only local
propagation, while the walk and cache live in an external
`StatisticsContext`.
## What changes are included in this PR?
- `StatisticsContext` owns the bottom-up walk + per-walk cache;
`compute(plan, args)`
resolves children, then calls the operator.
- `ExecutionPlan::statistics_from_inputs(input_stats, args)` — stateless
local
propagation from pre-resolved child stats; default delegates to deprecated
`partition_statistics`.
- `ExecutionPlan::child_stats_requests(partition) -> Vec<ChildStats>` —
per-child
directive (`At(Option<usize>)` / `Skip`) for which partition each child is
computed
at (e.g. broadcast joins request the build side overall; `UnionExec` skips
non-owning inputs).
- Removes the unreleased `statistics_with_args`; `partition_statistics` stays
deprecated.
- Migrates all operators/callers; updates the 55.0.0 upgrade guide.
## Are these changes tested?
Yes — existing statistics tests now run through
`StatisticsContext::compute`, plus new
unit tests for the cache and the `compute_statistics` benchmark.
## Are there any user-facing changes?
Yes (public `ExecutionPlan` API): `partition_statistics` deprecated in favor
of
`statistics_from_inputs`; new `StatisticsContext` / `ChildStats`; unreleased
`statistics_with_args` removed. Upgrade guide updated.
---
Reviewer note: `statistics_from_inputs` takes `args: &StatisticsArgs`
(currently just
`partition`) as a non-breaking extension seam rather than a bare `partition`
— happy
to change if preferred (see #22958).
----
Disclaimer: I used AI to assist in the code generation, I have manually
reviewed the output and it matches my intention and understanding.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
