askalt opened a new issue, #19905: URL: https://github.com/apache/datafusion/issues/19905
There is a `reset_plan_states` function which is intended to re-use an existing `ExecutionPlan`: https://github.com/apache/datafusion/blob/d90d0746d64bf6e91a81b3ec6954369bd0851bb2/datafusion/physical-plan/src/execution_plan.rs#L1401-L1423 Currently, it does not support dynamic filters and work table of the recursive query, because they are injected into nodes once on the planning stage and then never updated. Actually, to re-use a plan, dynamic filters must be reset. ## Possible implementation - Add `ExecutionPlan::dynamic_filters(...)` method which is intended to return current version of dynamic filters. - Introduce `PushedDynamicFilters` struct to pass into `ExecutionPlan::with_new_state` method. - Modify `reset_plan_states` to use the following algorithm: * Create an empty `PushedDynamicFilters` * Create an option `work_table` (`None` initially) * Transform plan up to bottom * When process a node: reset its state, use `with_new_state(pushed_filters)` and `with_new_state(work_table)` if `work_table` is Some. Then collect new dynamic filters from it and possibly a work table if it is a `RecursiveQueryExec` (successfull downcast). - For each node that polls dynamic filter support `with_new_state` where will be an attempt to down-cast state into `PushedDynamicFilters`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
