raghavyadav01 opened a new pull request, #27453:
URL: https://github.com/apache/datafusion/pull/27453

   ## Which issue does this PR close?
   
   - Closes #27452.
   
   ## Rationale for this change
   
   `AggregateExec::reset_state` resets its MIN/MAX dynamic filter in place and 
then clones the node, so every copy made with `reset_plan_states` shares one 
filter and one set of bounds. A copy that runs after another one starts from 
the bound the first one left and prunes row groups it still needs (`max` 
returns NULL, or 900 when copies run at the same time). #27142 fixed the same 
thing for `SortExec`.
   
   ## What changes are included in this PR?
   
   - `reset_state` still clears the old filter and bounds in place, because the 
scan below keeps the filter it was given at planning time and run, reset, run 
(#26054) depends on that. It then gives the copy a new 
`DynamicFilterPhysicalExpr` over the same columns and new bounds, as `SortExec` 
does.
   - Same trade-off as #27142: the copy's scan still holds the planning-time 
filter, so the aggregate dynamic filter no longer prunes anything in a reset 
copy. Results are correct; only that pruning is lost.
   - Updated the `reset_plan_states` limitations. Running the original plan 
after a copy was made is still unsupported, because it updates the filter the 
copy's scan reads. `HashJoinExec` copies still share their filter.
   
   ## Are these changes tested?
   
   Yes, in `core/tests/execution/agg_dynamic_filter_reset.rs`:
   
   - `reset_copies_have_independent_aggregate_dynamic_filters`: two copies run 
one after the other, then 10 rounds of concurrent copies. It fails on main 
(NULL instead of 1000).
   - `rerun_after_reset_keeps_aggregate_result`: run, reset, run gives the same 
result. It already passes on main; it guards the in-place reset this change 
keeps.
   
   I also ran the aggregate unit tests in `physical-plan`, the dynamic filter 
and filter pushdown tests in `core_integration`, the related sqllogictest 
files, and clippy.
   
   ## Are there any user-facing changes?
   
   No API changes. Reset copies of a plan with a MIN/MAX aggregate return 
correct results, and lose that aggregate's dynamic row-group pruning.
   
   I used Claude Code to help reproduce the issue and write the change. I 
reviewed and tested it myself.
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to