raghavyadav01 opened a new issue, #27452:
URL: https://github.com/apache/datafusion/issues/27452
### Describe the bug
Copies of a plan made with `reset_plan_states` share the `AggregateExec`
MIN/MAX dynamic filter. `AggregateExec::reset_state` resets the filter in place
and then clones the node, so every copy holds the same
`DynamicFilterPhysicalExpr` and the same bounds. A copy that runs after another
one starts from the bound the first one left, and its scan prunes row groups it
still needs.
This is the same sharing that #27142 fixed for TopK in `SortExec`.
Kristofferjoha spotted it for `AggregateExec` on #27130. @Kristofferjoha happy
to leave it to you if you already have something; otherwise I'll take it.
### To Reproduce
A parquet file with `x` = 1..1000 in 10 row groups (`max_row_group_size`
100), and `SELECT max(x) FROM t WHERE x % 2 = 0`:
```rust
let plan = ctx.sql("SELECT max(x) FROM t WHERE x % 2 =
0").await?.create_physical_plan().await?;
let first = reset_plan_states(Arc::clone(&plan))?;
let second = reset_plan_states(Arc::clone(&plan))?;
collect(first, ctx.task_ctx()).await?; // 1000
collect(second, ctx.task_ctx()).await?; // NULL
```
On main (73e0b0339):
| | first | second |
|---|---|---|
| run, `reset_plan_states`, run | 1000 | 1000 |
| two copies prepared, then run one after the other | 1000 | NULL |
| two copies run at the same time (20 runs) | 1000 | 900 in 12 of 20 |
### Expected behavior
Each copy returns 1000.
### Additional context
The `reset_plan_states` docs already list running a copy while another copy
is still executing as unsupported for `AggregateExec`, but the second case
above has nothing running at the same time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]