adriangb opened a new pull request, #25416: URL: https://github.com/apache/datafusion/pull/25416
## Which issue does this PR close? - Closes https://github.com/apache/datafusion/issues/25415. ## Rationale for this change A filter on a volatile `GROUP BY` key returns rows that do not satisfy the filter: ```sql CREATE TABLE v AS SELECT value AS a FROM generate_series(1, 10000); SELECT k, c FROM (SELECT random() < 0.5 AS k, count(*) AS c FROM v GROUP BY random() < 0.5) WHERE k; ``` ``` +-------+------+ | k | c | +-------+------+ | false | 2525 | <-- does not satisfy `WHERE k` | true | 2494 | +-------+------+ ``` `PushDownFilter` moves the predicate below the aggregate and replaces `k` with `random() < 0.5`. The filter then calculates `random()` again, and it gets a different value than the one that the aggregate uses for grouping. PostgreSQL keeps the filter above the aggregate and returns only the `true` row. ## What changes are included in this PR? In the `LogicalPlan::Aggregate` branch of `PushDownFilter`, a predicate is pushed below the aggregate only if it references non-volatile group keys. Predicates on volatile group keys stay above the aggregate. The `Projection` branch already does the same for volatile expressions. ## What is the testing strategy for this PR? - Unit test `test_filter_on_volatile_group_key_not_pushed_below_aggregate`: a non-volatile key predicate is still pushed, and the volatile key predicate stays above. The test fails without the fix. - `push_down_filter_regression.slt`: `EXPLAIN` plus three result checks (`WHERE k`, `WHERE NOT k` with `GROUP BY k`, and `WHERE k OR NOT k`, which must count all 10000 rows). ## Are there any user-facing changes? Queries that filter on a volatile grouping key now return correct results. Plans for non-volatile keys do not change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
