Phoenix500526 opened a new pull request, #27117:
URL: https://github.com/apache/datafusion/pull/27117
## Which issue does this PR close?
Closes #26545.
## Rationale for this change
Filtering a constant column above a global aggregate can return an extra
row. For example, this query should return no rows, but currently returns one
row containing `0`.
```sql
SELECT c
FROM (
SELECT 'all' AS k, COUNT(*) AS c
FROM (VALUES (1)) AS t(a)
) AS s
WHERE k = 'none';
```
Pushing the false predicate below `COUNT(*)` empties its input, but the
aggregate still produces a row. Aggregates with an empty grouping set, such as
`ROLLUP(a)`, have the same problem.
## What changes are included in this PR?
- Keep predicates without column references above aggregates that produce a
row on empty input.
- Share the empty-input check with `PropagateEmptyRelation`.
- Add regression queries for global aggregation and `ROLLUP`, and update a
Substrait snapshot where a redundant `TRUE` scan filter is now eliminated.
Constant folding can still remove a true filter or replace a false-filtered
aggregate with an empty relation. Ordinary grouping-key predicates can still be
pushed below aggregation.
## What is the testing strategy for this PR?
Added two result-based regressions in `push_down_filter_regression.slt`,
both verified to fail before the fix and pass afterward.
Passed locally at this commit
- `cargo fmt --all`
- `cargo clippy --all-targets --all-features -- -D warnings`
- The full extended workspace test suite with
`avro,json,backtrace,extended_tests,recursive_protection,parquet_encryption`,
including all 547 SQL logic test files and 228 Substrait integration tests.
Also checked optimized plans for false and true constant predicates and
ordinary grouping-key filters, and checked query results for bound parameters,
`CUBE`, and explicit empty grouping sets.
The additional `./dev/rust_lint.sh` suite stopped at the circular-dependency
check because the local Cargo configuration replaces crates.io with the `tuna`
registry. The complete lint suite therefore has not passed locally.
## Are there any user-facing changes?
Queries with constant filters above global aggregates or empty grouping sets
now return the correct number of rows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
