rgehan opened a new issue, #24107:
URL: https://github.com/apache/datafusion/issues/24107
### Describe the bug
Bloom filter pruning codepath evaluates pruning predicates even when no
bloom filters are defined for the row group.
For wide / expensive predicates, this can add visible latency.
### To Reproduce
```sql
-- 1. Generate a Parquet file with 30 columns / 100 row groups and NO bloom
SET datafusion.execution.parquet.bloom_filter_on_write = false;
SET datafusion.execution.parquet.max_row_group_size = 20000;
COPY (
SELECT
(i * 1) % 1000000 AS c0, (i * 2) % 1000000 AS c1, (i * 3) %
1000000 AS c2,
(i * 4) % 1000000 AS c3, (i * 5) % 1000000 AS c4, (i * 6) %
1000000 AS c5,
(i * 7) % 1000000 AS c6, (i * 8) % 1000000 AS c7, (i * 9) %
1000000 AS c8,
(i * 10) % 1000000 AS c9, (i * 11) % 1000000 AS c10, (i * 12) %
1000000 AS c11,
(i * 13) % 1000000 AS c12, (i * 14) % 1000000 AS c13, (i * 15) %
1000000 AS c14,
(i * 16) % 1000000 AS c15, (i * 17) % 1000000 AS c16, (i * 18) %
1000000 AS c17,
(i * 19) % 1000000 AS c18, (i * 20) % 1000000 AS c19, (i * 21) %
1000000 AS c20,
(i * 22) % 1000000 AS c21, (i * 23) % 1000000 AS c22, (i * 24) %
1000000 AS c23,
(i * 25) % 1000000 AS c24, (i * 26) % 1000000 AS c25, (i * 27) %
1000000 AS c26,
(i * 28) % 1000000 AS c27, (i * 29) % 1000000 AS c28, (i * 30) %
1000000 AS c29
FROM generate_series(1, 2000000) t(i)
) TO '/tmp/bloom_repro_wide.parquet' STORED AS PARQUET;
-- 2. Register the file and force bloom-filter pruning on read.
CREATE EXTERNAL TABLE t STORED AS PARQUET LOCATION
'/tmp/bloom_repro_wide.parquet';
SET datafusion.execution.parquet.bloom_filter_on_read = true;
-- 3. Run a wide predicate (all 30 columns) that min/max statistics can't
-- resolve, so every row group reaches the bloom-filter pruning pass --
-- even though the file has no bloom filters at all.
EXPLAIN ANALYZE
SELECT count(*) FROM t
WHERE c0 >= 0 AND c1 >= 0 AND c2 >= 0 AND c3 >= 0 AND c4 >= 0 AND c5 >= 0 AND
c6 >= 0 AND c7 >= 0 AND c8 >= 0 AND c9 >= 0 AND c10 >= 0 AND c11 >= 0
AND
c12 >= 0 AND c13 >= 0 AND c14 >= 0 AND c15 >= 0 AND c16 >= 0 AND c17
>= 0 AND
c18 >= 0 AND c19 >= 0 AND c20 >= 0 AND c21 >= 0 AND c22 >= 0 AND c23
>= 0 AND
c24 >= 0 AND c25 >= 0 AND c26 >= 0 AND c27 >= 0 AND c28 >= 0 AND c29
>= 0;
```
### Expected behavior
It shouldn't evaluate pruning predicates in the bloom filter codepath, if
there are no bloom filters in the first place.
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
