alamb commented on PR #8732: URL: https://github.com/apache/arrow-datafusion/pull/8732#issuecomment-1875987861
> May it is better to add some integration test？ But I don't know how check whether bloomfilter works with current code. I check it by the metric I add myself(like https://github.com/apache/arrow-datafusion/compare/main...my-vegetable-has-exploded:arrow-datafusion:metric-sbbf?expand=1) , Should I add it to this branch? I think an integration test as well as the bloom filter metrics would be good. Here are my recommended follow on steps 1. One PR to add the new metrics to distinguish filtering on bloom filters vs statistics 2. One PR with some integration tests to verify bloom filters are actually pruning (maybe following how it is done in https://github.com/apache/arrow-datafusion/blob/1179a76567892b259c88f08243ee01f05c4c3d5c/datafusion/core/tests/parquet/row_group_pruning.rs#L42) Also, I am making a PR to clarify and improve comments on the PruningPredicate here -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
