adriangb commented on code in PR #16424:
URL: https://github.com/apache/datafusion/pull/16424#discussion_r2161929117
##########
datafusion/core/tests/parquet/row_group_pruning.rs:
##########
@@ -421,7 +421,7 @@ macro_rules! int_tests {
.with_query(&format!("SELECT * FROM t where i{} in (100)",
$bits))
.with_expected_errors(Some(0))
.with_matched_by_stats(Some(0))
- .with_pruned_by_stats(Some(0))
+ .with_pruned_by_stats(Some(4))
Review Comment:
This is basically reverting the changes made in
https://github.com/apache/datafusion/pull/16447 because we are now not doing
file level pruning if there are only static filters.
@alamb maybe this is an argument to still do the file level pruning even if
there are no dynamic filters. I guess this pruning is _not_ happening anywhere
else. One could argue it should happen right after statistics are collected, in
the TableProvider, but that doesn't seem to be the case and maybe it's better
to just let it happen in `ParquetOpener` (or we could move this pruning up to
the `DataSourceExec` or something so it works for all sources) so that
TableProvider complexity is kept to a minimum.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
