adriangb commented on code in PR #18721:
URL: https://github.com/apache/datafusion/pull/18721#discussion_r2529670863
##########
datafusion/datasource/src/file_scan_config.rs:
##########
@@ -736,8 +739,21 @@ impl FileScanConfig {
}
}
- pub fn projected_stats(&self) -> Statistics {
- let statistics = self.file_source.statistics().unwrap();
+ /// Returns the unprojected table statistics, marking them as inexact if
filters are present.
+ ///
+ /// When filters are pushed down (including pruning predicates and bloom
filters),
+ /// we can't guarantee the statistics are exact because we don't know how
many
+ /// rows will be filtered out.
+ pub fn statistics(&self) -> Statistics {
+ if self.file_source.filter().is_some() {
+ self.statistics.clone().to_inexact()
+ } else {
+ self.statistics.clone()
+ }
+ }
Review Comment:
Now we don't have to rely on each FileSource to remember to do this 😄
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
