alamb opened a new issue, #13224: URL: https://github.com/apache/datafusion/issues/13224
### Is your feature request related to a problem or challenge? @Dandandan pointed out in https://github.com/apache/datafusion/pull/13187/files#r1824330274 that when applying a projection to a filter the resulting `StatisticsCalculation` is not properly updated > I think the global stats (`total_byte_size`) are not correct either, doesn't take into account the reduced number of columns. It should do something similar as `stats_projection` for `ProjectionExec` However, I did not want to try and add that in the bugfix PR https://github.com/apache/datafusion/pull/13187 because: - The `total_byte_size` calculation in filter also needs to take estimated selectivity into account - The calculation of `total_byte_size` in `stats_projection` is also somewhat suspect as it only accounts for "fixed sized" rows but still claims the size is known precisely ### Describe the solution you'd like Account for the projection in the filter statistics calculation somehow ### Describe alternatives you've considered https://github.com/apache/datafusion/blob/ac79ef3442e65f6197c7234da9fad964895b9101/datafusion/physical-plan/src/projection.rs#L261-L260 ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
