Dandandan commented on issue #20324: URL: https://github.com/apache/datafusion/issues/20324#issuecomment-3905071439
See https://github.com/apache/datafusion/pull/20160#issuecomment-3905053370 I think approaches to adaptiveness/selectivity tracking also need to work _during_ file scan - otherwise we're evaluating / reading the columns without using the selectivity tracking very much. We probably need to integrate more with the parquet reader (i.e. implement some config / hooks / in arrow-rs) so it knows it can stop decoding the column that will yield only "true" for that predicate and we can also update the predicate order based on new info, etc. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
