adriangb commented on PR #15261: URL: https://github.com/apache/datafusion/pull/15261#issuecomment-2843120846
Looking at how filter pushdown interacts with partition columns I think this will be a huge improvement for that. Currently the partition values get bound when the `FileStream` is created which is _after_ the predicate pushdown is applied. This means that partition values are not available in predicate pushdown, and instead happens upstream in a `FilterExec`. I feel like this change could help with that... but some details are missing: we somehow need to pipe the partition values into the `FileSource` so that it can in turn pass in the info to generate the partition columns on the fly if needed. Or something like that... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
