friendlymatthew opened a new pull request, #18309: URL: https://github.com/apache/datafusion/pull/18309
## Which issue does this PR close? - Addresses https://github.com/apache/datafusion/issues/14993 - Follows https://github.com/apache/datafusion/pull/18253 ## Rationale for this change This PR delegates the responsibility of projection pushdown evaluation from the `DataSource` trait layer (like `FileScanConfig`) down to the file source implementation itself (the `FileSource` trait level). Previously, `FileScanConfig::try_swapping_with_projection` contained all the logic to determine whether projections can be pushed down: checking for partition columns, aliases, and computing new projection indices. This meant the `DataSource` was responsible for implementation details that _should_ belong to the underlying file format Now, `FileSource::try_pushdown_projections` handles this evaluation. The default impl performs the naive check that mentioned above, and individual file sources like `ParquetSource` can override this method to provide format-specific pushdown behavior `DataSource::try_swapping_with_projection` now returns a tuple containing both a new data source and optional remaining projections, allowing for partial pushdown scenarios where some projection expressions cannot be evaluated by the file source and must remain in a `ProjectionExec` node. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
