Kontinuation opened a new pull request, #197: URL: https://github.com/apache/sedona-db/pull/197
This PR implements projection pushdown for RecordBatchReaderProvider and adds a regression test. Problem ------- Creating a DataFrame from a once-scannable RecordBatchReader and then selecting a subset of columns failed with: SedonaError: Input field name a does not match with the projection expression b. Cause: the provider ignored the projection indices passed to TableProvider::scan(), so the physical plan schema ([a,b]) did not match the pushed-down logical projection ([b]). Fix ---- * Capture projection in scan() * Build projected schema in RecordBatchReaderExec::new() * Apply projection to each emitted batch (both limited and unlimited execution paths) Test ---- Added test_projection_pushdown ensuring selecting only column b from a two-column RecordBatch works and returns the correct values. This fixes #186. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
