maronavenue opened a new issue, #10776: URL: https://github.com/apache/datafusion/issues/10776
### Describe the bug I have implemented a custom `TableProvider` which sources a test csv that can be found [here](https://github.com/maronavenue/datafusion-example/blob/main/tests/data/example.csv). I know DataFusion readily supports CSV as a format, but I am just using this as starting point in learning custom sources further. I started noticing that when I project a subset of the columns in reverse order, e.g `ctx.sql("SELECT c, b FROM custom_csv")`, it is not able to align the columns properly in the response. I am using `MemoryStream::try_new()` with projections as can be found [here](https://github.com/maronavenue/datafusion-example/blob/main/src/custom_table_provider_csv.rs#L161-L165). I have also tried `project_schema()` too and even manually parsing the `projection` from scan() but it always gives me the column index in the original order so I am unable to use it to deduce the right order in my final projection. Am I missing anything? Let me know if you need further details. ### To Reproduce 1. Clone my repo with the reproducer script: g...@github.com:maronavenue/datafusion-example.git. 2. Build and execute using `cargo run`. 3. Examine the results which I also printed below. The custom table provider detects the following columns: `b`, and `a` instead of `c` and `b` as confirmed by the native CsvExec support counterpart of the same source csv. ```bash +---+---+ | c | b | +---+---+ | 3 | 2 | | 6 | 5 | | 9 | 8 | +---+---+ projection: Some([1, 2]) +---+---+ | b | a | +---+---+ | 3 | 2 | | 6 | 5 | | 9 | 8 | +---+---+ ``` ### Expected behavior The expected behavior should be the columns that are being projected: ```bash +---+---+ | c | b | +---+---+ | 3 | 2 | | 6 | 5 | | 9 | 8 | +---+---+ projection: Some([1, 2]) +---+---+ | c | b | +---+---+ | 3 | 2 | | 6 | 5 | | 9 | 8 | +---+---+ ``` ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
