jayzhan211 commented on issue #12010: URL: https://github.com/apache/datafusion/issues/12010#issuecomment-2293399849
> We then union these execs together to make our data source. To do this, their schemas must match and we ensure they match our logical schema > create a new projection to reorder the fields, and compose this with the pushed-down projection. We have to do some manipulation of the file schema and partition columns (presumably some of this is unavoidable, but it seems unnecessary that we the logical schema, a file schema that we pass in, and a file schema found from the file) I'm not familiar with how people interact with Parquet file, thus not fully understand the difficulty about schema matching and why `SchemaAdapter` is not helping. It would be great if there is a minimum example that shows the difficulty of manipulating schema to make it works -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
