getChan commented on PR #17861: URL: https://github.com/apache/datafusion/pull/17861#issuecomment-3677525629
There is a compatibility issue with projection. I'm waiting for a release of arrow-avro that includes the necessary projection features. - DataFusion does not infer the schema from the file when the table schema is explicitly defined. - arrow-avro requires reading the Avro file metadata (avro.schema) to perform projection. - Consequently, projection is problematic when reading Avro tables with explicitly defined schemas. Please let me know if my understanding is incorrect or if there is a workaround. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
