jecsand838 commented on PR #17861: URL: https://github.com/apache/datafusion/pull/17861#issuecomment-3513709976
> Thanks for help. > as you said. my Implementation injects a schema into arrow-avro to perform projection, but doing so causes the metadata to be lost. > As a result, if I provide a schema for projection I cannot read the custom record name; conversely, if I don't provide a schema so I can read the custom record name, I can't perform projection. > I'll try to find another solution. Coming in a little bit late to this. @getChan Basically the way we treat "projection" in `arrow-avro` is as a function of schema resolution. To utilize it (for now), you'll need to use a reader schema as your projection schema and a valid writer schema (which 100% matches the underlying Avro data as written) as your base schema. I think to get this working we'll need to extend `arrow-avro` to include a new `with_projection()` method that's more compatible with DataFusion and correctly builds the correct decoder path under the hood. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
