jordepic commented on PR #10713: URL: https://github.com/apache/arrow-rs/pull/10713#issuecomment-5350191435
@kinshuk-bb Yes, for raw binary Avro containing multiple back-to-back datums encoded with the same known writer schema, this PR supports decoding them directly into RecordBatches without adding framing. Configure a Decoder with that writer schema, repeatedly call decode_datum on the remaining byte slice, advance by the returned consumed-byte count, and call flush whenever batch_is_full (plus once at the end). Each call decodes exactly one datum and leaves subsequent datums untouched. If by `.avro` you mean a standard Avro object-container file, that is already supported today: ReaderBuilder::build reads the container header/schema and iterates over RecordBatches. The new method is specifically for raw datum streams/messages without an object-container header or per-record framing. I am adding explicit coverage for concatenated raw datums and batch boundaries to make this use case clear. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
