allenbenz opened a new pull request, #193: URL: https://github.com/apache/avro-rs/pull/193
My use case is I'm serializing large counts of individual objects to publish to google pubsub in an async setting where the "real" work being done elsewhere is so inexpensive to do that overhead dominates performance concerns. Basically I want to use avro-rs to serialize with minimal allocation/cloning and I must not have headers or markers in the output. Specifics you can skip: > google's pubsub uses a stripped down version of the single record format (v1) - it is without any of the headers or markers, it only wants avro's raw binary encoding. Topics are configured with a the avro schema and the format so they are assumed when trying to publish to that topic. > > The high volume makes it undesirable to construct a SpecificSingleObjectWriter per call as it clones the avro schema. > The async setting prevents sharing the `SpecificSingleObjectWriter` safely as write_ref mutably borrows self. > > My current workaround is using SyncUnsafeCell to share a mutable reference to a SpecificSingleObjectWriter that has already written a header once is not future proof in the slightest. I'm not in love with `write_avro_datum_ref` as the name but it was close to the language of `SpecificSingleObjectWriter.write_ref` and `to_avro_datum`. I had `write_avro_datum_ref` take in a writer rather than make a Vec since that seems more sensible for serialization/allows it to be used in SpecificSingleObjectWriter. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@avro.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
