Re: Avro serialization in streaming context

2018-02-23 Thread Niels Basjes
HI, In Avro 1.8.2 we have added a "Message" format specification. In the Java classes there are now extra methods generated like toByteBuffer() and fromByteBuffer() that are intended for exactly the purpose of using Avro as the serialization specification in a streaming context. In fact at bol.com

Re: Avro serialization in streaming context

2018-02-23 Thread Commeau, Gabriel
Great! Will do. Thank you! On 2/22/18, 9:26 AM, "Doug Cutting" wrote: These sound to me like they'd be useful additions. Please file an issue in Jira and then issue a Github pull request against it. https://cwiki.apache.org/confluence/display/AVRO/How+To+Contribute

Re: Avro serialization in streaming context

2018-02-22 Thread Doug Cutting
These sound to me like they'd be useful additions. Please file an issue in Jira and then issue a Github pull request against it. https://cwiki.apache.org/confluence/display/AVRO/How+To+Contribute Thanks! Doug On Wed, Feb 21, 2018 at 10:18 PM, Commeau, Gabriel < gabriel.comm...@gmail.com> wrote

Avro serialization in streaming context

2018-02-21 Thread Commeau, Gabriel
Hi all, I’ve been working for several years now on a streaming data platform, and we’ve been using Avro to serialize the messages that flow through the distributed queue (Kafka/Kinesis). Because the message payload contains just one record or a small batch of Avro records, the serialization me