Thanks for the replies,
Originally I wanted to have a Kafka topic with multiple schema types, but
Ken's approach sounds like it could work well so I will try out the single
schema approach with a big union type at the root of the schema.
Josh
On Tue, Nov 15, 2016 at 4:44 PM, John McClean
One approach is to have separate Kafka topics per schema, which evolve with
use of a schema registry: https://github.com/confluentinc/schema-registry.
You'd write to the topic with the schema id in metadata. You'd write normal
avro storage files, knowing when to split them based on the changing
: 15 November 2016 12:46
To: user@avro.apache.org
Subject: Re: Alternative to Avro container files for long-term Avro storage
Hi Ken,
Thanks for the reply - that does sound like a good idea, however I don't think
it will work well for me - as I don't have a fixed number of message types. In
my
erely,
>
> Ken Jarrad.
>
>
>
> *From:* Josh [mailto:jof...@gmail.com]
> *Sent:* 15 November 2016 10:24
> *To:* user@avro.apache.org
> *Subject:* Alternative to Avro container files for long-term Avro storage
>
>
>
> Hi all,
>
>
>
> I am using a typical
: Alternative to Avro container files for long-term Avro storage
Hi all,
I am using a typical Avro->Kafka solution where data is serialized to Avro
before it gets written to Kafka and each message is prepended with a schema ID
which can be looked up in my schema repository.
Now, I want to store the d
