How about defining an avro union type containing all the schemas you wish to put on this topic (the schemas themselves could be defined independently and then bundled into an "uber-schema" at build time)?
That means any messages you put on the topic must match one of the schemas defined in the union. This would allow you to retain the compatibility features of the schema registry. Cheers, Andy On Mon, Jan 30, 2017 at 1:48 PM, Mike Cargal <mike.cargal....@icloud.com> wrote: > This helps some. W’re planning to write a non-homogeneous set of records > to a single topic (to preserve order). There would be no compatibility > between records of different types. I assume that if I set the schema > compatibility for this subject to “none” this would not be a problem. (can > you confirm?) > > Also of potential concern is deduplication. If I write type R1, R2, R3, > R4, R2, R1, … etc., will I only have 4 resulting schemas in the registry? > I see that it’s using a caching class to access the registry, but this > needs to be across many jobs. > > I suppose I’ll be sorting this out as I test, but any insight ahead of > time is appreciated. > > > On Jan 30, 2017, at 7:13 AM, Gerard Klijs <ger...@openweb.nl> wrote: > > > > Not really, as you can update the schema, and have multiple of them at > the > > same time. By default each schema has to backwards compatible, so you do > > have to exclude the specific topic you use with different schema's. With > > every write, the 'id' of the schema used is also written, so when you > > deserialise the messages, you know which schema to use for which message. > > > > Op zo 29 jan. 2017 om 17:35 schreef Mike Cargal <m...@cargal.net>: > > > >> I was just looking into using KafkaAvroSerializer to produce records to > a > >> Kafka topic. We are interested because the wire format has a reference > to > >> the schema so we don’t have to schema lookup information independently. > >> > >> We plan to keep a single topic that contain records using many different > >> schemas (it’s important to maintain the ordering of these records). > >> > >> In looking at the code, it appears that it registers the schema with the > >> registry with a topic+”-topic” subject. This would seem to imply an > >> assumption that a topic has a single schema associated with it (not many > >> schemas that can vary from record to record). > >> > >> Am I understanding this correctly? It seems like a surprising > constraint. > >
