Hi Dave, Would like to get a clarity on one thing. If i register more than one schema for a topic, i am providing topic-key, topic-value to the schema registry.
Id is created by schema registry and it will create different version of different schema. Still all schema have same id. Am i right ? If so, all avro messages holds same id. Then how multiple schemas on same topic possble ? Please clarify Thanks, Sreejith On 17-Aug-2017 9:49 pm, "Tauzell, Dave" <[email protected]> wrote: > > How does consumer know A is the avro class when there could be other > classes like B,C and D denoting different schemas?. > > There isn't a good way. One option is to have an avro wrapper that > contains type, version and data fields. Then you wrap everything. > Another option is to do what Kafka is doing and prepend some sort of fixed > length value to all messages that have the schema and version you are using > for that message. > > -Dave > > -----Original Message----- > From: Shajahan, Nishanth [mailto:[email protected]] > Sent: Thursday, August 17, 2017 11:02 AM > To: [email protected] > Subject: RE: Different Schemas on same Kafka Topic > > Thanks Dave. We may not want to start using schema registery immediately . > We would have java producers and consumers . I might also go with using > byte messages but when consumer de serilize how can they map the byte[] > to the correct Avro object For example: > > KafkaConsumer<String,A> consumer = new KafkaConsumer<>(consumerConfig,new > StringDeserializer(),new AvroDeserializer<>(A)); > > How does consumer know A is the avro class when there could be other > classes like B,C and D denoting different schemas?. > > > -Nishanth > > -----Original Message----- > From: Tauzell, Dave [mailto:[email protected]] > Sent: Thursday, August 17, 2017 8:30 AM > To: [email protected] > Subject: RE: Different Schemas on same Kafka Topic > > It does. The way it works is that the Avro serializer precedes each > message with a two-byte integer that references a schema id in the > confluent schema registry. The Avro deserializer looks at this value to > determine which schema to de-serialize with. In order for this to work > you need to use the java client on both ends and have the schema registry > setup. > > We have some slightly different needs ( including non-java languages) so > we are just using byte messages and then have our applications do the > serialization and deserialization. > > -Dave > > -----Original Message----- > From: Shajahan, Nishanth [mailto:[email protected]] > Sent: Wednesday, August 16, 2017 5:13 PM > To: [email protected] > Subject: Different Schemas on same Kafka Topic > > Hello, > > Does kafka support writing different avro record types(very different > schema) to the same topic . I guess we would have to write our own avro > serializer and de serializer to do this ?. Is there a preferred way to do > this ?.It would be great if some one can point me in the right direction. > > Thanks, > Nishanth > > This e-mail and any files transmitted with it are confidential, may > contain sensitive information, and are intended solely for the use of the > individual or entity to whom they are addressed. If you have received this > e-mail in error, please notify the sender by reply e-mail immediately and > destroy all copies of the e-mail and any attachments. > > >
