> How does consumer know A is the avro class when there could be other classes > like B,C and D denoting different schemas?.
There isn't a good way. One option is to have an avro wrapper that contains type, version and data fields. Then you wrap everything. Another option is to do what Kafka is doing and prepend some sort of fixed length value to all messages that have the schema and version you are using for that message. -Dave -----Original Message----- From: Shajahan, Nishanth [mailto:nshaj...@visa.com] Sent: Thursday, August 17, 2017 11:02 AM To: users@kafka.apache.org Subject: RE: Different Schemas on same Kafka Topic Thanks Dave. We may not want to start using schema registery immediately . We would have java producers and consumers . I might also go with using byte messages but when consumer de serilize how can they map the byte[] to the correct Avro object For example: KafkaConsumer<String,A> consumer = new KafkaConsumer<>(consumerConfig,new StringDeserializer(),new AvroDeserializer<>(A)); How does consumer know A is the avro class when there could be other classes like B,C and D denoting different schemas?. -Nishanth -----Original Message----- From: Tauzell, Dave [mailto:dave.tauz...@surescripts.com] Sent: Thursday, August 17, 2017 8:30 AM To: users@kafka.apache.org Subject: RE: Different Schemas on same Kafka Topic It does. The way it works is that the Avro serializer precedes each message with a two-byte integer that references a schema id in the confluent schema registry. The Avro deserializer looks at this value to determine which schema to de-serialize with. In order for this to work you need to use the java client on both ends and have the schema registry setup. We have some slightly different needs ( including non-java languages) so we are just using byte messages and then have our applications do the serialization and deserialization. -Dave -----Original Message----- From: Shajahan, Nishanth [mailto:nshaj...@visa.com] Sent: Wednesday, August 16, 2017 5:13 PM To: users@kafka.apache.org Subject: Different Schemas on same Kafka Topic Hello, Does kafka support writing different avro record types(very different schema) to the same topic . I guess we would have to write our own avro serializer and de serializer to do this ?. Is there a preferred way to do this ?.It would be great if some one can point me in the right direction. Thanks, Nishanth This e-mail and any files transmitted with it are confidential, may contain sensitive information, and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error, please notify the sender by reply e-mail immediately and destroy all copies of the e-mail and any attachments.
