Re: Reading Avro messages from Kafka using Structured Streaming in Spark 2.1

2017-05-12 Thread Michael Armbrust
I believe that Avro/Kafka messages have a few bytes at the beginning of the message to denote which schema is being used. Have you tried using the KafkaAvroDecoder inside of the map instead? On Fri, May 12, 2017 at 9:26 AM, Revin Chalil wrote: > Just following up on this;

RE: Reading Avro messages from Kafka using Structured Streaming in Spark 2.1

2017-05-12 Thread Revin Chalil
Just following up on this; would appreciate any responses on this. Thanks. From: Revin Chalil [mailto:rcha...@expedia.com] Sent: Wednesday, May 10, 2017 11:21 PM To: user@spark.apache.org Subject: Reading Avro messages from Kafka using Structured Streaming in Spark 2.1 I am trying to convert