Re: [Structured Streaming] Deserializing avro messages from kafka source using schema registry

[Michael Armbrust]

This isn't supported yet, but there is on going work at spark-avro
 to enable this use
case.  Stay tuned.

On Fri, Feb 9, 2018 at 3:07 PM, Bram  wrote:

> Hi,
>
> I couldn't find any documentation about avro message deserialization using
> pyspark structured streaming. My aim is using confluent schema registry to
> get per topic schema then parse the avro messages with it.
>
> I found one but it was using DirectStream approach
> https://stackoverflow.com/questions/30339636/spark-
> python-avro-kafka-deserialiser
>
> Can anyone show me how?
>
> Thanks
>
> Regards,
>
> Abraham
>

[Structured Streaming] Deserializing avro messages from kafka source using schema registry

[Bram]

Hi,

I couldn't find any documentation about avro message deserialization using
pyspark structured streaming. My aim is using confluent schema registry to
get per topic schema then parse the avro messages with it.

I found one but it was using DirectStream approach
https://stackoverflow.com/questions/30339636/spark-python-avro-kafka-deserialiser

Can anyone show me how?

Thanks

Regards,

Abraham