Hi,
I have questions regarding spark structured streaming deployment
recommendation
I have +- 100 kafka topics that can be processed using similar code block.
I am using pyspark 2.2.1
Here is the idea:
TOPIC_LIST = ["topic_a","topic_b"."topic_c"]
stream = {}
for t in TOPIC_LIST:
stream[t]
Hi,
I couldn't find any documentation about avro message deserialization using
pyspark structured streaming. My aim is using confluent schema registry to
get per topic schema then parse the avro messages with it.
I found one but it was using DirectStream approach
https://stackoverflow.com/questio