Hi, I'm working on integrating some pyspark code with Kafka. We'd like to use SSL/TLS, and so want to use Kafka 0.10. Because structured streaming is still marked alpha, we'd like to use Spark streaming. On this page, however, it indicates that the Kafka 0.10 integration in Spark does not support Python ( https://spark.apache.org/docs/latest/streaming-kafka-integration.html). I've been trying to figure out why, but have not been able to find anything. Is there any particular reason for this?
Thanks, Daniel
