+1 We're on CDH, and it will probably be a while before they support Kafka 0.10. At the same time, we don't use their Spark and we're looking forward to upgrading to 2.0.x and using structured streaming.
I was just going to write our own Kafka Source implementation which uses the existing KafkaRDD but it would be much easier to get buy-in for an official Spark module. Jeremy On Fri, Oct 7, 2016 at 12:41 PM, Michael Armbrust <mich...@databricks.com> wrote: > We recently merged support for Kafak 0.10.0 in Structured Streaming, but > I've been hearing a few people tell me that they are stuck on an older > version of Kafka and cannot upgrade. I'm considering revisiting > SPARK-17344 <https://issues.apache.org/jira/browse/SPARK-17344>, but it > would be good to have more information. > > Could people please vote or comment on the above ticket if a lack of > support for older versions of kafka would block you from trying out > structured streaming? > > Thanks! > > Michael >
