[ https://issues.apache.org/jira/browse/SPARK-16917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418384#comment-15418384 ]
Alexey Zotov commented on SPARK-16917: -------------------------------------- [~sowen] [~c...@koeninger.org] It really seems to be confusing: 1. Security is supported in new consumer API which is implemented starting from Kafka v0.9. _spark-streaming-kafka-0-8_2.11_ does not support new consumer API. Based on that it does not look compatible with secured Kafka v0.9. 2. _spark-streaming-kafka-0-10_2.11_ works with brokers 0.10 or higher. Based on the above reasoning looks like it is impossible to use Spark Streaming for secured Kafka v0.9. Please, let me know what I have missed in the above reasoning. Thanks! > Spark streaming kafka version compatibility. > --------------------------------------------- > > Key: SPARK-16917 > URL: https://issues.apache.org/jira/browse/SPARK-16917 > Project: Spark > Issue Type: Improvement > Components: Documentation > Affects Versions: 2.0.0 > Reporter: Sudev > Priority: Trivial > Labels: documentation > > It would be nice to have Kafka version compatibility information in the > official documentation. > It's very confusing now. > * If you look at this JIRA[1], it seems like Kafka is supported in Spark > 2.0.0. > * The documentation lists artifact for (Kafka 0.8) > spark-streaming-kafka-0-8_2.11 > Is Kafka 0.9 supported by Spark 2.0.0 ? > Since I'm confused here even after an hours effort googling on the same, I > think someone should help add the compatibility matrix. > [1] https://issues.apache.org/jira/browse/SPARK-12177 -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org