[
https://issues.apache.org/jira/browse/SPARK-16917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418384#comment-15418384
]
Alexey Zotov edited comment on SPARK-16917 at 8/12/16 5:24 AM:
---------------------------------------------------------------
[~sowen]
[~c...@koeninger.org]
It really seems to be confusing:
1. Security is supported in new consumer API which is implemented starting from
Kafka v0.9. _spark-streaming-kafka-0-8_2.11_ does not support new consumer API.
Based on that it does not look compatible with secured Kafka v0.9.
2. _spark-streaming-kafka-0-10_2.11_ works with brokers 0.10 or higher.
Based on the above reasoning looks like it is impossible to use Spark Streaming
for secured Kafka v0.9. Am I correct? If yes, then it would be great to mention
it somewhere in documentation.
Thanks!
was (Author: azotcsit):
[~sowen]
[~c...@koeninger.org]
It really seems to be confusing:
1. Security is supported in new consumer API which is implemented starting from
Kafka v0.9. _spark-streaming-kafka-0-8_2.11_ does not support new consumer API.
Based on that it does not look compatible with secured Kafka v0.9.
2. _spark-streaming-kafka-0-10_2.11_ works with brokers 0.10 or higher.
Based on the above reasoning looks like it is impossible to use Spark Streaming
for secured Kafka v0.9. Please, let me know what I have missed in the above
reasoning.
Thanks!
> Spark streaming kafka version compatibility.
> ---------------------------------------------
>
> Key: SPARK-16917
> URL: https://issues.apache.org/jira/browse/SPARK-16917
> Project: Spark
> Issue Type: Improvement
> Components: Documentation
> Affects Versions: 2.0.0
> Reporter: Sudev
> Priority: Trivial
> Labels: documentation
>
> It would be nice to have Kafka version compatibility information in the
> official documentation.
> It's very confusing now.
> * If you look at this JIRA[1], it seems like Kafka is supported in Spark
> 2.0.0.
> * The documentation lists artifact for (Kafka 0.8)
> spark-streaming-kafka-0-8_2.11
> Is Kafka 0.9 supported by Spark 2.0.0 ?
> Since I'm confused here even after an hours effort googling on the same, I
> think someone should help add the compatibility matrix.
> [1] https://issues.apache.org/jira/browse/SPARK-12177
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
