[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15922715#comment-15922715
]
Michael Armbrust commented on SPARK-18057:
------------------------------------------
So to summarize, it'll be unfortunate if Kafka breaks binary compatibility and
people are relying on the libraries that are bundled with Spark. I don't think
that should stop us from upgrading though, especially since we do provide
binary compatible APIs for reading/writing from kafka and API protocol
compatibility is no longer an issue. If enough users complain, we should
consider shading our dependency on Kafka.
Given that, as soon as there is a release that fixes [KAFKA-4879], I think we
should upgrade (assuming no other regressions). We should probably do the same
for DStreams, unless there are objections there based on the Kafka library
binary compatibility problems (I'm not sure if its more likely for applications
there to interact directly with the kafka library).
> Update structured streaming kafka from 10.0.1 to 10.2.0
> -------------------------------------------------------
>
> Key: SPARK-18057
> URL: https://issues.apache.org/jira/browse/SPARK-18057
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Reporter: Cody Koeninger
>
> There are a couple of relevant KIPs here,
> https://archive.apache.org/dist/kafka/0.10.1.0/RELEASE_NOTES.html
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
