[
https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16345963#comment-16345963
]
John Cheng commented on SPARK-18057:
------------------------------------
Apache Kafka is now at version 1.0. For people who want to use Spark streaming
against Kafka brokers on 1.0.0, it is preferable to use the
`org.apache.kafka:kafka-clients:jar:1.0.0` client.
"Most of the discussion on the performance impact of [upgrading to the 0.10.0
message
format|https://kafka.apache.org/0110/documentation.html#upgrade_10_performance_impact]
remains pertinent to the 0.11.0 upgrade. This mainly affects clusters that are
not secured with TLS since "zero-copy" transfer is already not possible in that
case. In order to avoid the cost of down-conversion, you should ensure that
consumer applications are upgraded to the latest 0.11.0 client."
> Update structured streaming kafka from 10.0.1 to 10.2.0
> -------------------------------------------------------
>
> Key: SPARK-18057
> URL: https://issues.apache.org/jira/browse/SPARK-18057
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Reporter: Cody Koeninger
> Priority: Major
>
> There are a couple of relevant KIPs here,
> https://archive.apache.org/dist/kafka/0.10.1.0/RELEASE_NOTES.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
