[
https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176037#comment-15176037
]
Cody Koeninger commented on SPARK-12177:
----------------------------------------
How is it a huge hassle to keep the known working implementation in a separate
subproject from the new beta consumer? Any of the kafka version churn hassle
would be associated with the new subproject. Work on the existing project
would pretty much be limited to changes in Spark that were (somehow?)
incompatible with it, or bugfixes. I'm not saying that existing code is
bug-free, but we've had 4 point revisions of spark with relatively little
change to it.
The fact that the 0.9 consumer is still considered beta by the Kafka project
and that things are going to change in 0.10 is an argument for keeping the
existing implementation as it is, not an argument for throwing it away
prematurely.
> Update KafkaDStreams to new Kafka 0.9 Consumer API
> --------------------------------------------------
>
> Key: SPARK-12177
> URL: https://issues.apache.org/jira/browse/SPARK-12177
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 1.6.0
> Reporter: Nikita Tarasenko
> Labels: consumer, kafka
>
> Kafka 0.9 already released and it introduce new consumer API that not
> compatible with old one. So, I added new consumer api. I made separate
> classes in package org.apache.spark.streaming.kafka.v09 with changed API. I
> didn't remove old classes for more backward compatibility. User will not need
> to change his old spark applications when he uprgade to new Spark version.
> Please rewiew my changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
