[
https://issues.apache.org/jira/browse/SPARK-17344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15558474#comment-15558474
]
Cody Koeninger commented on SPARK-17344:
----------------------------------------
I think this is premature until you have a fully operational battlestation, er,
structured stream, that has all the necessary features for 0.10
Regarding the conversation with Michael about possibly using the kafka protocol
directly as a way to work around the differences between 0.8 and 0.10, please
don't consider that. Every kafka consumer implementation I've ever used has
bugs, and we don't need to spend time writing another buggy one.
By contrast, writing a streaming source shim around the existing simple
consumer-based 0.8 spark rdd would be a weekend project, it just wouldn't have
stuff like SSL, dynamic topics, or offset committing.
> Kafka 0.8 support for Structured Streaming
> ------------------------------------------
>
> Key: SPARK-17344
> URL: https://issues.apache.org/jira/browse/SPARK-17344
> Project: Spark
> Issue Type: Sub-task
> Components: Streaming
> Reporter: Frederick Reiss
>
> Design and implement Kafka 0.8-based sources and sinks for Structured
> Streaming.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
