[
https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15487875#comment-15487875
]
Michael Armbrust commented on SPARK-15406:
------------------------------------------
Hey Cody, thanks for the input and for sharing your prototype! While I think
its fair to say that we don't have the resources to bring python DStreams up to
par with Scala at this time, that is not the case for the Spark SQL APIs. We
already have pretty good parity between Scala and Python here, and I'd really
like to maintain/improve that.
Can you help me understand better what specifically is broken with the stringly
typed interface? Ideally, we'd find ways to work around this, but that said,
I'm not opposed to also having a typed interface in addition to the
DataStreamReader/Writer. We do this in a few exceptional cases already (i.e.
we take an RDD[String] into the JSON data source).
> Structured streaming support for consuming from Kafka
> -----------------------------------------------------
>
> Key: SPARK-15406
> URL: https://issues.apache.org/jira/browse/SPARK-15406
> Project: Spark
> Issue Type: New Feature
> Reporter: Cody Koeninger
>
> Structured streaming doesn't have support for kafka yet. I personally feel
> like time based indexing would make for a much better interface, but it's
> been pushed back to kafka 0.10.1
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-33+-+Add+a+time+based+log+index
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
