[
https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15453324#comment-15453324
]
Cody Koeninger commented on SPARK-15406:
----------------------------------------
If people want to use older versions of kafka, why not use the existing
consumers? I'm not super interested in doing anything more than bug fixes for
out of date consumers.
This is a separate question from whether time indexing is a good feature. I
personally think if you're going for an sql / table like interface, "start from
2016-08-30 02:00' is a lot more usable than 'start from this list of 76 offset
ranges'.
But as I said on the mailing list, besides a different interface, I don't know
what people are really expecting to get out of structured streaming with kafka
(at least before there's a solution for processing time vs event time)
> Structured streaming support for consuming from Kafka
> -----------------------------------------------------
>
> Key: SPARK-15406
> URL: https://issues.apache.org/jira/browse/SPARK-15406
> Project: Spark
> Issue Type: New Feature
> Reporter: Cody Koeninger
>
> Structured streaming doesn't have support for kafka yet. I personally feel
> like time based indexing would make for a much better interface, but it's
> been pushed back to kafka 0.10.1
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-33+-+Add+a+time+based+log+index
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
