[
https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488312#comment-15488312
]
Cody Koeninger commented on SPARK-15406:
----------------------------------------
Unless I'm misunderstanding, you answered regarding a source, not a source
provider. How does an end user pass in a parameterized type through this
interface to indicate that their keys are e.g. longs, or MyType? If this is
easy, can you show a code example?
Using reflection off strings for ConsumerStrategy means it must have a zero arg
constructor, which is doable but again awkward compared to a serialized
instance.
If you agree neon isn't going to cut it, are you actually going to agree to
make changes, or is this going to be another 'well we're stuck with it because
binary compatibility'?
> Structured streaming support for consuming from Kafka
> -----------------------------------------------------
>
> Key: SPARK-15406
> URL: https://issues.apache.org/jira/browse/SPARK-15406
> Project: Spark
> Issue Type: New Feature
> Reporter: Cody Koeninger
>
> This is the parent JIRA to track all the work for the building a Kafka source
> for Structured Streaming. Here is the design doc for an initial version of
> the Kafka Source.
> https://docs.google.com/document/d/19t2rWe51x7tq2e5AOfrsM9qb8_m7BRuv9fel9i0PqR8/edit?usp=sharing
> ================== Old description =========================
> Structured streaming doesn't have support for kafka yet. I personally feel
> like time based indexing would make for a much better interface, but it's
> been pushed back to kafka 0.10.1
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-33+-+Add+a+time+based+log+index
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
