[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15553076#comment-15553076 ]
Ofir Manor commented on SPARK-15406: ------------------------------------ Thanks Cody! I missed the activity in the subtask PR... It is great to see basic support merge, I'll try it out next week. I hope that now that the "basics" are merged, you will be allowed to continue hacking it - personally looking forward to any sort of "assign" support (explicitly providing a startingOffset or providing pre-seeked consumer group) Just curious - given that 0.10.1 with KIP-79 should be released in a couple of weeks, are you planning to add a separate kafka 0.10.1 artifact to allow "seek to timestamp" (maybe an optional startingTimestamp instead of startingOffset)? I think you were passionate about it > Structured streaming support for consuming from Kafka > ----------------------------------------------------- > > Key: SPARK-15406 > URL: https://issues.apache.org/jira/browse/SPARK-15406 > Project: Spark > Issue Type: New Feature > Reporter: Cody Koeninger > > This is the parent JIRA to track all the work for the building a Kafka source > for Structured Streaming. Here is the design doc for an initial version of > the Kafka Source. > https://docs.google.com/document/d/19t2rWe51x7tq2e5AOfrsM9qb8_m7BRuv9fel9i0PqR8/edit?usp=sharing > ================== Old description ========================= > Structured streaming doesn't have support for kafka yet. I personally feel > like time based indexing would make for a much better interface, but it's > been pushed back to kafka 0.10.1 > https://cwiki.apache.org/confluence/display/KAFKA/KIP-33+-+Add+a+time+based+log+index -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org