[
https://issues.apache.org/jira/browse/SPARK-8337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609102#comment-14609102
]
Tathagata Das commented on SPARK-8337:
--------------------------------------
I will let you guys figure out the implementation among yourselves. Let me
chime in and say that I am more concerned about exposing offset ranges, than
supporting access to MessageAndMetadata. So it would be great if you can make a
PR with the first, and then we can focus on the latter in a later iteration.
In that respect, if you are making a PR, please use the other JIRA -
SPARK-8389, which was specifically for the offset ranges. I am reopening that
JIRA and marking it as a sub jira of this one.
> KafkaUtils.createDirectStream for python is lacking API/feature parity with
> the Scala/Java version
> --------------------------------------------------------------------------------------------------
>
> Key: SPARK-8337
> URL: https://issues.apache.org/jira/browse/SPARK-8337
> Project: Spark
> Issue Type: Bug
> Components: PySpark, Streaming
> Affects Versions: 1.4.0
> Reporter: Amit Ramesh
> Priority: Critical
>
> See the following thread for context.
> http://apache-spark-developers-list.1001551.n3.nabble.com/Re-Spark-1-4-Python-API-for-getting-Kafka-offsets-in-direct-mode-tt12714.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
