Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/7185#issuecomment-119372912
You can use the alternate version of createDirectStream (returns
DStream[T]) that allows you to specify messageHandler function
MessageAndMetadata => T . This is applied on every message from Kafka, and you
can encode both the message and its offset as a single Spark/RDD record.
BTW, this question unrelated to this PR best asked in the user-mailing list
or the relevant JIRA.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
