damccorm opened a new issue, #20034: URL: https://github.com/apache/beam/issues/20034
When creating a Pipeline through a KafkaIO object, I want to be able to specify the starting offset of consumption, and when traversing the message later, I can get the offset of the current message for storage in a relational database / NoSQL. This feature is used to implement the exactly-once semantics of spark streaming consumption. In the "Your own data store" section of the following url content, you can find how to achieve exactly-once semantics with spark streaming: [http://spark.apache.org/docs/latest/streaming-kafka-0-10-integration.html](http://spark.apache.org/docs/latest/streaming-kafka-0-10-integration.html) Imported from Jira [BEAM-8934](https://issues.apache.org/jira/browse/BEAM-8934). Original Jira may contain additional context. Reported by: zjfplayer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
