Github user koeninger commented on the pull request:
https://github.com/apache/spark/pull/3798#issuecomment-71565526
I think as long as offsets are available for advanced users that want them,
relying on checkpointing for the happy path should be ok. Will probably be
some design doc discussion on that shortly.
On Mon, Jan 26, 2015 at 4:29 PM, Hari Shreedharan <[email protected]>
wrote:
> @koeninger <https://github.com/koeninger> - One general question: Since
> each checkpoint has the last offset of each batch - could we not start
> pulling data from Kafka from that offset, rather than having the user
> "save" the offset and restart from there?
>
> â
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/3798#issuecomment-71552262>.
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
