Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/15387
I set auto commit to false, and still recreated the test failure.
That makes sense to me, consumer position should still be getting updated
in memory even if it isn't saved to storage anywhere.
At any rate, there are valid (albeit ill advised in my opinion) reasons to
turn on autocommit, so I'm not sure it matters.
On Fri, Oct 7, 2016 at 4:24 PM, Shixiong Zhu <[email protected]>
wrote:
> but polling ordinarily consumes messages and adjusts position.
>
> Even if enable.auto.commit is false? In the doc, it says automatically
> set as the last committed offset, so I guess setting enable.auto.commit
> to false will prevent positions from updating.
>
> * On each poll, consumer will try to use the last consumed offset as
the starting offset and fetch sequentially. The last
> * consumed offset can be manually set through {@link
#seek(TopicPartition, long)} or automatically set as the last committed
> * offset for the subscribed list of partitions
>
> â
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/spark/pull/15387#issuecomment-252364488>, or
mute
> the thread
>
<https://github.com/notifications/unsubscribe-auth/AAGAB5XiPWUJ07g1tn5RYzNxXA4csbiCks5qxriKgaJpZM4KQlB3>
> .
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
