[
https://issues.apache.org/jira/browse/KAFKA-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13831993#comment-13831993
]
Guozhang Wang commented on KAFKA-1144:
--------------------------------------
Imran, regarding the client rewrite, your proposal is also discussed as part of
the project. Basically, the consumer commitOffset function can be something
like:
void commit(List[String, Int, Long]) // topic, partition-id, offset
The wiki page will be updated soon so that you can take a look by then.
Guozhang
> commitOffsets can be passed the offsets to commit
> -------------------------------------------------
>
> Key: KAFKA-1144
> URL: https://issues.apache.org/jira/browse/KAFKA-1144
> Project: Kafka
> Issue Type: Improvement
> Components: consumer
> Affects Versions: 0.8
> Reporter: Imran Rashid
> Assignee: Neha Narkhede
> Attachments:
> 0001-allow-committing-of-arbitrary-offsets-to-facilitate-.patch,
> 0002-add-protection-against-backward-commits.patch
>
>
> This adds another version of commitOffsets that takes the offsets to commit
> as a parameter.
> Without this change, getting correct user code is very hard. Despite kafka's
> at-least-once guarantees, most user code doesn't actually have that
> guarantee, and is almost certainly wrong if doing batch processing. Getting
> it right requires some very careful synchronization between all consumer
> threads, which is both:
> 1) painful to get right
> 2) slow b/c of the need to stop all workers during a commit.
> This small change simplifies a lot of this. This was discussed extensively on
> the user mailing list, on the thread "are kafka consumer apps guaranteed to
> see msgs at least once?"
> You can also see an example implementation of a user api which makes use of
> this, to get proper at-least-once guarantees by user code, even for batches:
> https://github.com/quantifind/kafka-utils/pull/1
> I'm open to any suggestions on how to add unit tests for this.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
