[
https://issues.apache.org/jira/browse/KAFKA-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13718044#comment-13718044
]
Swapnil Ghike commented on KAFKA-987:
-------------------------------------
I think that any call to createMessageStreams will trigger a rebalance, that
will fill up the topicregistry and the checkpointing of offsets will start
regardless of whether new messages are being consumed or not. Hence, we should
probably update the cached checkpointedOffsets map in addPartitionTopicInfo().
May be I have missed something?
> Avoid checkpointing offsets in Kafka consumer that have not changed since the
> last commit
> -----------------------------------------------------------------------------------------
>
> Key: KAFKA-987
> URL: https://issues.apache.org/jira/browse/KAFKA-987
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Swapnil Ghike
> Assignee: Swapnil Ghike
> Labels: improvement
> Fix For: 0.8
>
> Attachments: kafka-987.patch, kafka-987-v2.patch
>
>
> We need to fix the Kafka zookeeper consumer to avoid checkpointing offsets
> that have not changed since the last offset commit. This will help reduce the
> write load on zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
