[
https://issues.apache.org/jira/browse/KAFKA-6266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16745701#comment-16745701
]
Zack Kobza commented on KAFKA-6266:
-----------------------------------
We recently encountered the same issue of a broker continuously logging:
{noformat}
Resetting first dirty offset of __consumer_offsets-33 to log start offset
97771750116 since the checkpointed offset 97760913467 is invalid.
(kafka.log.LogCleanerManager$)
{noformat}
And the broker's log cleaner stuck in a loop reprocessing __consumer_offsets-33
without moving onto other topic-partitions.
I verified the contents of the {{cleaner-offset-checkpoint}} file, used by the
log cleaner manager, and discovered the last cleaned offset in the file is
different than what was reported:
{noformat}
$ grep "__consumer_offsets 33" /data/2/kafka/cleaner-offset-checkpoint
__consumer_offsets 33 97771750116 #>>> offset from log message 97760913467
{noformat}
I found a duplicate entry in another {{cleaner-offset-checkpoint}} file:
{noformat}
$ find /data -name cleaner-offset-checkpoint | xargs grep "__consumer_offsets
33"
/data/2/kafka/cleaner-offset-checkpoint:__consumer_offsets 33 97771750116
/data/3/kafka/cleaner-offset-checkpoint:__consumer_offsets 33 97760913467
{noformat}
I deleted the duplicate entry in {{/data/3/kafka/cleaner-offset-checkpoint}},
because the __consumer_offsets-33 logs are located in {{/data/2/kafka}}, also
updating the entries size value and restarted the broker. No WARNs were logged
and the log cleaner successfully cleaned all consumer offsets partitions.
I do not know how the duplicate entry was created. We also upgraded from Kafka
0.11.0.1 to 1.1.1 recently.
> Kafka 1.0.0 : Repeated occurrence of WARN Resetting first dirty offset of
> __consumer_offsets-xx to log start offset 203569 since the checkpointed
> offset 120955 is invalid. (kafka.log.LogCleanerManager$)
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-6266
> URL: https://issues.apache.org/jira/browse/KAFKA-6266
> Project: Kafka
> Issue Type: Bug
> Components: offset manager
> Affects Versions: 1.0.0, 1.0.1
> Environment: CentOS 7, Apache kafka_2.12-1.0.0
> Reporter: VinayKumar
> Priority: Major
>
> I upgraded Kafka from 0.10.2.1 to 1.0.0 version. From then, I see the below
> warnings in the log.
> I'm seeing these continuously in the log, and want these to be fixed- so
> that they wont repeat. Can someone please help me in fixing the below
> warnings.
> {code}
> WARN Resetting first dirty offset of __consumer_offsets-17 to log start
> offset 3346 since the checkpointed offset 3332 is invalid.
> (kafka.log.LogCleanerManager$)
> WARN Resetting first dirty offset of __consumer_offsets-23 to log start
> offset 4 since the checkpointed offset 1 is invalid.
> (kafka.log.LogCleanerManager$)
> WARN Resetting first dirty offset of __consumer_offsets-19 to log start
> offset 203569 since the checkpointed offset 120955 is invalid.
> (kafka.log.LogCleanerManager$)
> WARN Resetting first dirty offset of __consumer_offsets-35 to log start
> offset 16957 since the checkpointed offset 7 is invalid.
> (kafka.log.LogCleanerManager$)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
