[ https://issues.apache.org/jira/browse/KAFKA-9867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083647#comment-17083647 ]
Ismael Juma commented on KAFKA-9867: ------------------------------------ Thanks for the report. Have you tried 2.4.1? > Log cleaner throws InvalidOffsetException for some partitions > ------------------------------------------------------------- > > Key: KAFKA-9867 > URL: https://issues.apache.org/jira/browse/KAFKA-9867 > Project: Kafka > Issue Type: Bug > Components: log cleaner > Affects Versions: 2.4.0 > Environment: AWS EC2 with data on EBS volumes. > Reporter: Bradley Peterson > Priority: Major > > About half the partitions for one topic are marked "uncleanable". This is > consistent across broker replicas – if a partition is uncleanable on one > broker, its replicas are also uncleanable. > The log-cleaner log seems to suggest out of order offsets. We've seen corrupt > indexes before, so I removed the indexes from the affected segments and let > Kafka rebuild them, but it hit the same error. > I don't know when the error first occurred because we've restarted the > brokers and rotated logs, but it is possible the broker's crashed at some > point. > How can I repair these partitions? > {noformat} > [2020-04-09 00:14:16,979] INFO Cleaner 0: Cleaning segment 223293473 in log > xxxxx-9 (largest timestamp Wed Nov 13 20:35:57 UTC 2019 > ) into 223293473, retaining deletes. (kafka.log.LogCleaner) > [2020-04-09 00:14:29,486] INFO Cleaner 0: Cleaning segment 226762315 in log > xxxxx-9 (largest timestamp Wed Nov 06 18:17:10 UTC 2019 > ) into 223293473, retaining deletes. (kafka.log.LogCleaner) > [2020-04-09 00:14:29,502] WARN [kafka-log-cleaner-thread-0]: Unexpected > exception thrown when cleaning log Log(/var/lib/kafka/xxxxx > -9). Marking its partition (xxxxx-9) as uncleanable (kafka.log.LogCleaner) > org.apache.kafka.common.errors.InvalidOffsetException: Attempt to append an > offset (226765178) to position 941 no larger than the last offset appended > (228774155) to /var/lib/kafka/xxxxx-9/00000000000223293473.index.cleaned. > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005)