[ https://issues.apache.org/jira/browse/KAFKA-9973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
leibo updated KAFKA-9973: ------------------------- Attachment: screenshot-1.png > __consumer_offsets record is invalid lead to log clean failed and > __consumer_offsets grows too big > -------------------------------------------------------------------------------------------------- > > Key: KAFKA-9973 > URL: https://issues.apache.org/jira/browse/KAFKA-9973 > Project: Kafka > Issue Type: Bug > Components: log cleaner > Affects Versions: 2.1.1 > Reporter: leibo > Priority: Major > Attachments: log-cleaner.log.1, screenshot-1.png > > > __consumer_offsets-34 grows to 13GB, and can't be compact successf, the error > log as bellow: > {code:java} > //代码占位符 > [2019-12-30 19:21:41,047] INFO [kafka-log-cleaner-thread-6]: Starting > (kafka.log.LogCleaner) > [2019-12-30 19:21:41,079] INFO [kafka-log-cleaner-thread-7]: Starting > (kafka.log.LogCleaner) > [2019-12-30 19:21:42,825] WARN [kafka-log-cleaner-thread-0]: Unexpected > exception thrown when cleaning log > Log(/home/kafka_2.12-2.1.1/data/kafka-logs/__consumer_offsets-34). Marking > its partition (__consumer_offsets-34) as uncleanable (kafka.log.LogCleaner) > org.apache.kafka.common.record.InvalidRecordException: Found invalid number > of record headers -47 > -- > [2019-12-30 20:23:51,340] INFO [kafka-log-cleaner-thread-5]: Starting > (kafka.log.LogCleaner) > [2019-12-30 20:23:51,361] INFO Cleaner 0: Building offset map for log > __consumer_offsets-34 for 52 segments in offset range [26842714, 73423730). > (kafka.log.LogCleaner) > [2019-12-30 20:23:51,398] INFO [kafka-log-cleaner-thread-6]: Starting > (kafka.log.LogCleaner) > [2019-12-30 20:23:51,493] INFO [kafka-log-cleaner-thread-7]: Starting > (kafka.log.LogCleaner) > [2019-12-30 20:23:59,596] WARN [kafka-log-cleaner-thread-0]: Unexpected > exception thrown when cleaning log > Log(/home/kafka_2.12-2.1.1/data/kafka-logs/__consumer_offsets-34). Marking > its partition (__consumer_offsets-34) as uncleanable (kafka.log.LogCleaner) > org.apache.kafka.common.record.InvalidRecordException: Found invalid number > of record headers -47 > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)