If you kill Kafka with SIGTERM, then naturally it will leave behind corrupt index files. It will safely rebuild them. Don't worry.
The Group Metadata Manager messages are normal -- that's just Kafka starting up. Perhaps the corrupted index log messages could be less scary, though. This is a very common question from new users... -Peter On Wed, Jul 6, 2016 at 06:50 Fabian Kochem < [email protected]> wrote: > Hi, > > this is my first time using a mailing list so please forgive me if I'm > doing something wrong. > > When I start Kafka, I get a ton of messages like this: > > WARN Found a corrupted index file, > /kafka/logs/__consumer_offsets-48/00000000000000000000.index, deleting > and rebuilding index... (kafka.log.Log) > INFO Recovering unflushed segment 0 in log __consumer_offsets-48. > (kafka.log.Log) > INFO Completed load of log __consumer_offsets-48 with log end offset 0 > (kafka.log.Log) > WARN Found a corrupted index file, > /kafka/logs/__consumer_offsets-19/00000000000000000000.index, deleting > and rebuilding index... (kafka.log.Log) > INFO Recovering unflushed segment 0 in log __consumer_offsets-19. > (kafka.log.Log) > INFO Completed load of log __consumer_offsets-19 with log end offset 0 > (kafka.log.Log) > WARN Found a corrupted index file, > /kafka/logs/__consumer_offsets-3/00000000000000000000.index, deleting > and rebuilding index... (kafka.log.Log) > INFO Recovering unflushed segment 0 in log __consumer_offsets-3. > (kafka.log.Log) > INFO Completed load of log __consumer_offsets-3 with log end offset 0 > (kafka.log.Log) > > And after that, there's an equal amount of messages which look like this: > > INFO [Group Metadata Manager on Broker 1001]: Loading offsets and group > metadata from [__consumer_offsets,22] > (kafka.coordinator.GroupMetadataManager) > INFO [Group Metadata Manager on Broker 1001]: Finished loading offsets > from [__consumer_offsets,22] in 11 milliseconds. > (kafka.coordinator.GroupMetadataManager) > INFO [Group Metadata Manager on Broker 1001]: Loading offsets and group > metadata from [__consumer_offsets,25] > (kafka.coordinator.GroupMetadataManager) > INFO [Group Metadata Manager on Broker 1001]: Finished loading offsets > from [__consumer_offsets,25] in 4 milliseconds. > (kafka.coordinator.GroupMetadataManager) > INFO [Group Metadata Manager on Broker 1001]: Loading offsets and group > metadata from [__consumer_offsets,28] > (kafka.coordinator.GroupMetadataManager) > INFO [Group Metadata Manager on Broker 1001]: Finished loading offsets > from [__consumer_offsets,28] in 2 milliseconds. > (kafka.coordinator.GroupMetadataManager) > > Did I misconfigure something? > > I only have one topic with a few consumers, so this seems a bit much. I > shutdown Kafka using SIGTERM. > > Thanks in advance, > Fabian Kochem >
