>If you kill Kafka with SIGTERM, then naturally it will leave behind corrupt index files. It will safely rebuild them. Don't worry.
I think you mean SIGKILL. SIGTERM should be handled normally an won't result in corrupted index files at startup. __ Sam Pegler Production Engineer T. +44(0) 07 562 867 486 <http://www.infectiousmedia.com/> 3-7 Herbal Hill / London / EC1R 5EJ www.infectiousmedia.com This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately, and do not disclose the contents to another person, use it for any purpose, or store, or copy the information in any medium. Please also destroy and delete the message from your computer. On 6 July 2016 at 14:54, Peter Davis <[email protected]> wrote: > If you kill Kafka with SIGTERM, then naturally it will leave behind corrupt > index files. It will safely rebuild them. Don't worry. > > The Group Metadata Manager messages are normal -- that's just Kafka > starting up. > > Perhaps the corrupted index log messages could be less scary, though. This > is a very common question from new users... > > -Peter > > On Wed, Jul 6, 2016 at 06:50 Fabian Kochem < > [email protected]> wrote: > > > Hi, > > > > this is my first time using a mailing list so please forgive me if I'm > > doing something wrong. > > > > When I start Kafka, I get a ton of messages like this: > > > > WARN Found a corrupted index file, > > /kafka/logs/__consumer_offsets-48/00000000000000000000.index, deleting > > and rebuilding index... (kafka.log.Log) > > INFO Recovering unflushed segment 0 in log __consumer_offsets-48. > > (kafka.log.Log) > > INFO Completed load of log __consumer_offsets-48 with log end offset 0 > > (kafka.log.Log) > > WARN Found a corrupted index file, > > /kafka/logs/__consumer_offsets-19/00000000000000000000.index, deleting > > and rebuilding index... (kafka.log.Log) > > INFO Recovering unflushed segment 0 in log __consumer_offsets-19. > > (kafka.log.Log) > > INFO Completed load of log __consumer_offsets-19 with log end offset 0 > > (kafka.log.Log) > > WARN Found a corrupted index file, > > /kafka/logs/__consumer_offsets-3/00000000000000000000.index, deleting > > and rebuilding index... (kafka.log.Log) > > INFO Recovering unflushed segment 0 in log __consumer_offsets-3. > > (kafka.log.Log) > > INFO Completed load of log __consumer_offsets-3 with log end offset 0 > > (kafka.log.Log) > > > > And after that, there's an equal amount of messages which look like this: > > > > INFO [Group Metadata Manager on Broker 1001]: Loading offsets and group > > metadata from [__consumer_offsets,22] > > (kafka.coordinator.GroupMetadataManager) > > INFO [Group Metadata Manager on Broker 1001]: Finished loading offsets > > from [__consumer_offsets,22] in 11 milliseconds. > > (kafka.coordinator.GroupMetadataManager) > > INFO [Group Metadata Manager on Broker 1001]: Loading offsets and group > > metadata from [__consumer_offsets,25] > > (kafka.coordinator.GroupMetadataManager) > > INFO [Group Metadata Manager on Broker 1001]: Finished loading offsets > > from [__consumer_offsets,25] in 4 milliseconds. > > (kafka.coordinator.GroupMetadataManager) > > INFO [Group Metadata Manager on Broker 1001]: Loading offsets and group > > metadata from [__consumer_offsets,28] > > (kafka.coordinator.GroupMetadataManager) > > INFO [Group Metadata Manager on Broker 1001]: Finished loading offsets > > from [__consumer_offsets,28] in 2 milliseconds. > > (kafka.coordinator.GroupMetadataManager) > > > > Did I misconfigure something? > > > > I only have one topic with a few consumers, so this seems a bit much. I > > shutdown Kafka using SIGTERM. > > > > Thanks in advance, > > Fabian Kochem > > >
