Hi Kafka experts I'm using kafka 0.9.0. I made sure that Fetch.MinBytes(fetch.min.bytes) is set to be 1 and Fetch.MaxWaitTime (fetch.wait.max.ms) is set to be 250 ms. There are no messages in the topic but client (I tried golang client `sarama-cluster` and `bin/kafka-console-consumer.sh`) keeps fetching and broker responses immediately in 1 ms. This spinning cause huge network and cpu consumption.
More details: 1. No client side error logs or outputs. 2. For empty topic, the broker will wait 250 ms as expected. 3. After a new message produced and consumed normally, the broker will be waiting 250 ms as no more message are produced. But once the broker performs "Deleting index", the broker will again response immediately with empty message. the `server.log` at the time when broker change from waiting 250 ms to responding immediately [2018-10-26 21:14:52,788] INFO [Group Metadata Manager on Broker 1]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager) [2018-10-26 21:15:21,679] INFO Rolled new log segment for 'sync-0' in 1 ms. (kafka.log.Log) [2018-10-26 21:15:21,679] INFO Scheduling log segment 0 for log sync-0 for deletion. (kafka.log.Log) [2018-10-26 21:15:21,680] INFO Rolled new log segment for 'Sync-0' in 0 ms. (kafka.log.Log) [2018-10-26 21:15:21,680] INFO Scheduling log segment 83 for log Sync-0 for deletion. (kafka.log.Log) [2018-10-26 21:16:21,680] INFO Deleting segment 0 from log sync-0. (kafka.log.Log) [2018-10-26 21:16:21,680] INFO Deleting index /data/log/kafka-logs/sync-0/00000000000000000000.index.deleted (kafka.log.OffsetIndex) [2018-10-26 21:16:21,680] INFO Deleting segment 83 from log Sync-0. (kafka.log.Log) [2018-10-26 21:16:21,680] INFO Deleting index /data/log/kafka-logs/Sync-0/00000000000000000083.index.deleted (kafka.log.OffsetIndex) Does anyone have ever faced this problem? -- Best Regards. Yang Jueji.
