Hi, We saw the following FATAL error in 2 of our brokers (3 in total, the active controller doesn’t have this) and they crashed in the same time.
[2016-12-16 16:12:47,085] FATAL [ReplicaFetcherThread-0-3], Halting because log truncation is not allowed for topic __consumer_offsets, Current leader 3's latest offset 5910081 is less than replica 1's latest offset 5910082 (kafka.server.ReplicaFetcherThread) Our solution is set topic __consumer_offsets unclean.leader.election.enable=true temporarily, and restart brokers. In this way we potentially lose offsets of some topics. Is there any better solutions? I saw related tickets https://issues.apache.org/jira/browse/KAFKA-3861 <https://issues.apache.org/jira/browse/KAFKA-3861>, https://issues.apache.org/jira/browse/KAFKA-3410 <https://issues.apache.org/jira/browse/KAFKA-3410> and understand why brokers crashed. But we didn’t see any scenarios mentioned in above tickets. Is there any other reason why this happened? There’s no broker restart involved in our case. How can we prevent it from happening? We’re using 0.9.0 with unclean.leader.election.enable=false and min.insync.replicas=2. Thanks, Jun
