[ https://issues.apache.org/jira/browse/KAFKA-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15688448#comment-15688448 ]
Jun Rao commented on KAFKA-1120: -------------------------------- [~wushujames], it seems that the controller did detect that broker came up. [2016-11-22 22:57:43,359] INFO [BrokerChangeListener on Controller 4]: Newly added brokers: 1, deleted brokers: , all live brokers: 1,2,3,4,5 (kafka.controller.ReplicaStateMachine$BrokerChangeListener) The following log entries seem to be caused by broker 1 being controlled shutdown again. [2016-11-22 22:57:50,218] DEBUG [Controller 4]: All shutting down brokers: 1 (kafka.controller.KafkaController) [2016-11-22 22:57:50,218] DEBUG [Controller 4]: Live brokers: 5,2,3,4 (kafka.controller.KafkaController) [2016-11-22 22:58:01,668] DEBUG [Controller 4]: All shutting down brokers: 1 (kafka.controller.KafkaController) [2016-11-22 22:58:01,668] DEBUG [Controller 4]: Live brokers: 5,2,3,4 (kafka.controller.KafkaController) > Controller could miss a broker state change > -------------------------------------------- > > Key: KAFKA-1120 > URL: https://issues.apache.org/jira/browse/KAFKA-1120 > Project: Kafka > Issue Type: Bug > Components: core > Affects Versions: 0.8.1 > Reporter: Jun Rao > > When the controller is in the middle of processing a task (e.g., preferred > leader election, broker change), it holds a controller lock. During this > time, a broker could have de-registered and re-registered itself in ZK. After > the controller finishes processing the current task, it will start processing > the logic in the broker change listener. However, it will see no broker > change and therefore won't do anything to the restarted broker. This broker > will be in a weird state since the controller doesn't inform it to become the > leader of any partition. Yet, the cached metadata in other brokers could > still list that broker as the leader for some partitions. Client requests > routed to that broker will then get a TopicOrPartitionNotExistException. This > broker will continue to be in this bad state until it's restarted again. -- This message was sent by Atlassian JIRA (v6.3.4#6332)