I did several test. one is with 10 brokers (remote server), one with 3 brokers. (local docker)
both exhibit the same behavior, I was thinking the same but from at least the kafka log, I don't see a rebalance happening. and I am sure my cpu is only used about half. and all broker still running. Nan On Mon, Aug 20, 2018 at 12:18 AM Shantanu Deshmukh <[email protected]> wrote: > How many brokers are there in your cluster? This error usually comes when > one of the brokers who is leader for a partition dies and you are trying to > access it. > > On Fri, Aug 17, 2018 at 9:23 PM Harish K <[email protected]> wrote: > > > Hi, > > I have installed Kafka and created topic but while data ingestion i > get > > some errors as follows.Any help would be really appreciated > > > > > > [2018-08-17 06:12:49,838] WARN Error while fetching metadata with > > correlation id 24 : > > {wikipedia=LEADER_NOT_AVAILABLE}(org.apache.kafka.clients.NetworkClient) > > > > *server log:* > > > > [2018-08-17 06:06:00,719] INFO Creating /controller (is it secure? false) > > (kafka.utils.ZKCheckedEphemeral) > > [2018-08-17 06:06:00,720] INFO Result of znode creation is: OK > > (kafka.utils.ZKCheckedEphemeral) > > [2018-08-17 06:06:00,720] INFO 0 successfully elected as leader > > (kafka.server.ZookeeperLeaderElector) > > [2018-08-17 06:06:00,736] ERROR Error while electing or becoming leader > on > > broker 0 (kafka.server.ZookeeperLeaderElector) > > kafka.common.KafkaException: Can't parse json string: null > > at kafka.utils.Json$.liftedTree1$1(Json.scala:40) > > at kafka.utils.Json$.parseFull(Json.scala:36) > > at > > > > > kafka.utils.ZkUtils$$anonfun$getReplicaAssignmentForTopics$1.apply(ZkUtils.scala:660) > > at > > > > > kafka.utils.ZkUtils$$anonfun$getReplicaAssignmentForTopics$1.apply(ZkUtils.scala:656) > > at > > > > > scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) > > at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) > > at kafka.utils.ZkUtils.getReplicaAssignmentForTopics(ZkUtils.scala:656) > > at > > > > > kafka.controller.KafkaController.initializeControllerContext(KafkaController.scala:742) > > at > > > > > kafka.controller.KafkaController.onControllerFailover(KafkaController.scala:333) > > at > > > > > kafka.controller.KafkaController$$anonfun$1.apply$mcV$sp(KafkaController.scala:160) > > at > > > kafka.server.ZookeeperLeaderElector.elect(ZookeeperLeaderElector.scala:85) > > at > > > > > kafka.server.ZookeeperLeaderElector$LeaderChangeListener$$anonfun$handleDataDeleted$1.apply$mcZ$sp(ZookeeperLeaderElector.scala:154) > > at > > > > > kafka.server.ZookeeperLeaderElector$LeaderChangeListener$$anonfun$handleDataDeleted$1.apply(ZookeeperLeaderElector.scala:154) > > at > > > > > kafka.server.ZookeeperLeaderElector$LeaderChangeListener$$anonfun$handleDataDeleted$1.apply(ZookeeperLeaderElector.scala:154) > > at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:213) > > at > > > > > kafka.server.ZookeeperLeaderElector$LeaderChangeListener.handleDataDeleted(ZookeeperLeaderElector.scala:153) > > at org.I0Itec.zkclient.ZkClient$9.run(ZkClient.java:825) > > at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:72) > > Caused by: java.lang.NullPointerException > > at > > > > > scala.util.parsing.combinator.lexical.Scanners$Scanner.<init>(Scanners.scala:44) > > at scala.util.parsing.json.JSON$.parseRaw(JSON.scala:51) > > at scala.util.parsing.json.JSON$.parseFull(JSON.scala:65) > > at kafka.utils.Json$.liftedTree1$1(Json.scala:37) > > ... 17 more > > > > > > *Controller Log:* > > > > [2018-08-17 06:05:54,644] INFO [Controller 0]: Controller starting up > > (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,659] INFO [Controller 0]: Broker 0 starting become > > controller state transition (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,661] INFO [Controller 0]: Initialized controller > epoch > > to 2294948 and zk version 2294947 (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,664] INFO [Controller 0]: Controller 0 incremented > > epoch to 2294949 (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,665] DEBUG [Controller 0]: Registering > > IsrChangeNotificationListener (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,705] INFO [Controller 0]: Controller startup > complete > > (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,715] DEBUG [Controller 0]: Controller resigning, > > broker id 0 (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,715] DEBUG [Controller 0]: De-registering > > IsrChangeNotificationListener (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,717] INFO [Partition state machine on Controller 0]: > > Stopped partition state machine (kafka.controller.PartitionStateMachine) > > [2018-08-17 06:05:54,718] INFO [Replica state machine on controller 0]: > > Stopped replica state machine (kafka.controller.ReplicaStateMachine) > > [2018-08-17 06:05:54,718] INFO [Controller 0]: Broker 0 resigned as the > > controller (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,727] INFO [Controller 0]: Broker 0 starting become > > controller state transition (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,728] INFO [Controller 0]: Initialized controller > epoch > > to 2294949 and zk version 2294948 (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,730] INFO [Controller 0]: Controller 0 incremented > > epoch to 2294950 (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,730] DEBUG [Controller 0]: Registering > > IsrChangeNotificationListener (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,777] INFO [Controller 0]: Broker 0 starting become > > controller state transition (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,778] INFO [Controller 0]: Initialized controller > epoch > > to 2294950 and zk version 2294949 (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,784] INFO [Controller 0]: Controller 0 incremented > > epoch to 2294951 (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,791] DEBUG [Controller 0]: Registering > > IsrChangeNotificationListener (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,835] INFO [Controller 0]: Broker 0 starting become > > controller state transition (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,838] INFO [Controller 0]: Initialized controller > epoch > > to 2294951 and zk version 2294950 (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,839] INFO [Controller 0]: Controller 0 incremented > > epoch to 2294952 (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,839] DEBUG [Controller 0]: Registering > > IsrChangeNotificationListener (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,925] INFO [Controller 0]: Broker 0 starting become > > controller state transition (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,926] INFO [Controller 0]: Initialized controller > epoch > > to 2294952 and zk version 2294951 (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,928] INFO [Controller 0]: Controller 0 incremented > > epoch to 2294953 (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,929] DEBUG [Controller 0]: Registering > > IsrChangeNotificationListener (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,965] INFO [Controller 0]: Broker 0 starting become > > controller state transition (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,966] INFO [Controller 0]: Initialized controller > epoch > > to 2294953 and zk version 2294952 (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,967] INFO [Controller 0]: Controller 0 incremented > > epoch to 2294954 (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,968] DEBUG [Controller 0]: Registering > > IsrChangeNotificationListener (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,992] INFO [Controller 0]: Broker 0 starting become > > controller state transition (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,993] INFO [Controller 0]: Initialized controller > epoch > > to 2294954 and zk version 2294953 (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,995] INFO [Controller 0]: Controller 0 incremented > > epoch to 2294955 (kafka.controller.KafkaController) > > [2018-08-17 06:05:54,996] DEBUG [Controller 0]: Registering > > IsrChangeNotificationListener (kafka.controller.KafkaController) > > >
