>From which consumer instance did you see these exceptions? Guozhang
On Thu, Jun 12, 2014 at 4:39 PM, Bhavesh Mistry <mistry.p.bhav...@gmail.com> wrote: > Hi Kafka Dev Team/ Users, > > We have high level consumer group consuming from 32 partitions for a > topic. We have been running 48 consumers in this group across multiple > servers. We have kept 16 as back-up consumers, and hoping when the > consumer dies, meaning when Zookeeper does not have an owner for a > particular partition. The back-up consumer will take over. But I do not > see this behavior after an active consumer died, the back-up consumer did > not pick the partitions. Please let us know what I can do to achieve > this. This is very likely scenario when rolling out new code on consumer > side (we will be dong incremental code roll out). Please see the exception > below. We are using version 0.8 for now. > > [mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a], > exception during rebalance > org.I0Itec.zkclient.exception.ZkNoNodeException: > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = > NoNode for > > /consumers/mupd_logmon_hb_events/ids/mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a > at > org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:47) > at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685) > at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:766) > at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:761) > at kafka.utils.ZkUtils$.readData(Unknown Source) > at kafka.consumer.TopicCount$.constructTopicCount(Unknown Source) > at > > kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.kafka$consumer$ZookeeperConsumerConnector$ZKRebalancerListener$$rebalance(Unknown > Source) > at > > kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(Unknown > Source) > at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141) > at > > kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(Unknown > Source) > at > > kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anon$1.run(Unknown > Source) > *Caused by: org.apache.zookeeper.KeeperException$NoNodeException: > KeeperErrorCode = NoNode for > > /consumers/mupd_logmon_hb_events/ids/mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a* > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:102) > at org.apache.zookeeper.KeeperException.create(KeeperException.java:42) > at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:921) > at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:950) > at org.I0Itec.zkclient.ZkConnection.readData(ZkConnection.java:103) > at org.I0Itec.zkclient.ZkClient$9.call(ZkClient.java:770) > at org.I0Itec.zkclient.ZkClient$9.call(ZkClient.java:766) > at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675) > > > 11 Jun 2014 14:12:16,710 ERROR > > [mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a_watcher_executor] > (kafka.utils.Logging$class.error:?) - > [mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a], error > during syncedRebalance > kafka.common.ConsumerRebalanceFailedException: > mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a can't > rebalance *after 4 retries* > at > > kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(Unknown > Source) > at > > kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anon$1.run(Unknown > Source) > > > > Thanks, > Bhavesh > -- -- Guozhang
