Hi Kafka Dev Team/ Users,
We have high level consumer group consuming from 32 partitions for a
topic. We have been running 48 consumers in this group across multiple
servers. We have kept 16 as back-up consumers, and hoping when the
consumer dies, meaning when Zookeeper does not have an owner for a
particular partition. The back-up consumer will take over. But I do not
see this behavior after an active consumer died, the back-up consumer did
not pick the partitions. Please let us know what I can do to achieve
this. This is very likely scenario when rolling out new code on consumer
side (we will be dong incremental code roll out). Please see the exception
below. We are using version 0.8 for now.
[mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a],
exception during rebalance
org.I0Itec.zkclient.exception.ZkNoNodeException:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
NoNode for
/consumers/mupd_logmon_hb_events/ids/mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a
at org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:47)
at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685)
at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:766)
at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:761)
at kafka.utils.ZkUtils$.readData(Unknown Source)
at kafka.consumer.TopicCount$.constructTopicCount(Unknown Source)
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.kafka$consumer$ZookeeperConsumerConnector$ZKRebalancerListener$$rebalance(Unknown
Source)
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(Unknown
Source)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141)
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(Unknown
Source)
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anon$1.run(Unknown
Source)
*Caused by: org.apache.zookeeper.KeeperException$NoNodeException:
KeeperErrorCode = NoNode for
/consumers/mupd_logmon_hb_events/ids/mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a*
at org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:921)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:950)
at org.I0Itec.zkclient.ZkConnection.readData(ZkConnection.java:103)
at org.I0Itec.zkclient.ZkClient$9.call(ZkClient.java:770)
at org.I0Itec.zkclient.ZkClient$9.call(ZkClient.java:766)
at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
11 Jun 2014 14:12:16,710 ERROR
[mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a_watcher_executor]
(kafka.utils.Logging$class.error:?) -
[mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a], error
during syncedRebalance
kafka.common.ConsumerRebalanceFailedException:
mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a can't
rebalance *after 4 retries*
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(Unknown
Source)
at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anon$1.run(Unknown
Source)
Thanks,
Bhavesh
