When I shutdown my consumer with crtl-c and tries to restart it quickly afterwards, I usually get ConsumerRebalanceFailedException (see below). The application then seems to hang.. or at least I'm sure if it is running any more.. If this exception is thrown, will the consumer the intelligently wait for the rebalancing to complete? and then resume consumption?
I found a page https://cwiki.apache.org/KAFKA/consumer-co-ordinator.htmlthat describes something about Consumer Co-ordinator.. according to this the consumer group remains in this state until the next rebalancing attempt is triggered. But when is it triggered? Could a shutdown hock with a consumer.commitOffsets help? Does the consumer.shutdown implicit commitOffsets? Exception in thread "main" kafka.common.ConsumerRebalanceFailedException: contentItem-consumer-group-1_cphhdfs01node09-1332175323213-e6a3010f can't rebalance after 4 retries at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:467) at kafka.consumer.ZookeeperConsumerConnector.consume(ZookeeperConsumerConnector.scala:204) at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:75) at kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:89) at com.infopaq.research.repository.uima.ContentItemClient.consume(ContentItemClient.java:75) at com.infopaq.research.repository.uima.ContentItemClient.main(ContentItemClient.java:111) Brgds, Peter Thygesen BTW: Great work, very interesting project.
