Hi all, I am using kafka* 0.8.2* broker and a *high level consumer API.* *Topic properties:* Number of partitions = 150 Replication factor = 2 Brokers count in cluster = 4
*High level Consumer properties: * There is only one consumer group for this topic. I am running 150 parallel consumer threads equally distributed among 15 nodes, zookeeper.session.timeout.ms=60000 rebalance.backoff.ms = 60000 zookeeper.sync.time.ms = 10000 auto.commit.interval.ms = 60000 rebalance.max.retries = 10 auto.offset.reset = largest The problem here is I am seeing consumers rebalancing very frequently (less than a minute). I tried to increase the zookeeper sync.time, max.retries and session.timeout but none of them solved the issue. Can anyone help me with this problem? What are the possible causes of consumer rebalancing? None of my consumer programs are crashing, they are long running java programs. Thank you *Chhavi Parasher * SeNIOR SOFTWARE ENGINEER Zapr Media Labs, 1st Floor, 621/B 12th Main Road, HAL II Stage, Indiranagar, Bangalore - 560038 +91-9663444755 chh...@zapr.in www.zapr.in
