@Neha, Can you share suggested consumer side GC settings? Consumer side GC settings are not standard since it is a function of your application that embeds the consumer. Your consumer application's memory patterns will dictate your GC settings. Sorry, I know that's not very helpful, but GC tuning is a dark art :-)
On Thu, Nov 13, 2014 at 9:13 AM, Guozhang Wang <wangg...@gmail.com> wrote: > Hey Chen, > > As Neha suggested, typical reason of too many rebalances is that your > consumers kept being timed out from ZK, and you can verify this by checking > in your consumer logs for sth. like "session timeout" entries (these are > not ERROR entries). > > Guozhang > > Guozhang > > On Wed, Nov 12, 2014 at 5:31 PM, Neha Narkhede <neha.narkh...@gmail.com> > wrote: > > > Does this help? > > > > > https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyaretheremanyrebalancesinmyconsumerlog > > ? > > > > On Wed, Nov 12, 2014 at 3:53 PM, Chen Wang <chen.apache.s...@gmail.com> > > wrote: > > > > > Hi there, > > > My kafka client is reading a 3 partition topic from kafka with 3 > threads > > > distributed on different machines. I am seeing frequent owner changes > on > > > the topics when running: > > > bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group > > > my_test_group --topic mytopic -zkconnect localhost:2181 > > > > > > The owner kept changing once a while, but I didn't see any exceptions > > > thrown from the consumer side. When checking broker log, its full of > > > INFO Closing socket connection to /IP. (kafka.network.Processor) > > > > > > Is this expected behavior? If so, how can I tell when the leader is > > > imbalanced, and rebalance is triggered? > > > Thanks, > > > Chen > > > > > > > > > -- > -- Guozhang >
