Hi Sahitya,
Try reducing max.partition.fetch.bytes in your consumer.
Then also increase heartbeat.interval.ms, this might help in to delay the
consumer rebalance of your inbound process is taking more time than this
- Abhinav
On Fri, May 13, 2016 at 5:42 AM sahitya agrawal <sahitya2...@gmail.com>
wrote:
> Hi,
>
> I am using new Kafka consumer API ( 0.9.0.0) . I created 100 partitions of
> a topic and started only one consumer to consume. Many of times , In
> consumer logs I see lot of rebalancing activity and no object is consumed
> due to that.
>
> Is this a known issue? Please let me know if some body can help with regard
> to this.
>
> My Consumer config:
> props.put("zookeeper.session.timeout.ms", "10000");
> props.put("rebalance.backoff.ms","10000");
> props.put("zookeeper.sync.time.ms","200");
> props.put("rebalance.max.retries","10");
> props.put("enable.auto.commit", "false");
> props.put("consumer.timeout.ms","20000");
> props.put("auto.offset.reset", "smallest");
>
> Thanks,
> Sahitya
>
