Kafka Consumer Offsets unavailable during rebalancing

2018-01-30 Thread Wouter Bancken
Hi, I'm trying to write an external tool to monitor consumer lag on Apache Kafka. For this purpose, I'm using the kafka-consumer-groups tool to fetch the consumer offsets. When using this tool, partition assignments seem to be unavailable temporarily during the creation of a new topic even if

Re: Kafka Consumer Offsets unavailable during rebalancing

2018-02-04 Thread Wouter Bancken
Can anyone clarify if this is a bug in Kafka or the expected behavior? Best regards, Wouter On 30 January 2018 at 21:04, Wouter Bancken <wouter.banc...@aca-it.be> wrote: > Hi, > > I'm trying to write an external tool to monitor consumer lag on Apache > Kafka. > > For

Re: Kafka Consumer Offsets unavailable during rebalancing

2018-02-04 Thread Wouter Bancken
log so the lag cannot be > predicted in advance. > > -hans > > > On Feb 4, 2018, at 11:51 AM, Wouter Bancken <wouter.banc...@aca-it.be> > wrote: > > > > Can anyone clarify if this is a bug in Kafka or the expected behavior? > > > > Best

Re: Kafka Consumer Offsets unavailable during rebalancing

2018-02-05 Thread Wouter Bancken
metadata.max.age.ms The period of time in milliseconds after which we > force a refresh of metadata even if we haven't seen any partition > leadership changes to proactively discover any new brokers or partitions > -hans > > > > On Feb 4, 2018, at 2:16 PM, Wouter Bancken <w

Re: Kafka Consumer Offsets unavailable during rebalancing

2018-02-13 Thread Wouter Bancken
I logged https://issues.apache.org/jira/browse/KAFKA-6557 Best regards, Wouter On 13 February 2018 at 12:50, Jan Filipiak <jan.filip...@trivago.com> wrote: > I would encourage you todo so. > I also think its not reasonable behavior > > On 13.02.2018 11:28, Wouter Bancken w

Re: Kafka Consumer Offsets unavailable during rebalancing

2018-02-13 Thread Wouter Bancken
We have upgraded our Kafka version as an attempt to solve this issue. However, the issue is still present in Kafka 1.0.0. Can I log a bug for this in JIRA? Wouter On 5 February 2018 at 09:22, Wouter Bancken <wouter.banc...@aca-it.be> wrote: > The consumers in consumer group 'X' do