On Fri, Jun 05, 2015 at 12:53:00AM -0400, Otis Gospodnetić wrote:
Hi Joel,
On Thu, Jun 4, 2015 at 8:52 PM, Joel Koshy jjkosh...@gmail.com wrote:
Hi Otis,
Yes this is a limitation in the old consumer. i.e., a number of
per-topic/partition mbeans remain even on a rebalance. Those need
On Thu, Jun 4, 2015 at 1:55 PM, Otis Gospodnetić otis.gospodne...@gmail.com
wrote:
Hi,
On Thu, Jun 4, 2015 at 4:26 PM, Scott Reynolds sreyno...@twilio.com
wrote:
I believe the JMX metrics reflect the consumer PRIOR to committing
offsets
to Kafka / Zookeeper. But when you query from the
Hi Otis,
Yes this is a limitation in the old consumer. i.e., a number of
per-topic/partition mbeans remain even on a rebalance. Those need to
be de-registered. So if you stop consuming from some partition after a
rebalance that lag mbean currently remain which is why it remains
flat. This is a
Hi,
On Thu, Jun 4, 2015 at 4:26 PM, Scott Reynolds sreyno...@twilio.com wrote:
I believe the JMX metrics reflect the consumer PRIOR to committing offsets
to Kafka / Zookeeper. But when you query from the command line using the
kafka tools, you are just getting the committed offsets.
Even if
I assume you are looking at a 'MaxLag' metric, which reports the worst case
lag over a set of partitions. Are you consuming multiple partitions, and
maybe one of them is stuck?
On Tue, Jun 2, 2015 at 4:00 PM, Otis Gospodnetic otis.gospodne...@gmail.com
wrote:
Hi,
I've noticed that when we
Hi Jason,
(note: Kafka 0.8.2. GA)
Got some new info below! Could be a Kafka metrics bug
On Thu, Jun 4, 2015 at 2:11 PM, Jason Rosenberg j...@squareup.com wrote:
I assume you are looking at a 'MaxLag' metric, which reports the worst case
lag over a set of partitions.
No, we're looking
Hi,
Here's something potentially useful.
1) Before: https://apps.sematext.com/spm-reports/s/eQ9WhLegW9 - the flat
Lag situation
2) I restarted the consumer whose lag is shown in the above graph
3) After restart: https://apps.sematext.com/spm-reports/s/4YGkcUP9ms - NO
lag at all!?
So that
Hi,
I've noticed that when we restart our Kafka consumers our consumer lag
metric sometimes looks weird.
Here's an example: https://apps.sematext.com/spm-reports/s/0Hq5zNb4hH
You can see lag go up around 15:00, when some consumers were restarted.
The weird thing is that the lag remains flat!
