Thanks a lot to everybody for your suggestions!
In addition to the Consumer lag (on the Consumers side though),
under-replicated partitions, offline partitions, active controller count, I am
also thinking of monitoring the total size of partitions to not exceed some MAX
(like 10G, for example) - to prevent disk out of space issues.
Now, can somebody shed light on my second question :) :"2. what do these
metrics mean: ReplicaManager -> LeaderCount ? and ReplicaManager ->
PartitionCount ?I have three topics created, with one partition each, and
replication = 1, however the values for both of the above attributes is
"53".... So I am not sure what the count '53' means here...."
thanks!MArina
From: Todd Palino <tpal...@gmail.com>
To: "users@kafka.apache.org" <users@kafka.apache.org>
Cc: Marina <ppi...@yahoo.com>
Sent: Tuesday, June 2, 2015 1:29 PM
Subject: Re: Kafka JMS metrics meaning
Under replicated is a must. Offline partitions is also good to monitor. We also
use the active controller metric (it's 1 or 0) in aggregate for a cluster to
know that the controller is running somewhere.
For more general metrics, all topics bytes in and bytes out is good. We also
watch the leader partitions count to know when to do a preferred replica
election. Specifically, we take the ratio of that number to the total partition
count for the broker and keep it near 50%
Most other things, like specific request type time and 99% metrics, we
generally only look at when we are doing performance testing or have a specific
concern.
-Todd
> On Jun 2, 2015, at 1:01 PM, Aditya Auradkar <aaurad...@linkedin.com.INVALID>
> wrote:
>
> Number of underreplicated partitions, total request time are some good bets.
>
> Aditya
>
> ________________________________________
> From: Otis Gospodnetic [otis.gospodne...@gmail.com]
> Sent: Tuesday, June 02, 2015 9:56 AM
> To: users@kafka.apache.org; Marina
> Subject: Re: Kafka JMS metrics meaning
>
> Hi,
>
>> On Tue, Jun 2, 2015 at 12:50 PM, Marina <ppi...@yahoo.com.invalid> wrote:
>>
>> Hi,
>> I have enabled JMX_PORT for KAfka server and am trying to understand some
>> of the metrics that are being exposed. I have two questions:
>> 1. what are the best metrics to monitor to quickly spot unhealthy Kafka
>> cluster?
>
> People loooove looking at consumer lag :)
>
> Otis
> --
> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> Solr & Elasticsearch Support * http://sematext.com/
>
> 2. what do these metrics mean: ReplicaManager -> LeaderCount ? and
>> ReplicaManager -> PartitionCount ?I have three topics created, with one
>> partition each, and replication = 1, however the values for both of the
>> above attributes is "53".... So I am not sure what the count '53' means
>> here....
>> thanksMarina
>>
