Hi Pushkar, Best way to see what metrics are available is to connect to a broker via JConsole to see the exposed mbeans.
You can iterate over them programmatically by using the MBean API. Also recommend chapter 10 of Kafka: The Definitive Guide, it covers the metrics really well. Cheers, Liam Clarke-Hutchinson On Thu, 16 Jul. 2020, 5:05 pm Pushkar Deole, <pdeole2...@gmail.com> wrote: > Thanks Liam... > Few questions: in your pattern the topic parameter is appended pattern: > 'kafka.server<type=BrokerTopicMetrics, name=(.+), > topic=(.+)><>OneMinuteRate' > however the kafka docs doesn't mention that > kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec > > does the topic parameter available in all BrokerTopicMetrics and can the > broker provide that parameter dynamically by iterating through all topics > or selective topics? Where will the logic reside to iterate through > available topics and export metrics for all of them? > > On Thu, Jul 16, 2020 at 10:04 AM Liam Clarke-Hutchinson < > liam.cla...@adscale.co.nz> wrote: > > > Whoops, just spotted a typo - the second $1 in the above snippet should > of > > course be $2. > > > > On Thu, Jul 16, 2020 at 4:33 PM Liam Clarke-Hutchinson < > > liam.cla...@adscale.co.nz> wrote: > > > > > Hi Pushkar, > > > > > > There are broker side metrics for messages in / bytes in / bytes out > per > > > topic per second. I use this jmx_exporter rule to export them: > > > - pattern: 'kafka.server<type=BrokerTopicMetrics, name=(.+), > > > topic=(.+)><>OneMinuteRate' > > > name: kafka_broker_per_topic_$1_one_minute_rate > > > labels: > > > topic: $1 > > > type: GAUGE > > > > > > You can't get the number of messages out per topic from the broker > > > because... I think it's somehow related to batching, or similar, it > > doesn't > > > count messages out, only bytes out. You can, however, get that metric > > from > > > the consumer if you're using the Java Kafka client, there's a per-topic > > > messages consumed per second metric exposed as an MBean. > > > > > > You could either use jmx_exporter to also export that from the client > > app, > > > or if possible, add some code that connects to the mbean inside the JVM > > and > > > then exports it via any pre-existing Prometheus registry. You might > want > > to > > > then use a Prometheus aggregating rule to collate all the per-consumer > > apps > > > into a per-consumer-group metric, unless the per consumer granularity > is > > of > > > interest to you: https://prometheus.io/docs/practices/rules/ > > > > > > Hope that helps, > > > > > > Kind regards, > > > > > > Liam Clarke-Hutchinsons > > > > > > On Thu, 16 Jul. 2020, 3:46 pm Pushkar Deole, <pdeole2...@gmail.com> > > wrote: > > > > > >> Thanks Claudia! For broker level metrics, we are also using same jmx > > >> exporter to export those metrics to prometheus. > > >> Are you fetching any per topic metrics from broker? e.g. messages > > produced > > >> on a certain topic or messages consumed from a certain topic. I am > > mainly > > >> interested in these metrics. > > >> > > >> I read in kafka docs that they are present at producer/consumer, > > however I > > >> am not sure how to fetch them from consumer/producer. > > >> > > >> On Wed, Jul 15, 2020 at 8:32 PM Claudia Kesslau <c.kess...@kasasi.de> > > >> wrote: > > >> > > >> > Hi, > > >> > > > >> > I use https://github.com/prometheus/jmx_exporter for collecting > > broker > > >> > metrics and integrating them into prometheus. > > >> > > > >> > Hope this helps. > > >> > Greetings, > > >> > Claudia > > >> > > > >> > -----Ursprüngliche Nachricht----- > > >> > Von: Pushkar Deole <pdeole2...@gmail.com> > > >> > Gesendet: Mittwoch, 15. Juli 2020 09:07 > > >> > An: users@kafka.apache.org > > >> > Betreff: Re: kafka per topic metrics > > >> > > > >> > We are using prometheus as metrics collection and storage system and > > >> > Grafana for displaying those metrics, so integration with them is > > >> required > > >> > > > >> > On Wed, Jul 15, 2020 at 11:11 AM rohit garg <rohit.gar...@gmail.com > > > > >> > wrote: > > >> > > > >> > > You can try using kafka manager and check it will fullfill most of > > >> > > requirement of yours. > > >> > > > > >> > > Thanks and Regards, > > >> > > Rohit > > >> > > > > >> > > On Wed, Jul 15, 2020, 10:33 Pushkar Deole <pdeole2...@gmail.com> > > >> wrote: > > >> > > > > >> > > > Hi All, > > >> > > > > > >> > > > Any inputs as to how the kafka consumer and producer metrics can > > be > > >> > > hooked > > >> > > > up to a monitoring system such as prometheus ? > > >> > > > > > >> > > > On Tue, Jul 14, 2020 at 4:22 PM Pushkar Deole < > > pdeole2...@gmail.com > > >> > > > >> > > > wrote: > > >> > > > > > >> > > > > i did find these metrics from confluent docs: however how can > i > > >> > > > > get > > >> > > this > > >> > > > > metric ? is it available at kafka broker? > > >> > > > > > > >> > > > > Per-Topic Metrics > > >> > > > > > > >> > > > > MBean: > > >> > > > > > > >> > > > > > >> > > > > kafka.producer:type=producer-topic-metrics,client-id=([-.w]+),topic=([ > > >> > > -.w]+) > > >> > > > > record-send-rateThe average number of records sent per second > > for > > >> > > > > a > > >> > > > topic. > > >> > > > > > > >> > > > > On Tue, Jul 14, 2020 at 3:27 PM Pushkar Deole > > >> > > > > <pdeole2...@gmail.com> > > >> > > > > wrote: > > >> > > > > > > >> > > > >> Hi All, > > >> > > > >> > > >> > > > >> Need some help on kafka metrics, i am interested in certain > > >> > > > >> metrics > > >> > > e.g. > > >> > > > >> i need to know the number of records published on a > particular > > >> > > > >> topic > > >> > > and > > >> > > > >> number of records consumed from that topic by a specific > > consumer > > >> > > > group, i > > >> > > > >> would need a total of these 2 and also average per second for > > >> them. > > >> > > > >> > > >> > > > >> Are those metrics available on kafka brokers and if yes then > > >> > > > >> which are those metrics that would give me above counts? > > >> > > > >> > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > >
