Hi,
we're using Kafka Minion (named KMinion since v2) for this purpose:
https://github.com/cloudhut/kminion
It can export all kinds of metrics for a cluster and consumer groups,
including their lag.
Regards,
Jonas Seydel
On 23.04.21 11:20, Dumitru Nicolae Marasoiu wrote:
Hi, what is
Did you test this out
https://github.com/lightbend/kafka-lag-exporter
?
We are using bash scripting to describe the consumer groups from the
cluster side into file and ingest to splunk for dashboarsing...
Note - Approach of describing from outside the consumer does not give the
'real time' view
Hi, what is suggested way to emit lag into Prometheus? Is there a Kafka
Streams job for it? Is Burrow or Kafka Lag Exporter?
Emitting lag from the consumer is not a great option of course, so looking
at independent jobs that emit lag.
Do you know any? Thank you
Hi Yong Gang,
Idempotence is not guaranteed if you increase max in flight requests.
Because if 5 requests are dispatched, but 2 requests fail and are retried,
the messages in those 2 requests will arrive after the messages in the
other three requests, so ordering will be changed.
Assuming
