[
https://issues.apache.org/jira/browse/NIFI-16375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-16375:
------------------------------------
Description:
The {{ConsumeKafka}} Processor should be updated to record the following lag
and record metrics for each invocation, supporting topic and partition-level
tracking using the recently-attributes Map of attributes for Counters and
Gauges.
* kafka.bytes.consumed
** Type: Counter
* kafka.records.consumed
** Type: Counter
* kafka.records.consumer.lag
** Type: Gauge
* kafka.records.parsed.errors
** Type: Counter
Although the Kafka MetricsReporter provides some capabilities, it does not
support all of these types of instrumentation details, so implementing the
Counters and Gauges using ProcessSession methods appears to be the best
approach.
was:
The {{ConsumeKafka}} Processor should be updated to record the following lag
and record metrics for each invocation, supporting topic and partition-level
tracking using the recently-attributes Map of attributes for Counters and
Gauges.
* kafka.bytes.consumed
** Type: Counter
* kafka.records.consumed
** Type: Counter
* kafka.records.consumer.log
** Type: Gauge
* kafka.records.parsed.errors
** Type: Counter
Although the Kafka MetricsReporter provides some capabilities, it does not
support all of these types of instrumentation details, so implementing the
Counters and Gauges using ProcessSession methods appears to be the best
approach.
> Add Lag and Records Metrics to ConsumeKafka
> -------------------------------------------
>
> Key: NIFI-16375
> URL: https://issues.apache.org/jira/browse/NIFI-16375
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: David Handermann
> Assignee: David Handermann
> Priority: Major
>
> The {{ConsumeKafka}} Processor should be updated to record the following lag
> and record metrics for each invocation, supporting topic and partition-level
> tracking using the recently-attributes Map of attributes for Counters and
> Gauges.
> * kafka.bytes.consumed
> ** Type: Counter
> * kafka.records.consumed
> ** Type: Counter
> * kafka.records.consumer.lag
> ** Type: Gauge
> * kafka.records.parsed.errors
> ** Type: Counter
> Although the Kafka MetricsReporter provides some capabilities, it does not
> support all of these types of instrumentation details, so implementing the
> Counters and Gauges using ProcessSession methods appears to be the best
> approach.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
