Eric Secules created NIP-45:
-------------------------------
Summary: Add Optional Counter / Gauge Labels to NiFi API
Key: NIP-45
URL: https://issues.apache.org/jira/browse/NIP-45
Project: NiFi Improvement Proposal
Issue Type: Improvement
Reporter: Eric Secules
h1. Add Optional Counter / Gauge Labels to NiFi API
h2. Motivation:
NiFi allows developers to write their own ComponentMetricReporter which can be
used to report existing NiFi counters and gauges to an external service.
Oftentimes the NiFi counters and gauges will end up in prometheus or some other
metrics storage which supports adding key/value attributes to each metric which
allow a users of the metrics storage and reporting layer (Grafana for example)
to create dashboards that aggregate based on these attributes.
Currently there is a usability gap if a Processor developer wants to record a
counter or gauge with one more more attributes related to the domain of the
processor. They would need to encode them in the counter or gauge name and then
write a companion ComponentMetricReporter which understands how to parse the
base metrics name and attributes out.
Concretely, the ConsumeKafka processor has several counters it records per
topic and best practices for reporting metics would dictate that the metric
name would be something like `consumekafka_records_acknowledged` with an
attribute for `topic` but instead the counter's name according to the UI is
`Records Acknowledged for <topic>`. Without framework support for metric
attributes it is impossible to develop a generic ComponentMetricReporter for a
metrics sink that accurately reports all metric attributes from all processors.
h2. Scope
Limited to adding new implementations of `recordGauge` and `adjustCounter` and
adding the attribute map to `GaugeRecord` and `CounterRecord` records
h2. Compatibility
Fully backwards compatible.
h2. Description
We will add 4-argument variants of `recordGauge` and `adjustCounter` which take
a `Map<String, String>`
`recordGauge` and `adjustCounter` with 3 arguments will simply call their 4
argument counterpart with an empty map.
The attribute map will feed into the `GaugeRecord` and `CounterRecord` records
where ComponentMetricReporters can use it.
h2. Verification
Add test coverage in `ComponentMetricReporterIT` for updating metrics that have
attributes. The test should validate that different attribute values are
correctly recorded.
h2. Alternatives
A workaround exists if a developer is willing to write a bespoke
`ComponentMetricsReporter` which parses the attribute key/value pairs out of
the metric names.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
