Hello and thanks for your interest in contributing to NiFi. If my understanding is correct, PrometheusReportingTask is an implementation of the ReportingTask interface that exposes data available in the ReportingContext and other internal interfaces on the Prometheus Meter Registry and ultimately the /metrics/prometheus endpoint.
Personally, I would be fine with adding additional metrics to the Prometheus metrics endpoint that are not part of a standard ReprotingTask, where it makes sense. I do not think ReportingTask implementations such as PrometheusReportingTask should access information via an http call to the NiFi REST API. Rather, we should expose the proper internal APIs and contextual data to make it available. The approaches I can think of are: - extending the ReportingContext to add the desired fields - adding to the Prometheus meter registry/endpoint metrics that are not available to other reporting tasks in the nidi-prometheus-reporting-task implementation. - adding a controller service for getting a shared instance of the Prometheus Meter Registry in other components, and then adding other components that write to that registry data that is not part of the ReportingContext (and do not make sense to add to the ReportingContext, but would be nice to have via Prometheus scraping) I hope this helps. Kevin > On Feb 9, 2021, at 2:48 AM, Noble Numbat <[email protected]> wrote: > > Hi everyone, > > We have added metrics to the Prometheus metrics endpoint in the API > (/nifi-api/flow/metrics/prometheus) to improve programmatic access to > NiFi metrics for the purpose of monitoring. We’d like to contribute > these back to the project for the benefit of others. Please find the > list of metrics below. > > Before I open a JIRA ticket and pull request, I have some questions to > clarify my understanding and determine what else I will need to add to > the code. > 1. How are the use cases different between the Prometheus metrics > endpoint in the API (/nifi-api/flow/metrics/prometheus) and the > PrometheusReportingTask? I note that the metrics are almost identical > between the two. > 2. Is the intent to keep the metrics in these two endpoints the same? > That is, if we add metrics to the Prometheus metrics endpoint in the > API, are we expected to add these to the PrometheusReportingTask as > well? > 3. If so, one way to get the metrics data into > PrometheusReportingTask.java is to make an API call to > /nifi-api/controller/config. Is that an acceptable way to get metrics > data for max_event_driven_threads and max_timer_driven_threads? > > For context, here are the metrics we’ve added; > nifi_repository_max_bytes{flowfile} > nifi_repository_max_bytes{content} > nifi_repository_max_bytes{provenance} > nifi_repository_used_bytes{flowfile} > nifi_repository_used_bytes{content} > nifi_repository_used_bytes{provenance} > jvm_deadlocked_thread_count > max_event_driven_threads > max_timer_driven_threads > jvm_heap_non_init > jvm_heap_non_committed > jvm_heap_non_max > jvm_heap_non_used > jvm_heap_committed > jvm_heap_init > jvm_heap_max > > thanks
