Mark Payne created NIFI-10167:
---------------------------------
Summary: Allow tracking of detailed metrics such as CPU Usage by
processors
Key: NIFI-10167
URL: https://issues.apache.org/jira/browse/NIFI-10167
Project: Apache NiFi
Issue Type: New Feature
Components: Core Framework
Reporter: Mark Payne
Assignee: Mark Payne
NiFi offers a great deal of detail on the canvas about how much data was
read/written in the past 5 minutes, number of tasks run, and how much time that
took.
However, a question we are often asked is "which processors are using up all of
my CPU?" From looking at tools like {{top}} we can see that NiFi is using a lot
of CPU. And we can guess, based on the fact that Processor A spent 50 minutes
of Processing time over the past 5 minutes that Processor A is likely using a
lot of resources. But we cannot really know for sure how much CPU it is using
vs. disk/socket I/O waits, etc.
This task will introduce more detailed metrics such as how many seconds of CPU
time a processor has consumed, how much time it has spent reading/writing to
the content repo, etc. These metrics should be exposed via the {{nifi.sh
diagnostics}} command.
We should make clear both how much of a Processor's time is spent using CPU, as
well as what percentage of the total CPU used by NiFi was spent for a given
Processor. This will significantly help understand the resource usage
characteristics of a given component.
However, tracking this may be expensive. So we should provide the ability to
turn this feature on/off and ideally also allow for sampling of metrics and
extrapolating out those numbers so that we can monitor these things only for a
percentage of invocations and then give reasonable estimations based on that.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
