[
https://issues.apache.org/jira/browse/NIFI-5466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586034#comment-16586034
]
ASF GitHub Bot commented on NIFI-5466:
--------------------------------------
Github user bbende commented on the issue:
https://github.com/apache/nifi/pull/2939
Just started reviewing, there are a bunch of files with missing license
headers...
```
[WARNING] Files with unapproved licenses:
/home/travis/build/apache/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/test/java/org/apache/nifi/controller/status/history/TestMetricRollingBuffer.java
/home/travis/build/apache/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/test/java/org/apache/nifi/controller/repository/metrics/TestSecondPrecisionEventContainer.java
/home/travis/build/apache/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/util/ComponentMetrics.java
/home/travis/build/apache/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/status/history/ComponentStatusHistory.java
/home/travis/build/apache/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/status/history/MetricRollingBuffer.java
/home/travis/build/apache/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/status/history/ComponentDetails.java
/home/travis/build/apache/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/status/history/EmptyStatusSnapshot.java
/home/travis/build/apache/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/status/history/IndexableMetric.java
/home/travis/build/apache/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/metrics/EmptyFlowFileEvent.java
```
> Improve efficiency of RingBufferEventRepository
> -----------------------------------------------
>
> Key: NIFI-5466
> URL: https://issues.apache.org/jira/browse/NIFI-5466
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Major
> Fix For: 1.8.0
>
>
> The implementation of the FlowFileEventRepository (i.e., the
> RingBufferEventRepository) is implemented in such a way that 300 buckets are
> kept for each component. Each bucket holds a set of 16+ stats (bytes read,
> bytes written, flowfiles in, flowfiles out, etc. as well as counters)
> Each time that a user refreshes stats, we calculate the 5-minute window for
> each component by iterating over all 300 buckets, then summing together the
> values of the stats. So, for every 1,000 processors, we have 1000 * 300 * 16
> = 4.8 MMÂ calculations being performed each time that the user refreshes.
> We can improve this significantly. To do this, we can keep a 'running tally'
> for each stat. Every time that we update a bucket in the
> FlowFileEventRepository, we should similarly update the running tally. Each
> time that we 'replace' a bucket in the FlowFileEventRepository, we should
> subtract from the running tally the values in the bucket, in addition to
> adding in the new value. This would result in amortizing the cost of
> calculation over the 5-minute period and means that every time that we get
> the processor stats we could do so by simply performing a 'get' operation
> without any calculations.
> But wait, there's more! The EventSumValue class has an {{add(FlowFileEvent)}}
> method. If the EventSumValue were to simply keep a boolean value indicating
> whether or not it was empty, then when adding to an empty bucket we could
> avoid the calculations all together, and when replacing an empty bucket we
> could avoid the calculation against the running tally as well.
> Similarly, in the VolatileComponentStatusRepository, when we perform a
> 'capture' we should detect any 'empty' stats objects and use a single static
> instance for this, which would significantly reduce the amount of heap that
> is used to store metrics.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
