[
https://issues.apache.org/jira/browse/NIFI-6801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958868#comment-16958868
]
ASF subversion and git services commented on NIFI-6801:
-------------------------------------------------------
Commit 147365285cbe385a85e58371dd9570dc01683936 in nifi's branch
refs/heads/master from Yolanda M. Davis
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=1473652 ]
NIFI-6801 - fix to ensure unique model instance is created for each connection.
Also increased default query interval.
Signed-off-by: Matthew Burgess <[email protected]>
This closes #3838
> Connection analytics models should be distinct for connections
> --------------------------------------------------------------
>
> Key: NIFI-6801
> URL: https://issues.apache.org/jira/browse/NIFI-6801
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.10.0
> Reporter: Yolanda M. Davis
> Assignee: Yolanda M. Davis
> Priority: Major
> Fix For: 1.11.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Connections should have Status Analytics Model instances that are unique for
> each connection such that calculations for the model can be referred to or
> updated when new observations are available. Currently the model appears to
> be shared across connection objects which leads to erroneous results,
> especially when a connection does not have enough observations to update the
> model (since the older values of a previous connection may still be present).
> This behavior can be seen when debugging is enabled for analytics and noting
> differences between a Timer Driven Thread execution of predictions vs the
> NiFi Web Server thread predictions for multiple connections (a flow with one
> connection would not exhibit this issue). The Time Driven thread may not
> retrieve enough connection status observations to refresh the model, and when
> that occurs, it may still contain calculations from a previous model and use
> that for predictions. Observers may see behavior where web server and timer
> driven predictions only have similar or matching predictions for one
> connection but others may be mismatched, such as in the below (where
> connection BBBB has similar prediction between threads but AAAA, and CCCC do
> not):
> {code:java}
> 2019-10-22 08:01:34,539 INFO [NiFi Web Server-23]
> o.a.nifi.reporting.StandardEventAccess >>>> AAAA: predicted time to COUNT
> backpressure = 00:44:13 / 2653416
> 2019-10-22 08:01:34,539 INFO [NiFi Web Server-23]
> o.a.nifi.reporting.StandardEventAccess >>>> AAAA: predicted time to BYTES
> backpressure = 01:09:13 / 4153511
> 2019-10-22 08:01:34,540 INFO [NiFi Web Server-23]
> o.a.nifi.reporting.StandardEventAccess >>>> CCCC: predicted time to COUNT
> backpressure = 00:32:31 / 1951609
> 2019-10-22 08:01:34,540 INFO [NiFi Web Server-23]
> o.a.nifi.reporting.StandardEventAccess >>>> CCCC: predicted time to BYTES
> backpressure = 00:32:28 / 1948504
> 2019-10-22 08:01:34,541 INFO [NiFi Web Server-23]
> o.a.nifi.reporting.StandardEventAccess >>>> BBBB: predicted time to COUNT
> backpressure = 07:29:14 / 26954837
> 2019-10-22 08:01:34,541 INFO [NiFi Web Server-23]
> o.a.nifi.reporting.StandardEventAccess >>>> BBBB: predicted time to BYTES
> backpressure = 05:24:56 / 19496954
> 2019-10-22 08:02:02,767 INFO [Timer-Driven Process Thread-2]
> o.a.nifi.reporting.StandardEventAccess >>>> AAAA: predicted time to COUNT
> backpressure = 04:08:36 / 14916150
> 2019-10-22 08:02:02,767 INFO [Timer-Driven Process Thread-2]
> o.a.nifi.reporting.StandardEventAccess >>>> AAAA: predicted time to BYTES
> backpressure = 04:17:49 / 15469243
> 2019-10-22 08:02:02,769 INFO [Timer-Driven Process Thread-2]
> o.a.nifi.reporting.StandardEventAccess >>>> CCCC: predicted time to COUNT
> backpressure = 01:13:42 / 4422829
> 2019-10-22 08:02:02,769 INFO [Timer-Driven Process Thread-2]
> o.a.nifi.reporting.StandardEventAccess >>>> CCCC: predicted time to BYTES
> backpressure = 00:57:15 / 3435109
> 2019-10-22 08:02:02,770 INFO [Timer-Driven Process Thread-2]
> o.a.nifi.reporting.StandardEventAccess >>>> BBBB: predicted time to COUNT
> backpressure = 07:28:46 / 26926608
> 2019-10-22 08:02:02,770 INFO [Timer-Driven Process Thread-2]
> o.a.nifi.reporting.StandardEventAccess >>>> BBBB: predicted time to BYTES
> backpressure = 05:24:28 / 19468725
> {code}
> A workaround for this is to increase the nifi.analytics.query.interval
> property or decrease the nifi.components.status.snapshot.frequency property
> to ensure obtaining enough observations for a refresh. However an appropriate
> fix is to ensure model instances are unique for each connection and to ensure
> that, by default, all threads will obtain enough observations for predictions.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
