[
https://issues.apache.org/jira/browse/NIFI-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17500065#comment-17500065
]
ASF subversion and git services commented on NIFI-9689:
-------------------------------------------------------
Commit 53a35ae4c999824b1c79c31954d2b9b719efd997 in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=53a35ae ]
NIFI-9689: When checking FlowFile Availability, consider swap queue and trigger
data to be swapped in, since calling poll() will no longer happen if no data is
available
Signed-off-by: Joe Gresock <[email protected]>
This closes #5821.
> Do not schedule processors when all FlowFiles are penalized
> -----------------------------------------------------------
>
> Key: NIFI-9689
> URL: https://issues.apache.org/jira/browse/NIFI-9689
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Major
> Fix For: 1.16.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When all FlowFiles in a given queue are penalized, the destination processor
> is still scheduled to run. As a result, the Processor is triggered, no data
> is available, and it generally returns. However, this means that the UI shows
> millions of tasks completed over a short period of time, when no work was
> actually done.
> Further, it means that we constantly schedule the processor to run as fast as
> it can, which wastes compute resources. Instead, we should detect that the
> head of the queue is penalized (and therefore all FlowFiles in the queue are
> penalized), and treat it the same as we would if the queue were empty, which
> allows the "bored yield duration" to take effect, significantly reducing CPU
> cycles.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
