[
https://issues.apache.org/jira/browse/NIFI-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne reopened NIFI-9689:
------------------------------
Re-opening as I encountered an issue where when all data is swapped out of a
queue, the connection may not be triggered to swap data back in, which can
result in the destination processor not running
> Do not schedule processors when all FlowFiles are penalized
> -----------------------------------------------------------
>
> Key: NIFI-9689
> URL: https://issues.apache.org/jira/browse/NIFI-9689
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Major
> Fix For: 1.16.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When all FlowFiles in a given queue are penalized, the destination processor
> is still scheduled to run. As a result, the Processor is triggered, no data
> is available, and it generally returns. However, this means that the UI shows
> millions of tasks completed over a short period of time, when no work was
> actually done.
> Further, it means that we constantly schedule the processor to run as fast as
> it can, which wastes compute resources. Instead, we should detect that the
> head of the queue is penalized (and therefore all FlowFiles in the queue are
> penalized), and treat it the same as we would if the queue were empty, which
> allows the "bored yield duration" to take effect, significantly reducing CPU
> cycles.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
