[
https://issues.apache.org/jira/browse/NIFI-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16775258#comment-16775258
]
ASF subversion and git services commented on NIFI-6068:
-------------------------------------------------------
Commit e26bacf69799fc98ce9b4674551ecb520488a6e6 in nifi's branch
refs/heads/master from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e26bacf ]
NIFI-6068, NIFI-6065: Updated StandardFunnel to avoid looping indefinitely and
instead transfer no more than 10,000 FlowFiles before returning from onTrigger.
Updated Local Ports to behavior in the same way. Updated Root Group Ports so
that instead of blocking for up to 100 milliseconds for an incoming request, it
blocks for up to 1 millisecond and if nothing is available yields for the
'bored yield duration'
Signed-off-by: Brandon Devries <[email protected]>
This closes #3328.
> StandardFunnel looping excessively
> ----------------------------------
>
> Key: NIFI-6068
> URL: https://issues.apache.org/jira/browse/NIFI-6068
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.9.0
> Reporter: Brandon DeVries
> Assignee: Mark Payne
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The StandardFunnel, when triggered, reads up to 100 FlowFiles at a time and
> writes them to the outgoing relationship. It does this inside of a loop,
> with the only exit conditions being:
> # No FlowFiles read (input queue empty)
> # No Available relationships to transfer to
> It does this using a thread from the timer driven thread pool.
> Unfortunately, this means that a flow can be created such that a constant
> trickle of data will prevent processors from running, as all threads are in
> use by funnels which will never let them go.
> To fix this, I'd suggest adding 2 checks. First, if session.get(100) returns
> fewer than 100 FlowFiles, transfer those files and then return. Essentially,
> while more files may subsequently come in, at that point the queue had been
> emptied.
> Second, put a cap on the number of FlowFiles transferred per call to
> onTrigger. While there may still be files to move downstream, after you've
> transferred 100K or so it might be time to give up the thread.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
