[
https://issues.apache.org/jira/browse/NIFI-6065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16774540#comment-16774540
]
Mark Payne commented on NIFI-6065:
----------------------------------
I think the reason behind it was to handle the case where there were large
numbers of small FlowFiles coming in constantly and if we did an "immediate
poll" and then returned, it ended up being CPU intensive without any yield at
all (the 'bored yield duration' does not apply to these ports, I don't think).
So we used a yield to avoid being too aggressive on CPU. But always yielding
resulted in poor performance with those large numbers of FlowFiles. But 100
millis is an excessively large time. I would recommend using 1 millisecond, or
even 10-100 microseconds. This should be a better trade off between thrashing
the CPU and waiting too long to check again.
> Root Group Ports "waiting" too long
> -----------------------------------
>
> Key: NIFI-6065
> URL: https://issues.apache.org/jira/browse/NIFI-6065
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Affects Versions: 1.9.0
> Reporter: Brandon DeVries
> Priority: Major
>
> In NIFI-408, a change was made to StandardRootGroupPort to poll for 100
> millis instead of doing a non-blocking poll, and yielding if there were no
> results. While I don't know the initial motivation for this change
> (thoughts, [~markap14]?), the current implementation is resulting in an
> excessive number of Timer-Driven threads stuck in the TIMED_WAITING state.
> Essentially, if you have a high number of low traffic remote ports, you lose
> a substantial chunk of your timer driven thread pool to simply "waiting".
> My first thought would be to revert the change, possibly even without the
> context.yield. Second would be to drastically reduce the poll timeout... 10
> or 5 millis, although any choice is somewhat arbitrary and flow-dependent.
> EDIT: which probably makes it a candidate for a nifi.property
> Any other thoughts or suggestions would be appreciated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
