[
https://issues.apache.org/jira/browse/NIFI-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663088#comment-15663088
]
Koji Kawamura commented on NIFI-2751:
-------------------------------------
I was trying to review the PR, but couldn't reproduce the ArithmeticException.
The stack trace Mark posted came from BinFiles, which is a super class of
MergeContent processor, and since MergeContent processor requires input
connections, I couldn't setup a flow as Pierre mentioned above.
The only possibility I think, is a case that MergeContent is scheduled, but
before it polls the incoming flow files, other thread added more flow files
into downstream relationship, and it becomes full:
{code: title=ProcessContext.pollFromSelfLoopsOnly}
private boolean pollFromSelfLoopsOnly() {
if (isTriggerWhenAnyDestinationAvailable()) {
// we can pull from any incoming connection, as long as at least
one downstream connection
// is available for each relationship.
// I.e., we can poll only from self if no relationships are
available
return !Connectables.anyRelationshipAvailable(connectable);
} else {
for (final Connection connection : connectable.getConnections()) {
// A downstream connection is full. We are only allowed to pull
from self-loops.
if (connection.getFlowFileQueue().isFull()) { <<<< HERE?
return true;
}
}
}
return false;
}
{code}
So I tried running MergeContent with Concurrent Tasks set higher (like 4 or
even 32), and let 'merged' connection get full, but ArithmeticException was not
thrown.
The change looks good to me, but since I couldn't reproduce the issue, I
couldn't give it a plus one. I'd like [~markap14] to confirm the fix.
> When a processor pulls a batch of FlowFiles, it keeps pulling from the same
> inbound connection instead of round-robin'ing
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-2751
> URL: https://issues.apache.org/jira/browse/NIFI-2751
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Reporter: Mark Payne
> Assignee: Pierre Villard
> Priority: Blocker
> Labels: beginner, easyfix, framework, newbie
> Fix For: 1.1.0
>
>
> When a Processor calls ProcessSession.get(int) or
> ProcessSession.get(FlowFileFilter), the FlowFiles only come from the first
> inbound connection, unless that connection is empty or doesn't have enough
> FlowFiles to complete the get() call. It should instead round-robin between
> the incoming connections, just as ProcessSession.get() does.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
