[
https://issues.apache.org/jira/browse/NIFI-1669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207478#comment-15207478
]
Aldrin Piri commented on NIFI-1669:
-----------------------------------
Mostly looks okay, but I am wondering if we should also use the consideration
of the ready bins in
https://github.com/apiri/incubator-nifi/commit/ffe47aaf3efb68f9cb112ddc0862c13116125e53#diff-c5c1d3346cdad127f59b6edda7ac1a44R207.
> MergeContent does not always honor the 'Maximum number of Bins' property
> ------------------------------------------------------------------------
>
> Key: NIFI-1669
> URL: https://issues.apache.org/jira/browse/NIFI-1669
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Critical
> Fix For: 0.6.0
>
> Attachments:
> 0001-NIFI-1669-Ensure-that-we-take-into-account-the-numbe.patch
>
>
> MergeContent exposes a "Maximum number of Bins" property. However, the way in
> which this property is used limits only the maximum number of 'in progress'
> bins. Once a bin has been declared 'ready' (i.e., the FlowFiles have been
> grouped together and are just waiting to be written out), that 'ready bin' is
> not counted toward the maximum number of allowable bins.
> As a result, if we have a very high volume of FlowFiles coming in, we could
> end up with MergeContent creating many bins and buffering potentially
> millions of FlowFiles, exhausting Java Heap and preventing backpressure and
> age-off from kicking in.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
