[
https://issues.apache.org/jira/browse/NIFI-6279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brian Ghigiarelli updated NIFI-6279:
------------------------------------
Description:
When MergeContent is set to Bin-Packing while using a Correlation Attribute
with a Minimum Number of Entries per Bin set, merged files are output that do
not meet the Minimum Number of Entries criteria when the number of FlowFiles in
the incoming queue(s) exceeds the Maximum Number of Bins set in the processor.
The processor seems to exhibit the behavior of "kicking out" FlowFiles that
were previously in bins when the number of bins is exhausted, transferring them
to the "merged" relationship.
The expected behavior, it would seem, is that these FlowFiles are not merged or
transferred until the appropriate criteria for each bin is met (i.e., minimum
or maximum number of entries or size).
This seems related to NIFI-379, NIFI-1418, NIFI-1669, and NIFI-3969.
An attached template from Apache NiFi 1.9.2 provides a mechanism for
reproducing this behavior. To observe it, simply start and run the
GenerateFlowFile processor for a few seconds. The Maximum Number of Bins will
quickly become overwhelmed by the number of items in the incoming queue, and
the MergeContent processor will merge and transfer those individual inputs.
was:
When MergeContent is set to Bin-Packing while using a Correlation Attribute
with a Minimum Number of Entries per Bin set, merged files are output that do
not meet the Minimum Number of Entries criteria when the number of FlowFiles in
the incoming queue(s) exceeds the Maximum Number of Bins set in the processor.
The processor seems to exhibit the behavior of "kicking out" FlowFiles that
were previously in bins when the number of bins is exhausted, transferring them
to the "merged" relationship.
The expected behavior, it would seem, is that these FlowFiles are not merged or
transferred until the appropriate criteria for each bin is met (i.e., minimum
or maximum number of entries or size).
This seems related to NIFI-379, NIFI-1418, and NIFI-1669.
An attached template from Apache NiFi 1.9.2 provides a mechanism for
reproducing this behavior. To observe it, simply start and run the
GenerateFlowFile processor for a few seconds. The Maximum Number of Bins will
quickly become overwhelmed by the number of items in the incoming queue, and
the MergeContent processor will merge and transfer those individual inputs.
> MergeContent ignoring Minimum Entries per Bin when incoming Queue size
> exceeds Maximum Number of Bins
> -----------------------------------------------------------------------------------------------------
>
> Key: NIFI-6279
> URL: https://issues.apache.org/jira/browse/NIFI-6279
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.5.0, 1.9.2
> Environment: Reproduced on Mac OS X, CentOS 7, and the latest Docker
> image.
> Reporter: Brian Ghigiarelli
> Priority: Major
> Attachments: MergeContent_Bin_with_Minimum_Entries.xml
>
>
> When MergeContent is set to Bin-Packing while using a Correlation Attribute
> with a Minimum Number of Entries per Bin set, merged files are output that do
> not meet the Minimum Number of Entries criteria when the number of FlowFiles
> in the incoming queue(s) exceeds the Maximum Number of Bins set in the
> processor.
> The processor seems to exhibit the behavior of "kicking out" FlowFiles that
> were previously in bins when the number of bins is exhausted, transferring
> them to the "merged" relationship.
> The expected behavior, it would seem, is that these FlowFiles are not merged
> or transferred until the appropriate criteria for each bin is met (i.e.,
> minimum or maximum number of entries or size).
> This seems related to NIFI-379, NIFI-1418, NIFI-1669, and NIFI-3969.
> An attached template from Apache NiFi 1.9.2 provides a mechanism for
> reproducing this behavior. To observe it, simply start and run the
> GenerateFlowFile processor for a few seconds. The Maximum Number of Bins will
> quickly become overwhelmed by the number of items in the incoming queue, and
> the MergeContent processor will merge and transfer those individual inputs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
