Koji Kawamura created NIFI-6599:
-----------------------------------
Summary: MergeRecord fails if 'fragment.count' attribute equals
the number of records within a FlowFile where it should wait for remaining
FlowFiles
Key: NIFI-6599
URL: https://issues.apache.org/jira/browse/NIFI-6599
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Reporter: Koji Kawamura
Assignee: Koji Kawamura
RecordBinManager.createThresholds has following code block:
{code:java}
if (MergeRecord.MERGE_STRATEGY_DEFRAGMENT.getValue().equals(mergeStrategy)) {
fragmentCountAttribute = MergeContent.FRAGMENT_COUNT_ATTRIBUTE;
if (!StringUtils.isEmpty(flowfile.getAttribute(fragmentCountAttribute))) {
minRecords =
Integer.parseInt(flowfile.getAttribute(fragmentCountAttribute));
}
} else {
fragmentCountAttribute = null;
}
{code}
The code uses 'fragment.count' as the minRecords. This is wrong because
'fragment.count' represents the number of fragments, i.e. number of FlowFiles
holding partial record set.
This causes a FlowFile to be sent 'failure' relationship where it should be
hold in the incoming connection. For example, when a FlowFile is split into two
FlowFiles, and each has 2 records in it, 'fragment.count' will be 2. In this
case, MergeContent thinks the minRecords is 2, where 4 is correct. Then the
first FlowFile is processed, while the 2nd one hasn't arrived, MergeContent
misunderstood that the bin reached to the minimum number of records. But since
there's only one FlowFile, it sends the FlowFile to 'failure'.
The issue can be reproduced by the attached template.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
