[
https://issues.apache.org/jira/browse/NIFI-6599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974268#comment-16974268
]
ASF subversion and git services commented on NIFI-6599:
-------------------------------------------------------
Commit 86cae184ff1cfe8e81f7e1d2d393c39c17bca02a in nifi's branch
refs/heads/master from Koji Kawamura
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=86cae18 ]
NIFI-6599 Fix MergeRecord failure in defragment mode
Signed-off-by: Pierre Villard <[email protected]>
This closes #3678.
> MergeRecord fails if 'fragment.count' attribute equals the number of records
> within a FlowFile where it should wait for remaining FlowFiles
> -------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-6599
> URL: https://issues.apache.org/jira/browse/NIFI-6599
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Reporter: Koji Kawamura
> Assignee: Koji Kawamura
> Priority: Major
> Attachments: NIFI-6599.xml
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> RecordBinManager.createThresholds has following code block:
> {code:java}
> if (MergeRecord.MERGE_STRATEGY_DEFRAGMENT.getValue().equals(mergeStrategy)) {
> fragmentCountAttribute = MergeContent.FRAGMENT_COUNT_ATTRIBUTE;
> if (!StringUtils.isEmpty(flowfile.getAttribute(fragmentCountAttribute))) {
> minRecords =
> Integer.parseInt(flowfile.getAttribute(fragmentCountAttribute));
> }
> } else {
> fragmentCountAttribute = null;
> }
> {code}
> The code uses 'fragment.count' as the minRecords. This is wrong because
> 'fragment.count' represents the number of fragments, i.e. number of FlowFiles
> holding partial record set.
> This causes a FlowFile to be sent 'failure' relationship where it should be
> hold in the incoming connection. For example, when a FlowFile is split into
> two FlowFiles, and each has 2 records in it, 'fragment.count' will be 2. In
> this case, MergeContent thinks the minRecords is 2, where 4 is correct. Then
> the first FlowFile is processed, while the 2nd one hasn't arrived,
> MergeContent misunderstood that the bin reached to the minimum number of
> records. But since there's only one FlowFile, it sends the FlowFile to
> 'failure'.
> The issue can be reproduced by the attached template.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
