[
https://issues.apache.org/jira/browse/NIFI-16598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18146700#comment-18146700
]
ASF subversion and git services commented on NIFI-16598:
--------------------------------------------------------
Commit 158b8bb25f98fd2b703a74505c766db8f0cbe809 in nifi's branch
refs/heads/main from Wojciech
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=158b8bb25f9 ]
NIFI-16598: Return retried FlowFiles to the active queue when swapping (#11934)
* NIFI-16598: Return retried FlowFiles to the active queue when swapping.
Relationship retry used putAll, which parks the FlowFile on the swap queue
behind later active FlowFiles. Session rollback already uses putToActive
(NIFI-16468); retries need the same path so YIELD_PROCESSOR keeps FIFO order.
> Relationship retry can put FlowFile back on swap queue, changing queue order
> ----------------------------------------------------------------------------
>
> Key: NIFI-16598
> URL: https://issues.apache.org/jira/browse/NIFI-16598
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Reporter: Wojciech Kochman
> Assignee: Wojciech Kochman
> Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When a Processor routes a FlowFile to a retried relationship (for example
> failure with YIELD_PROCESSOR), StandardProcessSession returns the FlowFile to
> its original queue during commit. That requeue used FlowFileQueue.put() /
> putAll(). If the source queue is in swap mode, SwappablePriorityQueue appends
> the FlowFile to the swap queue instead of the active queue.
> The FlowFile was at the front of the active queue when it was polled, so it
> now sits behind every remaining active FlowFile. Later FlowFiles are
> processed first. This breaks ordering for any prioritizer, most noticeably
> First In First Out.
> Session rollback already avoids this path via FlowFileQueue.putToActive()
> (NIFI-16468). Relationship retry still went through the normal commit
> enqueue, so the same swap-mode reordering remains.
> This can be reproduced with a queue whose swap threshold is 10,000. Queue
> 10,001 FlowFiles, process the first one, and transfer it to a relationship
> configured for retry with YIELD_PROCESSOR. The next pull returns the second
> FlowFile instead of the first.
> To address this, use putToActive() from StandardProcessSession when
> committing a FlowFile whose destination is the same queue it was polled from
> (the retry path).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)