Wojciech Kochman created NIFI-16598:
---------------------------------------
Summary: Relationship retry can put FlowFile back on swap queue,
changing queue order
Key: NIFI-16598
URL: https://issues.apache.org/jira/browse/NIFI-16598
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Wojciech Kochman
When a Processor routes a FlowFile to a retried relationship (for example
failure with YIELD_PROCESSOR), StandardProcessSession returns the FlowFile to
its original queue during commit. That requeue used FlowFileQueue.put() /
putAll(). If the source queue is in swap mode, SwappablePriorityQueue appends
the FlowFile to the swap queue instead of the active queue.
The FlowFile was at the front of the active queue when it was polled, so it now
sits behind every remaining active FlowFile. Later FlowFiles are processed
first. This breaks ordering for any prioritizer, most noticeably First In First
Out.
Session rollback already avoids this path via FlowFileQueue.putToActive()
(NIFI-16468). Relationship retry still went through the normal commit enqueue,
so the same swap-mode reordering remains.
This can be reproduced with a queue whose swap threshold is 10,000. Queue
10,001 FlowFiles, process the first one, and transfer it to a relationship
configured for retry with YIELD_PROCESSOR. The next pull returns the second
FlowFile instead of the first.
To address this, use putToActive() from StandardProcessSession when committing
a FlowFile whose destination is the same queue it was polled from (the retry
path).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)