[ 
https://issues.apache.org/jira/browse/NIFI-16599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Handermann resolved NIFI-16599.
-------------------------------------
    Fix Version/s: 2.13.0
       Resolution: Fixed

> Preserve FlowFile order across restart for queues that are swapping
> -------------------------------------------------------------------
>
>                 Key: NIFI-16599
>                 URL: https://issues.apache.org/jira/browse/NIFI-16599
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework
>            Reporter: Mark Payne
>            Assignee: Mark Payne
>            Priority: Major
>             Fix For: 2.13.0
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> When a connection holds more FlowFiles than its swap threshold, FlowFiles can 
> come out of the queue in the wrong order after NiFi restarts. There are three 
> causes.
> If more FlowFiles are restored from the FlowFile Repository than the active 
> queue can hold, but there are no swap files, whichever FlowFiles are restored 
> first fill the active queue. The restore order is arbitrary, so newer 
> FlowFiles can be processed ahead of older ones. NIFI-16580 fixed this only 
> for the case where swap files exist.
> A selective drop rewrites a swap file under a new name that uses the current 
> time. On restart, swap files are sorted by the time in their names, so the 
> rewritten file is placed behind swap files that hold newer FlowFiles.
> Swap files written within the same millisecond are ordered by a random UUID 
> in their names, so their order after a restart is random.
> SwappablePriorityQueue should restore FlowFiles in priority order when there 
> are no swap files. FileSystemSwapManager should name swap files so that a 
> rewritten file keeps its place and files written in the same millisecond sort 
> in the order they were written.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to