Mark Payne updated NIFI-32:
    Fix Version/s: 1.2.0

> Swap File format should include summary at end for faster NiFi system startup
> -----------------------------------------------------------------------------
>                 Key: NIFI-32
>                 URL: https://issues.apache.org/jira/browse/NIFI-32
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Core Framework
>            Reporter: Mark Payne
>            Assignee: Mark Payne
>            Priority: Minor
>             Fix For: 1.2.0
> When NiFi is restarted with many hundreds or thousands of Swap Files, it can 
> take a very long time to startup.
> If when we write out the Swap Files, we append a summary of the Swap File 
> (queue identifier, number of FlowFiles, size of FlowFiles, and ContentClaim 
> info/count map), we can then read far less data for restart. This means we 
> would serialize the summary to a ByteArrayOutputStream, then get the length, 
> and then write out the summary, followed by length. On restore, we read just 
> the last 8 bytes, and that allows us to generate the offset into the file to 
> start reading the summary.

This message was sent by Atlassian JIRA

Reply via email to