[
https://issues.apache.org/jira/browse/NIFI-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser reassigned NIFI-15938:
--------------------------------------
Assignee: Claudio Mattioni
> TailFile should improve multi-file startup and persisted state recovery
> -----------------------------------------------------------------------
>
> Key: NIFI-15938
> URL: https://issues.apache.org/jira/browse/NIFI-15938
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 2.9.0
> Reporter: Claudio Mattioni
> Assignee: Claudio Mattioni
> Priority: Major
> Labels: Processor, TailFile
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> TailFile should improve multi-file startup performance and make persisted
> state recovery more robust.
> Current behavior and risks:
> - In multi-file mode, TailFile can perform eager file listing and repeated
> state membership checks during startup.
> - Persisted rollover state can cause recovery problems when
> tailingPostRollover state exists but rolled files are no longer present.
> - Malformed persisted numeric/checksum state can cause recovery failures
> instead of being handled defensively.
> - The read loop can reuse buffers in a way that risks incorrect behavior.
> - Buffer size validation should reject zero-sized or overflowing values
> before buffer allocation.
> Proposed changes:
> - Improve multi-file startup by replacing eager file listing with streaming
> file discovery.
> - Replace repeated O(n²) state membership checks with HashSet membership.
> - Restore persisted tailingPostRollover state correctly.
> - Avoid crashes when rollover state exists but no rolled files remain.
> - Harden recovery against malformed persisted numeric and checksum state.
> - Fix buffer reuse in the read loop.
> - Tighten buffer-size validation to prevent zero-sized or overflow-sized
> buffers.
> - Add regression tests covering the fixed TailFile scenarios.
> Verification performed:
> - .\mvnw.cmd --% -pl :nifi-standard-processors -am
> -Dtest=TestTailFile,TestTailFileSimpleScenarios,TestTailFileGeneratedScenarios
> -Dsurefire.failIfNoSpecifiedTests=false test
> - .\mvnw.cmd --% -pl :nifi-standard-processors -Pcontrib-check validate
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
