Claudio Mattioni created NIFI-15938:
---------------------------------------
Summary: TailFile should improve multi-file startup and persisted
state recovery
Key: NIFI-15938
URL: https://issues.apache.org/jira/browse/NIFI-15938
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 2.9.0
Reporter: Claudio Mattioni
TailFile should improve multi-file startup performance and make persisted state
recovery more robust.
Current behavior and risks:
- In multi-file mode, TailFile can perform eager file listing and repeated
state membership checks during startup.
- Persisted rollover state can cause recovery problems when tailingPostRollover
state exists but rolled files are no longer present.
- Malformed persisted numeric/checksum state can cause recovery failures
instead of being handled defensively.
- The read loop can reuse buffers in a way that risks incorrect behavior.
- Buffer size validation should reject zero-sized or overflowing values before
buffer allocation.
Proposed changes:
- Improve multi-file startup by replacing eager file listing with streaming
file discovery.
- Replace repeated O(n²) state membership checks with HashSet membership.
- Restore persisted tailingPostRollover state correctly.
- Avoid crashes when rollover state exists but no rolled files remain.
- Harden recovery against malformed persisted numeric and checksum state.
- Fix buffer reuse in the read loop.
- Tighten buffer-size validation to prevent zero-sized or overflow-sized
buffers.
- Add regression tests covering the fixed TailFile scenarios.
Verification performed:
- .\mvnw.cmd --% -pl :nifi-standard-processors -am
-Dtest=TestTailFile,TestTailFileSimpleScenarios,TestTailFileGeneratedScenarios
-Dsurefire.failIfNoSpecifiedTests=false test
- .\mvnw.cmd --% -pl :nifi-standard-processors -Pcontrib-check validate
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
