Aaron He created FLINK-40685:
--------------------------------
Summary: ForSt skips local cache cleanup for checkpoint-owned files
Key: FLINK-40685
URL: https://issues.apache.org/jira/browse/FLINK-40685
Project: Flink
Issue Type: Bug
Components: Runtime / State Backends
Reporter: Aaron He
h2. Problem
ForSt can keep a local cached copy of an SST file whose source is owned by a
checkpoint.
When the backend no longer references that file, it must preserve the
checkpoint-owned source. However, the current cleanup also skips releasing the
local cached copy.
This can retain unnecessary files in the local cache. Unbounded disk growth and
production impact have not been established.
h2. Why this happens
MappingEntry.referenceCountReachedZero() returns immediately when the file is
marked NOT_OWNED. The local cache cleanup appears after that return, so it is
never reached for these files.
h2. Expected behavior
Release the local cache entry when the final mapping reference is released,
while preserving the checkpoint-owned source. Active readers must remain
usable; physical deletion should follow the cache's existing reference-counting
behavior.
h2. Proposed fix
Move eligible local cache cleanup before the ownership check. Keep the
ownership check for deleting the source file, and preserve existing cleanup for
backend-owned files.
h2. Planned work
I will work on the fix and add regression tests covering the ownership handoff
and active readers.
h2. Reproduction to add
# Create an SST file and populate its local cache entry.
# Transfer ownership using a FileStateHandle.
# Release the final mapping reference.
# Check that the cache entry is invalidated, while the checkpoint file remains
readable.
Also check that an active reader can finish safely and that physical cache-file
deletion waits for its reference to be released.
h2. Evidence
The cleanup order is still present on Apache master at 07b4ff26f545, inspected
on 2026-09-16. This finding is based on source inspection; no tests or runtime
reproduction have been performed for this draft.
[MappingEntry
cleanup|https://github.com/apache/flink/blob/07b4ff26f545e0d9373949fd82573074ae442291/flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/fs/filemapping/MappingEntry.java#L141-L157]
h2. Proposed release note
ForSt reclaims local cached copies when mappings to checkpoint-owned files are
released.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
