[
https://issues.apache.org/jira/browse/FLINK-26147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dawid Wysakowicz updated FLINK-26147:
-------------------------------------
Description:
Native format supports incremental RocksDB savepoints. For those savepoints
Flink puts all SST files inside the savepoints directory. This means such
savepoints are self-contained and relocatable. However, when restored in CLAIM
mode, subsequent checkpoints might reuse some SST files, which in turn might
block deleting the savepoints directory at the time the savepoint is subsumed.
Later on Flink will delete the reused shared SST files, but it won’t retry
deleting the savepoints directory. Therefore, it is possible Flink leaves an
empty savepoints directory if it was restored in CLAIM mode.
We should find a way to clean up that left over directory.
> Deleting incremental savepoints directory in CLAIM mode
> -------------------------------------------------------
>
> Key: FLINK-26147
> URL: https://issues.apache.org/jira/browse/FLINK-26147
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Checkpointing
> Affects Versions: 1.15.0
> Reporter: Piotr Nowojski
> Priority: Major
>
> Native format supports incremental RocksDB savepoints. For those savepoints
> Flink puts all SST files inside the savepoints directory. This means such
> savepoints are self-contained and relocatable. However, when restored in
> CLAIM mode, subsequent checkpoints might reuse some SST files, which in turn
> might block deleting the savepoints directory at the time the savepoint is
> subsumed. Later on Flink will delete the reused shared SST files, but it
> won’t retry deleting the savepoints directory. Therefore, it is possible
> Flink leaves an empty savepoints directory if it was restored in CLAIM mode.
> We should find a way to clean up that left over directory.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
