Stephan Ewen created FLINK-8539: ----------------------------------- Summary: Introduce "CompletedCheckpointStorageLocation" to explicitly handle disposal of checkpoint storage locations Key: FLINK-8539 URL: https://issues.apache.org/jira/browse/FLINK-8539 Project: Flink Issue Type: Sub-task Components: State Backends, Checkpointing Reporter: Stephan Ewen Assignee: Stephan Ewen Fix For: 1.5.0
The storage location of completed checkpoints misses a proper representation. Because of that, there is no place that can handle the deletion of a checkpoint directory, or the dropping of a checkpoint specific table. Current workaround for file systems is, for example, that every file disposal checks if the parent directory is now empty, and deletes it if that is the case. That is not only inefficient, but prohibitively expensive on some systems, like Amazon S3. Properly representing the storage location for completed checkpoints allows us to add a disposal call for that location. That {{CompletedCheckpointStorageLocation}} can also be used to capture "external pointers", metadata, and even allow us to use custom serialization and deserialization of the metadata in the future, making the abstraction more extensible by allowing users to introduce new types of state handles. -- This message was sent by Atlassian JIRA (v7.6.3#76005)