sameer-sde commented on issue #11961: URL: https://github.com/apache/seatunnel/issues/11961#issuecomment-5593840421
Hi, I'd like to work on this. Looking at `CheckpointStorage` (checkpoint-storage-api), checkpoints and savepoints currently share the same storage path and `PipelineState` format with no version tag. My proposed approach: 1. Introduce a small envelope/header (e.g. a `formatVersion` field) written alongside `PipelineState` bytes, so any reader can detect which format a stored file uses before deserializing. 2. Separate the storage namespace for savepoints vs. regular checkpoints (e.g. distinct path prefixes or a `type` discriminator), since savepoints are meant to be long-lived and manually triggered, while checkpoints are transient and auto-pruned via `max-retained`. 3. Keep backward compatibility: if no version tag is present, assume the current (legacy) format. Before I start coding, wanted to confirm this direction makes sense, or if there's a preferred approach already discussed elsewhere. Happy to open a design doc if that's preferred for a change this size. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
