StephanEwen opened a new pull request #11179: [FLINK-16178][FLINK-16192][checkpointing] Clean up checkpoint metadata code and remove remaining bits of "legacy state" URL: https://github.com/apache/flink/pull/11179 ## NOTE: This removes savepoint compatibility with Flink 1.2 ! For background, please consult [FLINK-16192](https://issues.apache.org/jira/browse/FLINK-16192) ## What is the purpose of the change **This is part of the preparation for implementing Fault Tolerance for Operator Coordinators for the new Source Interface (FLIP-27).** This PR contains various cleanups / simplifications / refactorings around the checkpoint metadata serialization / deserialization code, to make it easier to implement the handling of Operator Coordinator State. ...and to finally drop some really old and unused code. I felt a bit like an archaeologist digging through this ;-) ## Brief change log The main changes are: - 5436317bc99830bdc21b5799fcabf8f833120932 : *[FLINK-16178][refactor] Make the versioned Checkpoint Metadata Serializers only responsible for deserialization.* The metadata is always serialized with the latest serializer. Deserialization uses a version-specific serializer. That is the same functionality as before, but cleanly represented in the code. The code previously pretended to be version-dynamic on serialization side as well, but still could only handle the latest version on serialization. Earlier versions failed with a `ClassCastException` or `UnsupportedMethodException`. Weird constructs were necessary, like a `SavepointV1Serializer implements SavepointSerializer<SavepointV2>` (mismatch between handled version and generic type). - 247bd519b8609610c80704bb4a7c0e5a8375494a : *[FLINK-16192][checkpointing] Remove remaining bits of "legacy state" and remove Savepoint 1.2 compatibility* "Legacy State" Refers to the state originally created by the old `Checkpointed` interface, before state was re-scalable. This was later replaced by `CheckpointedFunction` (and `ListCheckpointed` as the shortcut). This state was no longer supported since Flink 1.4. However, all 1.2 savepoints in migration tests (and some 1.3 savepoints for tests) used that "legacy state", that is why the code was retained and support for "legacy state" was hackily activated (via some static flags) for migration tests. Removing that code would actually not reduce any savepoint compatibility, because compatibility for 1.2 and 1.3 was anyways only given for jobs that did not use that "legacy state". However, removing this code makes all 1.2 savepoints from tests unusable. A [mailing list discussion](https://lists.apache.org/x/thread.html/rd36579fe754f7557da4ae1dda47820c84b21009c35e6e317671e83b0@%3Cdev.flink.apache.org%3E) suggested to drop that support, tather than elaborate re-building of these the test data for Flink 1.2 with different state type. ## Verifying this change This change is already covered by existing tests, such as the savepoint tests and savepoint migration tests. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): **no** - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no** - The serializers: **no** - The runtime per-record code paths (performance sensitive): **no** - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: **yes** - The S3 file system connector: **no** ## Documentation - Does this pull request introduce a new feature? **no** - If yes, how is the feature documented? **not applicable**
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
