je-ik commented on issue #21897: URL: https://github.com/apache/beam/issues/21897#issuecomment-1171041116
What do you mean by backward compatible savepoint in Flink? Beam's Flink runner uses Flink's checkpoint/savepoint mechanism, thus it should be compatible under the same circumstances as native Flink. The problem is that Beam adds additional layer of abstraction that can cause state incompatibility - e.g. when a translation of a Beam PTransform changes, when a Coder changes, etc. There currently is not any mechanism that can transfer incompatible savepoints between versions, the only option is to run a "bootstrap" version of your job, that creates the state from some historical data. Reading the a savepoint and converting it to preserve compatibility across beam versions *could* be possible under some circumstances, but is not possible in general. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
