Hi, We are using beam + non-portable flink runner and now we are trying to upgrade from 2.31.0 to 2.32.0. Savepoint restore failed with:
Caused by: java.lang.IllegalStateException: Could not Java-deserialize TypeSerializer while restoring checkpoint metadata for serializer snapshot 'org.apache.beam.runners.flink.translation.types.CoderTypeSerializer$LegacySnapshot'. Please update to the TypeSerializerSnapshot interface that removes Java Serialization to avoid this problem in the future. at org.apache.flink.api.common.typeutils.TypeSerializerConfigSnapshot.restoreSerializer(TypeSerializerConfigSnapshot.java:145) at org.apache.flink.runtime.state.StateSerializerProvider.previousSchemaSerializer(StateSerializerProvider.java:186) at org.apache.flink.runtime.state.StateSerializerProvider.currentSchemaSerializer(StateSerializerProvider.java:161) at org.apache.flink.runtime.state.RegisteredOperatorStateBackendMetaInfo.getPartitionStateSerializer(RegisteredOperatorStateBackendMetaInfo.java:112) at org.apache.flink.runtime.state.OperatorStateRestoreOperation.restore(OperatorStateRestoreOperation.java:93) at org.apache.flink.runtime.state.DefaultOperatorStateBackendBuilder.build(DefaultOperatorStateBackendBuilder.java:80) ... 16 more Caused by: java.io.InvalidClassException: org.apache.beam.sdk.transforms.windowing.IntervalWindow$IntervalWindowCoder; local class incompatible: stream classdesc serialVersionUID = -4033763546256145084, local class serialVersionUID = -3349635311633734096 That's because IntervalWindowCoder changed [1]. As I understand restore savepoint may work but there is no guarantee and this may happens. >From my perspective this leads to very bad experience for users using beam. Is there any plan to change it? Flink itself should support coders evolution. Is this handled better in portable runner? How is this handled by another runners, for example in Dataflow? Any plan to change this? L. [1] https://github.com/apache/beam/commit/7d98ad2b6554258eeccf9f7c1b017f9a001bfd87#diff-9bd0fd4fa4cbbd6ddb29b194c39d6d46a4f229aa0f1db9ed79ec2adea7971854R178
