Hi, Zakelly. Thanks for your suggestion and I think it makes sense. I think both two options should be adopted. The option 2 is just what I mentioned in step 2. At the same time, we still need option 1 because we should make sure the existing user code could run normally without modifying any codes before we remove the deprecated method ( TypeSerializerSnapshot#resolveSchemaCompatibility).
Hi, Yanfei. Thanks for your feedback. Will Typeserializer#resolveSchemaCompatibility become an abstract method that must be implemented here? Considering the compatibility before removing the deprecated method ( TypeSerializerSnapshot#resolveSchemaCompatibility), we may still need to have a default implementation for Typeserializer#resolveSchemaCompatibility. When we remove the deprecated method (In the step 3), I think it deserves a discussion whether we need to make it abstract or not (it will break changes twice). Does it support the migration from built-in serializer -> custom serializer -> built-in serializer ? I think after reversing the direction of resolving schema compatibility, users could just consider the compatibility result in the newest serializer. If the built-in serializer cannot work well, users could just specify their new serializer. It could make users always have ways to restore from the snapshot if they want to do so. On Thu, Oct 13, 2022 at 11:37 AM yanfei lei <fredia...@gmail.com> wrote: > Hi Hangxiang, > > > Thanks for raising the discussion, +1 for reversing the direction of > resolving schema compatibility. > > As you described, in 'Step 1', Typeserializer#resolveSchemaCompatibility > will return TYPE.INCOMPATIBLE default, > Typeserializer#resolveSchemaCompatibility is a default method; in 'Step > 2&3', you proposed deprecating > TypeSerializerSnapshot#resolveSchemaCompatibility for the long run, will > Typeserializer#resolveSchemaCompatibility become an abstract method that > must be implemented here? > > > After this FILP， the new serializer claims its compatibility with the old > serializer， does it support the migration from > > built-in serializer -> custom serializer -> built-in serializer ? > > as you mentioned, some built-in serializers are final class, we can't > change the #resolveSchemaCompatibility() method of them. > > > Best, > > Yanfei > > Zakelly Lan <zakelly....@gmail.com> 于2022年10月13日周四 00:56写道： > > > Hi Hangxiang, > > > > Thanks for driving this. It is reasonable to let the new serializer > > claim its compatibility with the old serializer. However, I think > > there is a little confusion as you described in your proposed change > > 'Step 1'. You mean that we let the new serializer check the > > compatibility first, and if it gives the 'INCOMPATIBLE' then we let > > the old serializer do the check as before. The 'INCOMPATIBLE' from the > > new serializer means 'do not know' or 'unresolved' here, but sometimes > > the new serializer should express a decisive meaning that it is really > > not compatible with the older one, which is the 'INCOMPATIBLE' should > > mean. There is a semantic ambiguity or missing. I think there are two > > options to resolve this: > > 1. Provide 'UNRESOLVED' in type of the compatibility checking result, > > and let the proposed new interface return it by default. OR > > 2. Replace any use of the old interface with the new interface. In the > > default implementation of the new interface, call the old interface to > > leverage the old result. This approach provides the ability to totally > > replace original checking logic (by implementing the new interface in > > new serializer) while maintaining good backward compatibility. > > > > What do you think? > > > > Best, > > Zakelly. > > > > > > On Wed, Oct 12, 2022 at 8:41 PM Hangxiang Yu <master...@gmail.com> > wrote: > > > > > > Dear Flink developers, > > > > > > I would like to start a discussion thread on FLIP-263[1] proposing to > > > improve the usability of resolving schema compatibility. > > > > > > Currently, the place for compatibility checks is > > > TypeSerializerSnapshot#resolveSchemaCompatibility > > > which belongs to the old serializer, There are no ways for users to > > specify the > > > compatibility with the old serializer in the new customized serializer. > > > > > > The FLIP hopes to reverse the direction of resolving schema > compatibility > > > to improve the usability of resolving schema compatibility. > > > > > > [1] > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-263%3A+Improve+resolving+schema+compatibility > > > -- Best, Hangxiang.
