Hi Zhanghao, Chen. Thanks a lot for driving this FLIP!
I took a quick look at the proposal as well as Flink’s current Kryo usage and its impact scope, and I have a couple of questions: - a. One goal of this FLIP is to improve state compatibility between Kryo 1.x and 2.x. I noticed that some other serialization-related classes in the latest Flink code still hardcode `KryoSerializer` instantiation (e.g., `CompactCoordinator`, `CompactOperator`, `PojoTypeInfo`, `DataTypeFactoryImpl`). This indicates that Kryo’s impact is broader than just `GenericTypeInfo`. So if we only change the logic in `GenericTypeInfo#createSerializer`, it seems we may not fully achieve this goal. - b. Following from (a), if we want to ensure full Kryo compatibility, we’d likely need to cover those cases as well. In that case, the current FLIP title may not accurately reflect the scope of the work, since the new configurable serializer would need to play the same role and cover the same impact scope as Kryo today. It might be good to first clarify the scope of this FLIP. For example, should it focus only on introducing the new configuration item and improving `GenericTypeInfo` serialization/performance, while leaving state compatibility aside? Or do we want to cover both, in which case we could outline a broader plan for the compatibility-related changes. Please correct me if I misunderstood anything, and please let know what's you opinion~ Best, Yuepeng Pan At 2025-07-28 11:38:42, "Zhanghao Chen" <zhanghao.c...@outlook.com> wrote: >Hi everyone, > >I'd like to start a discussion of FLIP-538: Support Custom Generic Type >Serializer [1]. > >Flink handles most common types efficiently with the built-in serializers and >leaves the remaining to the generic type serializer Kryo in a hardcoded way. >We propose supporting custom generic type serializers with two major >motivations: > > 1. >State-compatibility: Flink 2.0 upgraded Kryo from v2.24.0 to v5.6.2 with >FLIP-317 [2], and hence dropped state compatibility with 1.x jobs that rely on >Kryo v2.24.0. Users can copy the impl from previous Flink release for a smooth >upgrade. > > 2. >Performance: new serializer frameworks like Apache Fory [3] provides 3x >speedup compared to Kryo by our internal testing. We also plan to cooperate >with the Fory community to maintain an implementation under the Fory repo. > >Looking forward to suggestions on it! > >[1] >https://cwiki.apache.org/confluence/display/FLINK/FLIP+538%3A+Support+Custom+Generic+Type+Serializer >[2] >https://cwiki.apache.org/confluence/display/FLINK/FLIP-317%3A+Upgrade+Kryo+from+2.24.0+to+5.5.0 >[3] https://fory.apache.org/ > >Best, >Zhanghao Chen
