stevenzwu commented on issue #1438: URL: https://github.com/apache/iceberg/issues/1438#issuecomment-693518355
Regarding `DataFile`, in addition to the TypeSerializer (for state) we also need to implement the custom Kryo Serializers for the network transfer btw writer and committer, since the network buffer can be included in the checkpoints due to unaligned checkpoints. Once @chenjunjiedada confirmed that it is a rolling upgrade case, maybe we should bring this to the Flink community and see if Flink provides any serialization compatibility guarantee for that operation mode. As @JingsongLi mentioned, It is cumbersome to implement custom serializers for all the Iceberg classes. It would be nice to avoid it. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
