Thanks Jinsong for initiating this discussion. I agree that it's very important to upgrade the Paimon version when we restart our flink jobs with the new Paimon version. Is it necessary for Paimon to introduce some unit tests or e2e tests in addition to annotations to enforce compatibility?
Best, Fang Yong On Mon, May 27, 2024 at 3:25 PM yu zelin <yuzelin....@gmail.com> wrote: > Hi Jingsong, > > +1 for this. Just find a state compatibility problem: > https://github.com/apache/paimon/issues/3401. > It's important to ensure users can stop a Flink job which uses lower > version Paimon and restart it with latest version Paimon. > > Best regards, > Zelin Yu > > On Mon, May 27, 2024 at 2:32 PM Jingsong Li <jingsongl...@gmail.com> > wrote: > > > Hi Paimon Dev, > > > > We have made many changes that do not guarantee the compatibility of > > Flink job state, but in fact, we can guarantee it. We only need to > > consider the design of VersionedSerializer when making modifications. > > > > I plan to introduce an annotation that reminds contributors to ensure > > compatibility when making subsequent modifications to annotated > > classes. > > > > For example, StateCompatibility annotation. > > > > What do you think? > > > > Best, > > Jingsong > > >
