Thanks for the feedback!
> To streamline our process, I propose we build a mechanism to expire
old clients after a certain period, ensuring that only actively
supported Flink/Spark/MR versions are in use.
That sounds pretty good, +1 for the proposal.
I don't know much about other engines, but I can offer some background about
the Flink community:
We only actively maintains the master branch(target 2.0 for now), and the
last two major releases(1.20.x and 1.19.x).
For the most recent third major release version(1.18.x), there is usually a
final release at an appropriate time, and then support is no longer
guaranteed).
Best regards,
Weijie
Ethan Feng <ethanf...@apache.org> 于2024年12月19日周四 11:43写道：
> Hi Weijie,
>
> I appreciate your thoughts on removing the outdated Flink versions. I
> completely agree that it's essential to focus on maintaining the still
> relevant versions.
>
> To streamline our process, I propose we build a mechanism to expire
> old clients after a certain period, ensuring that only actively
> supported Flink/Spark/MR versions are in use. This would simplify our
> support efforts and encourage users to update to newer versions.
>
> Regarding your question about how many Flink versions we should retain
> support for, keeping support for the last {4} or {5} major versions
> could be a good approach, depending on user feedback.
>
> Let me know your thoughts!
>
> Thanks,
> Ethan Feng
>
> Nicholas Jiang <nicholasji...@apache.org> 于2024年12月19日周四 11:36写道：
> >
> > Hi weijie,
> >
> > Thanks for driving the out-of-dated flink versions. +1 for removing the
> out-of-dated flink versions because the support of different Flink versions
> are similar.
> >
> > IMO, We could firstly remove the support for Flink 1.14 and 1.15
> version, which versions are very old and few functional features.
> Meanwhile, the support of other Flink versions could be gradually removed
> in the Celeborn major version.
> >
> > BTW, how many Flink versions should we retain support for?
> >
> > Regards,
> > Nicholas Jiang
> >
> > On 2024/12/18 06:18:17 weijie guo wrote:
> > > Hi all,
> > >
> > > I would like to suggest that we consider ending the support for some
> of the
> > > older Flink versions. The main reasons are as follows:
> > >
> > > 1. Currently, the minimum supported version is Flink-1.14. But Flink
> > > releases like 1.14 and 1.15 have been out of date for a long time, and
> > > some important features (speculative execution, AQE, etc.) of Flink
> batch
> > > are missing.
> > >
> > > 2. The Flink community currently does not support versions lower than
> 1.18,
> > > considering that Flink 1.20 will be the first LTS version, the
> existing 1.x
> > > users will gradually upgrade to 1.20 also.
> > >
> > > 3. There is a big difference between the shuffle API of the old
> version and
> > > the current Flink code base, and it is tedious to maintain
> compatibility.
> > > Moreover, new features such as JM Failover and hybrid shuffle cannot be
> > > supported in out-of-dated Flink release. If users do have a need to
> adapt
> > > older Flink versions, it's not difficult for them to do it themselves,
> but
> > > this can reduce our maintenance burden.
> > >
> > > For now, communities such as Apache Iceberg have taken a more radical
> > > strategy that in line with Flink's official supported version. We don't
> > > have to go that far yet, but it feels like at least we can remove the
> > > versions 1.14 and 1.15 for now.
> > >
> > >
> > > Best regards,
> > >
> > > Weijie
> > >
>
