Narrowly on Java 11: the problem is that it'll take some breaking changes, more than would be usually appropriate in a minor release, I think. I'm still not convinced there is a burning need to use Java 11 but stay on 2.4, after 3.0 is out, and at least the wheels are in motion there. Java 8 is still free and being updated.
On Fri, Sep 20, 2019 at 12:48 PM Ryan Blue <rb...@netflix.com.invalid> wrote: > > Hi everyone, > > In the DSv2 sync this week, we talked about a possible Spark 2.5 release > based on the latest Spark 2.4, but with DSv2 and Java 11 support added. > > A Spark 2.5 release with these two additions will help people migrate to > Spark 3.0 when it is released because they will be able to use a single > implementation for DSv2 sources that works in both 2.5 and 3.0. Similarly, > upgrading to 3.0 won't also require also updating to Java 11 because users > could update to Java 11 with the 2.5 release and have fewer major changes. > > Another reason to consider a 2.5 release is that many people are interested > in a release with the latest DSv2 API and support for DSv2 SQL. I'm already > going to be backporting DSv2 support to the Spark 2.4 line, so it makes sense > to share this work with the community. > > This release line would just consist of backports like DSv2 and Java 11 that > assist compatibility, to keep the scope of the release small. The purpose is > to assist people moving to 3.0 and not distract from the 3.0 release. > > Would a Spark 2.5 release help anyone else? Are there any concerns about this > plan? > > > rb > > > -- > Ryan Blue > Software Engineer > Netflix --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
