+1 on dropping Java 8 in Spark 4.0, and I even hope Spark 4.0 can only support Java 17 and the upcoming Java 21.
发件人: Denny Lee <denny.g....@gmail.com> 日期: 2023年6月7日 星期三 07:10 收件人: Sean Owen <sro...@gmail.com> 抄送: David Li <lidav...@apache.org>, "dev@spark.apache.org" <dev@spark.apache.org> 主题: Re: JDK version support policy? +1 on dropping Java 8 in Spark 4.0, saying this as a fan of the fast-paced (positive) updates to Arrow, eh?! On Tue, Jun 6, 2023 at 4:02 PM Sean Owen <sro...@gmail.com<mailto:sro...@gmail.com>> wrote: I haven't followed this discussion closely, but I think we could/should drop Java 8 in Spark 4.0, which is up next after 3.5? On Tue, Jun 6, 2023 at 2:44 PM David Li <lidav...@apache.org<mailto:lidav...@apache.org>> wrote: Hello Spark developers, I'm from the Apache Arrow project. We've discussed Java version support [1], and crucially, whether to continue supporting Java 8 or not. As Spark is a big user of Arrow in Java, I was curious what Spark's policy here was. If Spark intends to stay on Java 8, for instance, we may also want to stay on Java 8 or otherwise provide some supported version of Arrow for Java 8. We've seen dependencies dropping or planning to drop support. gRPC may drop Java 8 at any time [2], possibly this September [3], which may affect Spark (due to Spark Connect). And today we saw that Arrow had issues running tests with Mockito on Java 20, but we couldn't update Mockito since it had dropped Java 8 support. (We pinned the JDK version in that CI pipeline for now.) So at least, I am curious if Arrow could start the long process of migrating Java versions without impacting Spark, or if we should continue to cooperate. Arrow Java doesn't see quite so much activity these days, so it's not quite critical, but it's possible that these dependency issues will start to affect us more soon. And looking forward, Java is working on APIs that should also allow us to ditch the --add-opens flag requirement too. [1]: https://lists.apache.org/thread/phpgpydtt3yrgnncdyv4qdq1gf02s0yj<https://mailshield.baidu.com/check?q=Nz%2bGj2hdKguk92URjA7sg0PfbSN%2fXUIMgrHTmW45gOOKEr3Shre45B7TRzhEpb%2baVsnyuRL%2fl%2f0cu7IVGHunSGDVnxM%3d> [2]: https://github.com/grpc/proposal/blob/master/P5-jdk-version-support.md<https://mailshield.baidu.com/check?q=s89S3eo8GCJkV7Mpx7aG1SXId7uCRYGjQMA6DeLuX9duS86LhIODZMJfeFdGMWdFzJ8S7minyHoC7mCrzHagbJXCXYTBH%2fpZBpfTbw%3d%3d> [3]: https://github.com/grpc/grpc-java/issues/9386<https://mailshield.baidu.com/check?q=R0HtWZIkY5eIxpz8jtqHLzd0ugNbcaXIKW2LbUUxpIn0t9Y9yAhuHPuZ4buryfNwRnnJTA%3d%3d>
