On 27 Oct 2017, at 03:21, Zhang, Liyun <liyun.zh...@intel.com<mailto:liyun.zh...@intel.com>> wrote:
Hi all: 1. I want to build spark on jdk9 and test it with Hadoop on jdk9 env. I search for jiras related to JDK9. I only found SPARK-13278<https://issues.apache.org/jira/browse/SPARK-13278>. This means now spark can build or run successfully on JDK9 ? Best Regards Kelly Zhang/Zhang,Liyun Don't know about spark itself, but I do know that getting Hadoop on JDK9 is still a WiP, primarily because they've locked it down so much (good) including the things hadoop gets at to make things like kerberos auth work (bad) https://issues.apache.org/jira/browse/HADOOP-11123 A large part of the issues are with moving its dependencies to Java 9 compatible ones (Log4J, mockito, JUnit) —all of those which only surface in testing and in the build itself won't be relevant for Spark standalone. Other than that, YARN doesn't work Most of the work has been done by one person (Akira @ NEC); if there are other people willing to help, including building & testing Spark against (locally built) JDK9 Hadoop artifacts life would be better. We could maybe build & release some alpha-quality Hadoop 3.1.x-alpha-JDK9 artifacts if that would help FWIW, there's long been some background chatter between the old Sun JDK team & the ASF big data stack devs; modules is something wonderful which will kill the need for shading and reduce/eliminate classpath hell. Earlier on there was some discussion about having proper 2d arrays & direct memory access of some structures, but that's not in this version. Give it time. Oracle are being aggressive about retiring Java 8: by Sept 2018 they plan to not provide public updates for it. Which means building against Java 9 dev time is here for everyone
