> On 30 Apr 2015, at 21:40, Marcelo Vanzin <van...@cloudera.com> wrote: > > As for the idea, I'm +1. Spark is the only reason I still have jdk6 > around - exactly because I don't want to cause the issue that started > this discussion (inadvertently using JDK7 APIs). And as has been > pointed out, even J7 is about to go EOL real soon.
+1, perhaps with a roadmap for people to plan for > > Even Hadoop is moving away (I think 2.7 will be j7-only). Hive 1.1 is > already j7-only. And when Hadoop moves away from something, it's an > event worthy of headlines. The constraint here was that there were too many people "stuck" in Java 6, and java 7 wasn't compelling enough to pull people off a JVM they trusted to be stable at large scale. One problem with production hadoop is that across 5000 14-core servers, all race conditions will surface —leading to a reluctance to upgrade JVMs or even OS's. There was also the fact that for a long time Hadoop wouldn't build on OSX on Java 7 (HADOOP-9350). Even today, OS/X's JDK has better rendering than java7+, leaving it nice to have around for the IDEs. After Hadoop 2.5 shipped an announcement was made that 2.6 would be the last Java 1.6 release, with the switch taking place in November. Moving ASF Jenkins up was probably the hardest bit ( HADOOP-10530 ). Switching to JDK7 has enabled moving kerberos support to Java 8 (HADOOP-10786; some changes in the internal kerberos classes used directly for kerberos to work properly). See HADOOP-11090 for the JDK8 migration; Hadoop trunk will be switching to Java 8 before long > They're still on Jetty 6! While moving off Jetty entirely wherever possible, leaving jetty 6 on the transitive-maven-classpath in the hope of not breaking code that expects it to be there. It's not that the project likes Jetty 6 (there are threads whose sole aim is to detect jetty startup failures), but that moving off it is felt to be better than upgrading. > > As for pyspark, https://github.com/apache/spark/pull/5580 should get > rid of the last incompatibility with large assemblies, by keeping the > python files in separate archives. If we remove support for Java 6, > then we don't need to worry about the size of the assembly anymore. zzhang's patch drops to Java 6 just to rebuild the assembly Jar; you can still build Java7-only classes. So it will work even before the pyspark patch goes in.
