So from my point of view, I'd do it maybe 1-2 years after all the major Hadoop vendors have stopped supporting Java 6. We're not there yet, but we will be soon. The reason is that the cost of staying on Java 6 is much smaller to us (as developers) than the cost of fragmenting the Spark community by having a big chunk of users unable to upgrade past a certain version of Spark (or requiring them to use a modified third-party version, which is a similar thing). There's very little stuff in Java 7 or 8 that would make the project much better if we dropped support for 6 -- this Jetty issue might be one, but we can certainly work around it.
We've done a lot of stuff to reach the broadest set of users, including pushing back the versions of Python and NumPy we support, and in my experience it's been well worth it. In surveys I've seen the majority of users (something like 75%) updating to each new Spark release within a few months of it coming out, which is awesome for keeping the community healthy. Matei > On Oct 19, 2014, at 9:48 AM, Corey Nolet <cjno...@gmail.com> wrote: > > A concrete plan and a definite version upon which the upgrade would be > applied sounds like it would benefit the community. If you plan far enough > out (as Hadoop has done) and give the community enough of a notice, I can't > see it being a problem as they would have ample time upgrade. > > > > On Sat, Oct 18, 2014 at 9:20 PM, Marcelo Vanzin <van...@cloudera.com > <mailto:van...@cloudera.com>> wrote: > Hadoop, for better or worse, depends on an ancient version of Jetty > (6), that is even on a different package. So Spark (or anyone trying > to use a newer Jetty) is lucky on that front... > > IIRC Hadoop is planning to move to Java 7-only starting with 2.7. Java > 7 is also supposed to be EOL some time next year, so a plan to move to > Java 7 and, eventually, Java 8 would be nice. > > On Sat, Oct 18, 2014 at 5:44 PM, Matei Zaharia <matei.zaha...@gmail.com > <mailto:matei.zaha...@gmail.com>> wrote: > > I'd also wait a bit until these are gone. Jetty is unfortunately a much > > hairier topic by the way, because the Hadoop libraries also depend on > > Jetty. I think it will be hard to update. However, a patch that shades > > Jetty might be nice to have, if that doesn't require shading a lot of other > > stuff. > > > > Matei > > > >> On Oct 18, 2014, at 4:37 PM, Koert Kuipers <ko...@tresata.com > >> <mailto:ko...@tresata.com>> wrote: > >> > >> my experience is that there are still a lot of java 6 clusters out there. > >> also distros that bundle spark still support java 6 > >> On Oct 17, 2014 8:01 PM, "Andrew Ash" <and...@andrewash.com > >> <mailto:and...@andrewash.com>> wrote: > >> > >>> Hi Spark devs, > >>> > >>> I've heard a few times that keeping support for Java 6 is a priority for > >>> Apache Spark. Given that Java 6 has been publicly EOL'd since Feb 2013 > >>> <http://www.oracle.com/technetwork/java/eol-135779.html > >>> <http://www.oracle.com/technetwork/java/eol-135779.html>> and the last > >>> public update was Apr 2013 > >>> <https://en.wikipedia.org/wiki/Java_version_history#Java_6_updates > >>> <https://en.wikipedia.org/wiki/Java_version_history#Java_6_updates>>, why > >>> are we still maintaing support for 6? The only people using it now must > >>> be > >>> paying for the extended support to continue receiving security fixes. > >>> > >>> Bumping the lower bound of Java versions up to Java 7 would allow us to > >>> upgrade from Jetty 8 to 9, which is currently a conflict with the > >>> Dropwizard framework and a personal pain point. > >>> > >>> Java 6 vs 7 for Spark links: > >>> Try with resources > >>> <https://github.com/apache/spark/pull/2575/files#r18152125 > >>> <https://github.com/apache/spark/pull/2575/files#r18152125>> for > >>> SparkContext et al > >>> Upgrade to Jetty 9 > >>> <https://github.com/apache/spark/pull/167#issuecomment-54544494 > >>> <https://github.com/apache/spark/pull/167#issuecomment-54544494>> > >>> Warn when not compiling with Java6 > >>> <https://github.com/apache/spark/pull/859 > >>> <https://github.com/apache/spark/pull/859>> > >>> > >>> > >>> Who are the people out there that still need Java 6 support? > >>> > >>> Thanks! > >>> Andrew > >>> > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > > <mailto:dev-unsubscr...@spark.apache.org> > > For additional commands, e-mail: dev-h...@spark.apache.org > > <mailto:dev-h...@spark.apache.org> > > > > > > -- > Marcelo > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > <mailto:dev-unsubscr...@spark.apache.org> > For additional commands, e-mail: dev-h...@spark.apache.org > <mailto:dev-h...@spark.apache.org> > >
