, October 30, 2013 8:45 AM
To: user@spark.incubator.apache.org
Subject: RE: spark-0.8.0 and hadoop-2.1.0-beta
I am also working on porting the trunk code onto 2.2.0. Seems quite many API
changes but many of them are just a rename work.
While Yarn 2.1.0 beta also add some client API for easy
don't find the replacement or they are not need anymore.
Best Regards,
Raymond Liu
From: Matei Zaharia [mailto:matei.zaha...@gmail.com]
Sent: Wednesday, October 30, 2013 2:35 AM
To: user@spark.incubator.apache.org
Subject: Re: spark-0.8.0 and hadoop-2.1.0-beta
I'm curious, Viren, do
I’m curious, Viren, do you have a patch you could post to build this against
YARN 2.1 / 2.2? It would be nice to see how big the changes are.
Matei
On Sep 30, 2013, at 10:14 AM, viren kumar wrote:
> I was able to get Spark 0.8.0 to compile with Hadoop/Yarn 2.1.0-beta, by
> following some of t
I was able to get Spark 0.8.0 to compile with Hadoop/Yarn 2.1.0-beta, by
following some of the changes described here:
http://hortonworks.com/blog/stabilizing-yarn-apis-for-apache-hadoop-2-beta-and-beyond/
That should help you build most of it. One change not covered there is the
change from Proto
Hi Terence,
YARN's API changed in an incompatible way in Hadoop 2.1.0, so I'd suggest
sticking with 2.0.x for now. We may create a different branch for this version.
Unfortunately due to the API change it may not be possible to support this
version while also supporting other widely-used versio
