Hi, sorry to bother again.
As a newbie to the project it's hard to judge whether I'm doing anything wrong, the documentation is outdated or the Maven/SBT files have diverged from the actual code by defining older/now incompatible versions or something else going wrong. Any hints? Also an unrelated note/question: I see tons of pull requests being accepted without a JIRA but the documentation says to create a JIRA issue first[1]. So I assume it's okay to just send pull requests? Thanks for your help. Cheers, Lars [1] <https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark> On Fri, Feb 28, 2014 at 6:41 PM, Lars Francke <lars.fran...@gmail.com> wrote: > Hey, > > so currently it doesn't work because of > <https://github.com/apache/spark/pull/6#issuecomment-36343187> > > IntelliJ reports a lot of warnings with default settings and I haven't > found a way to tell IntellJ to use different Hadoop versions yet. > mvn clean compile -Pyarn fails as well (compilation errror > > Your command works indeed. Default yarn version is 0.23.7 which > doesn't seem to work with the default 2.2.0 Hadoop version (anymore?) > > I was basically trying to follow the documentation: > <http://spark.incubator.apache.org/docs/latest/building-with-maven.html> > > mvn clean compile -Pyarn-alpha -Dhadoop.version=2.0.0-cdh4.5.0 > -Dyarn.version=2.0.0-cdh4.5.0 fails as well as does mvn clean compile > -Pyarn-alpha > > Thanks for showing me a configuration that works. Unfortunately the > default ones and at least one of the documented ones fail. > > Cheers, > Lars > > > On Fri, Feb 28, 2014 at 3:05 PM, Tom Graves <tgraves...@yahoo.com> wrote: >> what build command are you using? What do you mean when you say YARN >> branch? >> >> The yarn builds have been working fine for me with maven. Build command I >> use against hadoop 2.2 or higher: mvn -Dyarn.version=2.2.0 >> -Dhadoop.version=2.2.0 -Pyarn clean package -DskipTests >> >> Tom >> >> >> >> On Friday, February 28, 2014 6:14 AM, Lars Francke <lars.fran...@gmail.com> >> wrote: >> >> Hey, >> >> I'm trying to dig into Spark's code but am running into a couple of problems. >> >> 1) The yarn-common directory is not included in the Maven build >> causing things to fail because the dependency is missing. If I see the >> history correct it used to be a Maven module but is not anymore. >> >> 2) When I try to include the yarn-common directory in the build things >> start going bad. Compilation failures all over the place and I think >> there are some dependency issues in there as well. >> >> This leads me to believe that either the Maven build system isn't >> maintained for YARN or the whole YARN branch isn't. What's the status >> here? >> >> Without YARN things build fine for me using Maven. >> >> Thanks for your help. >> >> Cheers, >> Lars
