Thanks Arush. I did look into the dependency tree but couldn't figure which dependency was bringing the wrong Hadoop-yarn-common in.
I'll try he quick fix first. Sarwar On Thu, 29 Jan 2015 at 09:33 Arush Kharbanda <[email protected]> wrote: > Hi Sarwar, > > For a quick fix you can exclude dependencies for yarn(you wont be needing > them if you are running locally). > > libraryDependencies += > "log4j" % "log4j" % "1.2.15" exclude("javax.jms", "jms") > > > You can also analyze your dependencies using this plugin > > https://github.com/jrudolph/sbt-dependency-graph > > > Thanks > Arush > > On Thu, Jan 29, 2015 at 4:20 AM, Sarwar Bhuiyan <[email protected]> > wrote: > >> Hello all, >> >> I'm trying to build the sample application on the spark 1.2.0 quickstart >> page (https://spark.apache.org/docs/latest/quick-start.html) using the >> following build.sbt file: >> >> name := "Simple Project" >> >> version := "1.0" >> >> scalaVersion := "2.10.4" >> >> libraryDependencies += "org.apache.spark" %% "spark-core" % "1.2.0" >> >> Upon calling sbt package, it downloaded a lot of dependencies but >> eventually failed with some warnings and errors. Here's the snippet: >> >> [warn] :::::::::::::::::::::::::::::::::::::::::::::: >> [warn] :: UNRESOLVED DEPENDENCIES :: >> [warn] :::::::::::::::::::::::::::::::::::::::::::::: >> [warn] :: org.apache.hadoop#hadoop-yarn-common;1.0.4: not found >> [warn] :: org.apache.hadoop#hadoop-yarn-client;1.0.4: not found >> [warn] :: org.apache.hadoop#hadoop-yarn-api;1.0.4: not found >> [warn] :::::::::::::::::::::::::::::::::::::::::::::: >> [warn] :::::::::::::::::::::::::::::::::::::::::::::: >> [warn] :: FAILED DOWNLOADS :: >> [warn] :: ^ see resolution messages for details ^ :: >> [warn] :::::::::::::::::::::::::::::::::::::::::::::: >> [warn] :: >> org.eclipse.jetty.orbit#javax.transaction;1.1.1.v201105210645!javax.transaction.orbit >> [warn] :: >> org.eclipse.jetty.orbit#javax.servlet;3.0.0.v201112011016!javax.servlet.orbit >> [warn] :: >> org.eclipse.jetty.orbit#javax.mail.glassfish;1.4.1.v201005082020!javax.mail.glassfish.orbit >> [warn] :: >> org.eclipse.jetty.orbit#javax.activation;1.1.0.v201105071233!javax.activation.orbit >> [warn] :::::::::::::::::::::::::::::::::::::::::::::: >> >> >> >> Upon checking the maven repositories there doesn't seem to be any >> hadoop-yarn-common 1.0.4. I've tried explicitly setting a dependency to >> hadoop-yarn-common 2.4.0 for example but to no avail. I've also tried >> setting a number of different repositories to see if maybe one of them >> might have that dependency. Still no dice. >> >> What's the best way to resolve this for a quickstart situation? Do I have >> to set some sort of profile or environment variable which doesn't try to >> bring the 1.0.4 yarn version? >> >> Any help would be greatly appreciated. >> >> Sarwar >> >> > > > -- > > [image: Sigmoid Analytics] <http://htmlsig.com/www.sigmoidanalytics.com> > > *Arush Kharbanda* || Technical Teamlead > > [email protected] || www.sigmoidanalytics.com >
