Re: is Branch-1.1 SBT build broken for yarn-alpha ?
Maven is just telling you that there is no version 1.1.0 of yarn-parent, and indeed, it has not been released. To build the branch you would need to mvn install to compile and make available local copies of artifacts along the way. (You may have these for 1.1.0-SNAPSHOT locally already). Use Maven, not SBT, for building releases. On Thu, Aug 21, 2014 at 12:39 AM, Chester Chen ches...@alpinenow.com wrote: *[FATAL] Non-resolvable parent POM: Could not find artifact org.apache.spark:yarn-parent_2.10:pom:1.1.0 in central ( http://repo.maven.apache.org/maven2 http://repo.maven.apache.org/maven2) and 'parent.relativePath' points at wrong local POM @ line 20, column 11* - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: is Branch-1.1 SBT build broken for yarn-alpha ?
Weird that Patrick did not face this while creating the RC. Essentially the yarn alpha pom.xml has not been updated properly in the 1.1 branch. Just change version to '1.1.1-SNAPSHOT' for yarn/alpha/pom.xml (to make it same as any other pom). Regards, Mridul On Thu, Aug 21, 2014 at 5:09 AM, Chester Chen ches...@alpinenow.com wrote: I just updated today's build and tried branch-1.1 for both yarn and yarn-alpha. For yarn build, this command seem to work fine. sbt/sbt -Pyarn -Dhadoop.version=2.3.0-cdh5.0.1 projects for yarn-alpha sbt/sbt -Pyarn-alpha -Dhadoop.version=2.0.5-alpha projects I got the following Any ideas Chester ᚛ |branch-1.1|$ *sbt/sbt -Pyarn-alpha -Dhadoop.version=2.0.5-alpha projects* Using /Library/Java/JavaVirtualMachines/1.6.0_51-b11-457.jdk/Contents/Home as default JAVA_HOME. Note, this will be overridden by -java-home if it is set. [info] Loading project definition from /Users/chester/projects/spark/project/project [info] Loading project definition from /Users/chester/.sbt/0.13/staging/ec3aa8f39111944cc5f2/sbt-pom-reader/project [warn] Multiple resolvers having different access mechanism configured with same name 'sbt-plugin-releases'. To avoid conflict, Remove duplicate project resolvers (`resolvers`) or rename publishing resolver (`publishTo`). [info] Loading project definition from /Users/chester/projects/spark/project org.apache.maven.model.building.ModelBuildingException: 1 problem was encountered while building the effective model for org.apache.spark:spark-yarn-alpha_2.10:1.1.0 *[FATAL] Non-resolvable parent POM: Could not find artifact org.apache.spark:yarn-parent_2.10:pom:1.1.0 in central ( http://repo.maven.apache.org/maven2 http://repo.maven.apache.org/maven2) and 'parent.relativePath' points at wrong local POM @ line 20, column 11* at org.apache.maven.model.building.DefaultModelProblemCollector.newModelBuildingException(DefaultModelProblemCollector.java:195) at org.apache.maven.model.building.DefaultModelBuilder.readParentExternally(DefaultModelBuilder.java:841) at org.apache.maven.model.building.DefaultModelBuilder.readParent(DefaultModelBuilder.java:664) at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:310) at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:232) at com.typesafe.sbt.pom.MvnPomResolver.loadEffectivePom(MavenPomResolver.scala:61) at com.typesafe.sbt.pom.package$.loadEffectivePom(package.scala:41) at com.typesafe.sbt.pom.MavenProjectHelper$.makeProjectTree(MavenProjectHelper.scala:128) at com.typesafe.sbt.pom.MavenProjectHelper$$anonfun$12.apply(MavenProjectHelper.scala:129) at com.typesafe.sbt.pom.MavenProjectHelper$$anonfun$12.apply(MavenProjectHelper.scala:129) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) at scala.collection.AbstractTraversable.map(Traversable.scala:105) at com.typesafe.sbt.pom.MavenProjectHelper$.makeProjectTree(MavenProjectHelper.scala:129) at com.typesafe.sbt.pom.MavenProjectHelper$$anonfun$12.apply(MavenProjectHelper.scala:129) at com.typesafe.sbt.pom.MavenProjectHelper$$anonfun$12.apply(MavenProjectHelper.scala:129) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) at scala.collection.AbstractTraversable.map(Traversable.scala:105) at com.typesafe.sbt.pom.MavenProjectHelper$.makeProjectTree(MavenProjectHelper.scala:129) at com.typesafe.sbt.pom.MavenProjectHelper$.makeReactorProject(MavenProjectHelper.scala:49) at com.typesafe.sbt.pom.PomBuild$class.projectDefinitions(PomBuild.scala:28) at SparkBuild$.projectDefinitions(SparkBuild.scala:165) at sbt.Load$.sbt$Load$$projectsFromBuild(Load.scala:458) at sbt.Load$$anonfun$24.apply(Load.scala:415) at sbt.Load$$anonfun$24.apply(Load.scala:415) at scala.collection.immutable.Stream.flatMap(Stream.scala:442) at sbt.Load$.loadUnit(Load.scala:415) at sbt.Load$$anonfun$15$$anonfun$apply$11.apply(Load.scala:256) at sbt.Load$$anonfun$15$$anonfun$apply$11.apply(Load.scala:256) at sbt.BuildLoader$$anonfun$componentLoader$1$$anonfun$apply$4$$anonfun$apply$5$$anonfun$apply$6.apply(BuildLoader.scala:93) at
Re: is Branch-1.1 SBT build broken for yarn-alpha ?
Do we have Jenkins tests these ? Should be pretty easy to setup just to test basic build Sent from my iPhone On Aug 21, 2014, at 6:45 AM, Mridul Muralidharan mri...@gmail.com wrote: Weird that Patrick did not face this while creating the RC. Essentially the yarn alpha pom.xml has not been updated properly in the 1.1 branch. Just change version to '1.1.1-SNAPSHOT' for yarn/alpha/pom.xml (to make it same as any other pom). Regards, Mridul On Thu, Aug 21, 2014 at 5:09 AM, Chester Chen ches...@alpinenow.com wrote: I just updated today's build and tried branch-1.1 for both yarn and yarn-alpha. For yarn build, this command seem to work fine. sbt/sbt -Pyarn -Dhadoop.version=2.3.0-cdh5.0.1 projects for yarn-alpha sbt/sbt -Pyarn-alpha -Dhadoop.version=2.0.5-alpha projects I got the following Any ideas Chester ᚛ |branch-1.1|$ *sbt/sbt -Pyarn-alpha -Dhadoop.version=2.0.5-alpha projects* Using /Library/Java/JavaVirtualMachines/1.6.0_51-b11-457.jdk/Contents/Home as default JAVA_HOME. Note, this will be overridden by -java-home if it is set. [info] Loading project definition from /Users/chester/projects/spark/project/project [info] Loading project definition from /Users/chester/.sbt/0.13/staging/ec3aa8f39111944cc5f2/sbt-pom-reader/project [warn] Multiple resolvers having different access mechanism configured with same name 'sbt-plugin-releases'. To avoid conflict, Remove duplicate project resolvers (`resolvers`) or rename publishing resolver (`publishTo`). [info] Loading project definition from /Users/chester/projects/spark/project org.apache.maven.model.building.ModelBuildingException: 1 problem was encountered while building the effective model for org.apache.spark:spark-yarn-alpha_2.10:1.1.0 *[FATAL] Non-resolvable parent POM: Could not find artifact org.apache.spark:yarn-parent_2.10:pom:1.1.0 in central ( http://repo.maven.apache.org/maven2 http://repo.maven.apache.org/maven2) and 'parent.relativePath' points at wrong local POM @ line 20, column 11* at org.apache.maven.model.building.DefaultModelProblemCollector.newModelBuildingException(DefaultModelProblemCollector.java:195) at org.apache.maven.model.building.DefaultModelBuilder.readParentExternally(DefaultModelBuilder.java:841) at org.apache.maven.model.building.DefaultModelBuilder.readParent(DefaultModelBuilder.java:664) at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:310) at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:232) at com.typesafe.sbt.pom.MvnPomResolver.loadEffectivePom(MavenPomResolver.scala:61) at com.typesafe.sbt.pom.package$.loadEffectivePom(package.scala:41) at com.typesafe.sbt.pom.MavenProjectHelper$.makeProjectTree(MavenProjectHelper.scala:128) at com.typesafe.sbt.pom.MavenProjectHelper$$anonfun$12.apply(MavenProjectHelper.scala:129) at com.typesafe.sbt.pom.MavenProjectHelper$$anonfun$12.apply(MavenProjectHelper.scala:129) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) at scala.collection.AbstractTraversable.map(Traversable.scala:105) at com.typesafe.sbt.pom.MavenProjectHelper$.makeProjectTree(MavenProjectHelper.scala:129) at com.typesafe.sbt.pom.MavenProjectHelper$$anonfun$12.apply(MavenProjectHelper.scala:129) at com.typesafe.sbt.pom.MavenProjectHelper$$anonfun$12.apply(MavenProjectHelper.scala:129) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) at scala.collection.AbstractTraversable.map(Traversable.scala:105) at com.typesafe.sbt.pom.MavenProjectHelper$.makeProjectTree(MavenProjectHelper.scala:129) at com.typesafe.sbt.pom.MavenProjectHelper$.makeReactorProject(MavenProjectHelper.scala:49) at com.typesafe.sbt.pom.PomBuild$class.projectDefinitions(PomBuild.scala:28) at SparkBuild$.projectDefinitions(SparkBuild.scala:165) at sbt.Load$.sbt$Load$$projectsFromBuild(Load.scala:458) at sbt.Load$$anonfun$24.apply(Load.scala:415) at sbt.Load$$anonfun$24.apply(Load.scala:415) at scala.collection.immutable.Stream.flatMap(Stream.scala:442) at sbt.Load$.loadUnit(Load.scala:415) at sbt.Load$$anonfun$15$$anonfun$apply$11.apply(Load.scala:256) at
Re: is Branch-1.1 SBT build broken for yarn-alpha ?
Just tried on master branch, and the master branch works fine for yarn-alpha On Wed, Aug 20, 2014 at 4:39 PM, Chester Chen ches...@alpinenow.com wrote: I just updated today's build and tried branch-1.1 for both yarn and yarn-alpha. For yarn build, this command seem to work fine. sbt/sbt -Pyarn -Dhadoop.version=2.3.0-cdh5.0.1 projects for yarn-alpha sbt/sbt -Pyarn-alpha -Dhadoop.version=2.0.5-alpha projects I got the following Any ideas Chester ᚛ |branch-1.1|$ *sbt/sbt -Pyarn-alpha -Dhadoop.version=2.0.5-alpha projects* Using /Library/Java/JavaVirtualMachines/1.6.0_51-b11-457.jdk/Contents/Home as default JAVA_HOME. Note, this will be overridden by -java-home if it is set. [info] Loading project definition from /Users/chester/projects/spark/project/project [info] Loading project definition from /Users/chester/.sbt/0.13/staging/ec3aa8f39111944cc5f2/sbt-pom-reader/project [warn] Multiple resolvers having different access mechanism configured with same name 'sbt-plugin-releases'. To avoid conflict, Remove duplicate project resolvers (`resolvers`) or rename publishing resolver (`publishTo`). [info] Loading project definition from /Users/chester/projects/spark/project org.apache.maven.model.building.ModelBuildingException: 1 problem was encountered while building the effective model for org.apache.spark:spark-yarn-alpha_2.10:1.1.0 *[FATAL] Non-resolvable parent POM: Could not find artifact org.apache.spark:yarn-parent_2.10:pom:1.1.0 in central ( http://repo.maven.apache.org/maven2 http://repo.maven.apache.org/maven2) and 'parent.relativePath' points at wrong local POM @ line 20, column 11* at org.apache.maven.model.building.DefaultModelProblemCollector.newModelBuildingException(DefaultModelProblemCollector.java:195) at org.apache.maven.model.building.DefaultModelBuilder.readParentExternally(DefaultModelBuilder.java:841) at org.apache.maven.model.building.DefaultModelBuilder.readParent(DefaultModelBuilder.java:664) at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:310) at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:232) at com.typesafe.sbt.pom.MvnPomResolver.loadEffectivePom(MavenPomResolver.scala:61) at com.typesafe.sbt.pom.package$.loadEffectivePom(package.scala:41) at com.typesafe.sbt.pom.MavenProjectHelper$.makeProjectTree(MavenProjectHelper.scala:128) at com.typesafe.sbt.pom.MavenProjectHelper$$anonfun$12.apply(MavenProjectHelper.scala:129) at com.typesafe.sbt.pom.MavenProjectHelper$$anonfun$12.apply(MavenProjectHelper.scala:129) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) at scala.collection.AbstractTraversable.map(Traversable.scala:105) at com.typesafe.sbt.pom.MavenProjectHelper$.makeProjectTree(MavenProjectHelper.scala:129) at com.typesafe.sbt.pom.MavenProjectHelper$$anonfun$12.apply(MavenProjectHelper.scala:129) at com.typesafe.sbt.pom.MavenProjectHelper$$anonfun$12.apply(MavenProjectHelper.scala:129) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) at scala.collection.AbstractTraversable.map(Traversable.scala:105) at com.typesafe.sbt.pom.MavenProjectHelper$.makeProjectTree(MavenProjectHelper.scala:129) at com.typesafe.sbt.pom.MavenProjectHelper$.makeReactorProject(MavenProjectHelper.scala:49) at com.typesafe.sbt.pom.PomBuild$class.projectDefinitions(PomBuild.scala:28) at SparkBuild$.projectDefinitions(SparkBuild.scala:165) at sbt.Load$.sbt$Load$$projectsFromBuild(Load.scala:458) at sbt.Load$$anonfun$24.apply(Load.scala:415) at sbt.Load$$anonfun$24.apply(Load.scala:415) at scala.collection.immutable.Stream.flatMap(Stream.scala:442) at sbt.Load$.loadUnit(Load.scala:415) at sbt.Load$$anonfun$15$$anonfun$apply$11.apply(Load.scala:256) at sbt.Load$$anonfun$15$$anonfun$apply$11.apply(Load.scala:256) at sbt.BuildLoader$$anonfun$componentLoader$1$$anonfun$apply$4$$anonfun$apply$5$$anonfun$apply$6.apply(BuildLoader.scala:93) at sbt.BuildLoader$$anonfun$componentLoader$1$$anonfun$apply$4$$anonfun$apply$5$$anonfun$apply$6.apply(BuildLoader.scala:92) at sbt.BuildLoader.apply(BuildLoader.scala:143) at sbt.Load$.loadAll(Load.scala:312) at sbt.Load$.loadURI(Load.scala:264) at sbt.Load$.load(Load.scala:260)
