Hi Sean, I confirmed that if I take the Spark 1.2.0 release (a428c446), undo the guava PR [1], and use -Dmaven.install.skip=false with the workflow above, the problem is fixed.
RJ [1] https://github.com/apache/spark/commit/c9f743957fa963bc1dbed7a44a346ffce1a45cf2#diff-6382f8428b13fa6082fa688178f3dbcc On Wed, Jan 14, 2015 at 2:59 PM, RJ Nowling <rnowl...@gmail.com> wrote: > Thanks, Sean. > > Yes, Spark is incorrectly copying the spark assembly jar to > com/google/guava in the maven repository. This is for the 1.2.0 release, > just to clarify. > > I reverted the patches that shade Guava and removed the parts disabling > the install plugin and it seemed to fix the issue. > > It seems that Spark poms are inheriting something from Guava. > > RJ > > On Wed, Jan 14, 2015 at 2:33 PM, Sean Owen <so...@cloudera.com> wrote: > >> Guava is shaded, although one class is left in its original package. >> This shouldn't have anything to do with Spark's package or namespace >> though. What are you saying is in com/google/guava? >> >> You can un-skip the install plugin with -Dmaven.install.skip=false >> >> On Wed, Jan 14, 2015 at 7:26 PM, RJ Nowling <rnowl...@gmail.com> wrote: >> > Hi all, >> > >> > I'm trying to upgrade some Spark RPMs from 1.1.0 to 1.2.0. As part of >> the >> > RPM process, we build Spark with Maven. With Spark 1.2.0, though, the >> > artifacts are placed in com/google/guava and there is no >> org/apache/spark. >> > >> > I saw that the pom.xml files had been modified to prevent the install >> > command and that the guava dependency was modified. Could someone who >> is >> > more familiar with the Spark maven files comment on what might be >> causing >> > this oddity? >> > >> > Thanks, >> > RJ >> > >> > We build Spark like so: >> > $ mvn -Phadoop-2.4 -Dmesos.version=0.20.0 -DskipTests clean package >> > >> > Then build a local Maven repo: >> > >> > mvn -Phadoop-2.4 \ >> > -Dmesos.version=0.20.0 \ >> > -DskipTests install:install-file \ >> > >> -Dfile=assembly/target/scala-2.10/spark-assembly-1.2.0-hadoop2.4.0.jar \ >> > -DcreateChecksum=true \ >> > -DgeneratePom=true \ >> > -DartifactId=spark-assembly_2.1.0 \ >> > -DlocalRepositoryPath=../maven2 >> > >
