I'm building with mvn -Phadoop-2.4 -DskipTests install
Yeah, commenting out the unused dependency in the root pom.xml resolves it. I'm a little surprised that it cropped up now as well, I had built against multiple different snapshots of 1.3 over the past couple weeks with no problems. Is it worth me putting in a PR to remove that class and the dependency altogether? On Wed, Feb 25, 2015 at 11:42 AM, Patrick Wendell <pwend...@gmail.com> wrote: > This has been around for multiple versions of Spark, so I am a bit > surprised to see it not working in your build. > > - Patrick > > On Wed, Feb 25, 2015 at 9:41 AM, Patrick Wendell <pwend...@gmail.com> > wrote: > > Hey Cody, > > > > What build command are you using? In any case, we can actually comment > > out the "unused" thing now in the root pom.xml. It existed just to > > ensure that at least one dependency was listed in the shade plugin > > configuration (otherwise, some work we do that requires the shade > > plugin does not happen). However, now there are other things there. If > > you just comment out the line in the root pom.xml adding this > > dependency, does it work? > > > > - Patrick > > > > On Wed, Feb 25, 2015 at 7:53 AM, Cody Koeninger <c...@koeninger.org> > wrote: > >> So when building 1.3.0-rc1 I see the following warning: > >> > >> [WARNING] spark-streaming-kafka_2.10-1.3.0.jar, unused-1.0.0.jar define > 1 > >> overlappping classes: > >> > >> [WARNING] - org.apache.spark.unused.UnusedStubClass > >> > >> > >> and when trying to build an assembly of a project that was previously > using > >> 1.3 snapshots without difficulty, I see the following errors: > >> > >> > >> [error] (*:assembly) deduplicate: different file contents found in the > >> following: > >> > >> [error] > >> > /Users/cody/.m2/repository/org/apache/spark/spark-streaming-kafka_2.10/1.3.0/spark-streaming-kafka_2.10-1.3.0.jar:org/apache/spark/unused/UnusedStubClass.class > >> > >> [error] > >> > /Users/cody/.m2/repository/org/spark-project/spark/unused/1.0.0/unused-1.0.0.jar:org/apache/spark/unused/UnusedStubClass.class > >> > >> > >> This persists even after a clean / rebuild of both 1.3.0-rc1 and the > >> project using it. > >> > >> > >> I can just exclude that jar in the assembly definition, but is anyone > else > >> seeing similar issues? If so, might be worth resolving rather than make > >> users mess with assembly exclusions. > >> > >> I see that this class was introduced a while ago, related to SPARK-3812 > but > >> the jira issue doesn't have much detail. >
