This has been around for multiple versions of Spark, so I am a bit surprised to see it not working in your build.
- Patrick On Wed, Feb 25, 2015 at 9:41 AM, Patrick Wendell <pwend...@gmail.com> wrote: > Hey Cody, > > What build command are you using? In any case, we can actually comment > out the "unused" thing now in the root pom.xml. It existed just to > ensure that at least one dependency was listed in the shade plugin > configuration (otherwise, some work we do that requires the shade > plugin does not happen). However, now there are other things there. If > you just comment out the line in the root pom.xml adding this > dependency, does it work? > > - Patrick > > On Wed, Feb 25, 2015 at 7:53 AM, Cody Koeninger <c...@koeninger.org> wrote: >> So when building 1.3.0-rc1 I see the following warning: >> >> [WARNING] spark-streaming-kafka_2.10-1.3.0.jar, unused-1.0.0.jar define 1 >> overlappping classes: >> >> [WARNING] - org.apache.spark.unused.UnusedStubClass >> >> >> and when trying to build an assembly of a project that was previously using >> 1.3 snapshots without difficulty, I see the following errors: >> >> >> [error] (*:assembly) deduplicate: different file contents found in the >> following: >> >> [error] >> /Users/cody/.m2/repository/org/apache/spark/spark-streaming-kafka_2.10/1.3.0/spark-streaming-kafka_2.10-1.3.0.jar:org/apache/spark/unused/UnusedStubClass.class >> >> [error] >> /Users/cody/.m2/repository/org/spark-project/spark/unused/1.0.0/unused-1.0.0.jar:org/apache/spark/unused/UnusedStubClass.class >> >> >> This persists even after a clean / rebuild of both 1.3.0-rc1 and the >> project using it. >> >> >> I can just exclude that jar in the assembly definition, but is anyone else >> seeing similar issues? If so, might be worth resolving rather than make >> users mess with assembly exclusions. >> >> I see that this class was introduced a while ago, related to SPARK-3812 but >> the jira issue doesn't have much detail. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
