Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/10780#discussion_r52161121
--- Diff: network/yarn/pom.xml ---
@@ -96,6 +113,61 @@
</execution>
</executions>
</plugin>
+
+ <!-- probes to validate that those dependencies which must be shaded
are -->
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-antrun-plugin</artifactId>
+ <executions>
+ <execution>
+ <phase>verify</phase>
+ <goals>
+ <goal>run</goal>
+ </goals>
+ <configuration>
+ <target>
+ <macrodef name="shaded">
+ <attribute name="resource"/>
+ <sequential>
+ <fail message="Not found ${shade}@{resource}">
+ <condition>
+ <not>
+ <resourceexists>
+ <zipentry zipfile="${shuffle.jar}"
name="${shade}@{resource}"/>
+ </resourceexists>
+ </not>
+ </condition>
+ </fail>
+ </sequential>
+ </macrodef>
+ <echo>Verifying dependency shading</echo>
+ <shaded
resource="com/fasterxml/jackson/core/JsonParser.class" />
+ <shaded
resource="com/fasterxml/jackson/annotation/JacksonAnnotation.class" />
+ <shaded
resource="com/fasterxml/jackson/databind/JsonSerializer.class" />
+ </target>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
</plugins>
</build>
+
+ <profiles>
+ <profile>
+ <id>leveldb-provided</id>
--- End diff --
I've just gone and cut out anything related to leveldb. The two versions
are in sync; I don't know any pressing reasons for upgrading it anywhere
(hadoop is in sync all the way up to trunk); when the time comes for upgrading,
hopefully the NM plugins will be in separate JVMs by then.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
