[
https://issues.apache.org/jira/browse/SPARK-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13987686#comment-13987686
]
Sean Owen commented on SPARK-1698:
----------------------------------
(Copying an earlier comment that went to the mailing list, but didn't make it
here:)
#1 and #2 are not relevant the issue of jar size. These can be problems in
general, but don't think there have been issues attributable to file clashes.
Shading has mechanisms to deal with this anyway.
#3 is a problem in general too, but is not specific to shading. Where versions
collide, build processes like Maven and shading must be used to resolve them.
But this happens regardless of whether you shade a fat jar.
#4 is a real problem specific to Java 6. It does seem like it will be important
to identify and remove more unnecessary dependencies to work around it.
But shading per se is not the problem, and it is important to make a packaged
jar for the app. What are you proposing? Dependencies to be removed?
> Improve spark integration
> -------------------------
>
> Key: SPARK-1698
> URL: https://issues.apache.org/jira/browse/SPARK-1698
> Project: Spark
> Issue Type: Improvement
> Components: Build, Deploy
> Reporter: Guoqiang Li
> Assignee: Guoqiang Li
> Fix For: 1.0.0
>
>
> Use the shade plugin to create a big JAR with all the dependencies can cause
> a few problems
> 1. Missing jar's meta information
> 2. Some file is covered, eg: plugin.xml
> 3. Different versions of the jar may co-exist
> 4. Too big, java 6 does not support
--
This message was sent by Atlassian JIRA
(v6.2#6252)
