[
https://issues.apache.org/jira/browse/SPARK-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13987682#comment-13987682
]
Guoqiang Li edited comment on SPARK-1698 at 5/5/14 6:26 AM:
------------------------------------------------------------
[~srowen]
About [SPARK-1681|https://issues.apache.org/jira/browse/SPARK-1681] there is
only one solution: The datanucleus jars is added to the CLASSPATH.
Well,there may be other better solution, but I didn't find it
I disagree with [PR 610|https://github.com/apache/spark/pull/610],It's not
perfect.
[The PR 598|https://github.com/apache/spark/pull/598] reference
[HADOOP-7939|https://issues.apache.org/jira/browse/HADOOP-7939],I think that is
better.
There is [another solution|https://github.com/witgo/spark/tree/standalone]
reference [Packaging a multimodule maven Spring app in a standalone
jar|http://ath3nd.wordpress.com/2013/12/25/packaging-a-multimodule-maven-spring-app-in-a-standalone-jar/].
But this involves [SI-6660 REPL: load transitive dependencies of JARs on
classpath|https://issues.scala-lang.org/browse/SI-6660]
was (Author: gq):
[~srowen]
About [SPARK-1681|https://issues.apache.org/jira/browse/SPARK-1681] there is
only one solution: The datanucleus jars is added to the CLASSPATH.
Well,there may be other better solution, but I didn't find it
I disagree with [PR 610|https://github.com/apache/spark/pull/610],It's not
perfect.
[The PR 598|https://github.com/apache/spark/pull/598] reference
[HADOOP-7939|https://issues.apache.org/jira/browse/HADOOP-7939],I think that is
better.
There is [another solution|https://github.com/witgo/spark/tree/standalone]
reference [Invalid or corrupt JAR File built by Maven shade
plugin|http://stackoverflow.com/questions/13021423/invalid-or-corrupt-jar-file-built-by-maven-shade-plugin].
But this involves [SI-6660 REPL: load transitive dependencies of JARs on
classpath|https://issues.scala-lang.org/browse/SI-6660]
> Improve spark integration
> -------------------------
>
> Key: SPARK-1698
> URL: https://issues.apache.org/jira/browse/SPARK-1698
> Project: Spark
> Issue Type: Improvement
> Components: Build, Deploy
> Reporter: Guoqiang Li
> Assignee: Guoqiang Li
> Fix For: 1.1.0
>
>
> Use the shade plugin to create a big JAR with all the dependencies can cause
> a few problems
> 1. Missing jar's meta information
> 2. Some file is covered, eg: plugin.xml
> 3. Different versions of a jar may co-exist
> 4. Too big, java 6 does not support
--
This message was sent by Atlassian JIRA
(v6.2#6252)
