[
https://issues.apache.org/jira/browse/SPARK-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13987699#comment-13987699
]
Guoqiang Li edited comment on SPARK-1698 at 5/2/14 2:36 PM:
------------------------------------------------------------
[~srowen]
In [The PR 598|https://github.com/apache/spark/pull/598] #1,#2,#4 do not occur
and #3 is very easy to find.
The directory structure of spark similar to hadoop 2.3.0.
There are three subcomponents: core,examples,hive,The directory structure of
spark:
{code}
+- SPARK_HOME
| +- bin
| \- *
| +- sbin
| \- *
| +- RELEASE
| +- conf
| \- *
| +- python
| \- *
| \- share
| \- spark
| +- core
| +- lib
| \- *.jar
| +- spark-core*.jar
| +- spark-repl*.jar
| +- spark-yarn*.jar
| +- spark-bagel*.jar
| +- spark-graphx*.jar
| +- spark-sql*.jar
| +- spark-catalyst*.jar
| +- spark-mllib*.jar
| \- spark-streaming*.jar
| +- hive
| +- lib
| \- *.jar
| \- spark-hive*.jar
| +- examples
| +- lib
| \- *.jar
| \- spark-examples*.jar
{code}
was (Author: gq):
[~srowen]
In [The PR 598|https://github.com/apache/spark/pull/598] #1,#2,#4 do not occur
and #3 is very easy to find.
The directory structure of spark similar to hadoop 2.3.0.
There are three subcomponents: core,examples,hive,The directory structure of
spark:
{code}
+- SPARK_HOME
| +- bin
| \- *
| +- sbin
| \- *
| +- RELEASE
| +- conf
| \- *
| +- python
| \- *
| +- share
| \- spark
| +- core
| +- lib
| \- *.jar
| +- spark-core*.jar
| +- spark-repl*.jar
| +- spark-yarn*.jar
| +- spark-bagel*.jar
| +- spark-graphx*.jar
| +- spark-sql*.jar
| +- spark-catalyst*.jar
| +- spark-mllib*.jar
| \- spark-streaming*.jar
| +- hive
| +- lib
| \- *.jar
| \- spark-hive*.jar
| +- examples
| +- lib
| \- *.jar
| \- spark-examples*.jar
{code}
> Improve spark integration
> -------------------------
>
> Key: SPARK-1698
> URL: https://issues.apache.org/jira/browse/SPARK-1698
> Project: Spark
> Issue Type: Improvement
> Components: Build, Deploy
> Reporter: Guoqiang Li
> Assignee: Guoqiang Li
> Fix For: 1.0.0
>
>
> Use the shade plugin to create a big JAR with all the dependencies can cause
> a few problems
> 1. Missing jar's meta information
> 2. Some file is covered, eg: plugin.xml
> 3. Different versions of a jar may co-exist
> 4. Too big, java 6 does not support
--
This message was sent by Atlassian JIRA
(v6.2#6252)
