Repository: zeppelin Updated Branches: refs/heads/master 1ccbd602a -> 6a82e7aab
[ZEPPELIN-1099] Build and run Spark without spark-dependencies ### What is this PR for? When users use SPARK_HOME, they won't want to include spark-dependencies. This PR makes it possible and results reducing whole size of package. ### What type of PR is it? [Feature] ### Todos * [x] - Remove dependencies of spark-dependencies ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-1099 ### How should this be tested? * Run `mvn clean package -DskipTests -pl '!spark-dependencies'` * Set SPARK_HOME in zeppelin-env.sh * Start Zeppelin * Run `sc.version` in a paragraph and check the version * Stop Zeppelin * Set SPARK_HOME in a different version * Start, check and stop I've tested it with 1.6.1 and 1.5.2. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jongyoul Lee <[email protected]> Closes #1115 from jongyoul/ZEPPELIN-1099 and squashes the following commits: 192129f [Jongyoul Lee] Removed spark-dependencies from spark Added spark-{reple,hive} in spark/pom.xml Changed exception to throwable because of Error Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/6a82e7aa Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/6a82e7aa Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/6a82e7aa Branch: refs/heads/master Commit: 6a82e7aabcabcf1de1ec197ebe5a76c36d2934c4 Parents: 1ccbd60 Author: Jongyoul Lee <[email protected]> Authored: Sun Jul 3 01:44:53 2016 +0900 Committer: Jongyoul Lee <[email protected]> Committed: Sun Jul 10 14:47:46 2016 +0900 ---------------------------------------------------------------------- spark/pom.xml | 15 +++++++++++---- .../zeppelin/interpreter/InterpreterFactory.java | 2 +- 2 files changed, 12 insertions(+), 5 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/6a82e7aa/spark/pom.xml ---------------------------------------------------------------------- diff --git a/spark/pom.xml b/spark/pom.xml index 28770f8..3a8a0fe 100644 --- a/spark/pom.xml +++ b/spark/pom.xml @@ -38,7 +38,7 @@ <jsoup.version>1.8.2</jsoup.version> <mockito.version>1.10.19</mockito.version> <powermock.version>1.6.4</powermock.version> - <spark.version>1.4.1</spark.version> + <spark.version>1.6.2</spark.version> <scala.version>2.10.4</scala.version> <scala.binary.version>2.10</scala.binary.version> </properties> @@ -67,9 +67,16 @@ </dependency> <dependency> - <groupId>${project.groupId}</groupId> - <artifactId>zeppelin-spark-dependencies</artifactId> - <version>${project.version}</version> + <groupId>org.apache.spark</groupId> + <artifactId>spark-repl_${scala.binary.version}</artifactId> + <version>${spark.version}</version> + <scope>provided</scope> + </dependency> + + <dependency> + <groupId>org.apache.spark</groupId> + <artifactId>spark-hive_${scala.binary.version}</artifactId> + <version>${spark.version}</version> <scope>provided</scope> </dependency> http://git-wip-us.apache.org/repos/asf/zeppelin/blob/6a82e7aa/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterFactory.java ---------------------------------------------------------------------- diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterFactory.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterFactory.java index b53c2c6..3fed07f 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterFactory.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterFactory.java @@ -168,7 +168,7 @@ public class InterpreterFactory implements InterpreterGroupFactory { cleanCl.put(interpreterDirString, ccl); } } - } catch (ClassNotFoundException e) { + } catch (Throwable t) { // nothing to do } }
