Repository: zeppelin
Updated Branches:
refs/heads/branch-0.6 7b7216338 -> 9a28d31f7
ZEPPELIN-1384. Spark interpreter binary compatibility to scala 2.10 / 2.11
broken
### What is this PR for?
As described in ZEPPELIN-1384, we may hit the following error when running
zeppelin on spark 1.6 if we build zeppelin this way
```
dev/change_scala_version.sh 2.11
mvn -DskipTests -Drat.skip=true -Pscala-2.11 -Pspark-2.0 -Dspark.version=2.0.0
-Phadoop-2.6 -Pyarn -Ppyspark -Psparkr clean package
```
```
java.lang.NoSuchMethodError:
scala.runtime.VolatileByteRef.create(B)Lscala/runtime/VolatileByteRef;
at scala.xml.MetaData$.iterate$1(MetaData.scala:39)
at scala.xml.MetaData$.normalize(MetaData.scala:45)
at scala.xml.Elem.<init>(Elem.scala:99)
at
org.apache.spark.ui.jobs.StagePage$$anonfun$26.apply(StagePage.scala:57)
at
org.apache.spark.ui.jobs.StagePage$$anonfun$26.apply(StagePage.scala:55)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.List.foreach(List.scala:318)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at org.apache.spark.ui.jobs.StagePage.<init>(StagePage.scala:55)
at org.apache.spark.ui.jobs.StagesTab.<init>(StagesTab.scala:34)
at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:57)
at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:195)
at org.apache.spark.ui.SparkUI$.createLiveUI(SparkUI.scala:146)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:473)
at
org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_1(SparkInterpreter.java:440)
at
org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:354)
at
org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:137)
at
org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:743)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
```
The root cause is that scala-xml is removed from scala 2.11 to a separate
library, so here we have class conflict of scala-xml api. In this PR, I make
the scope of scala-xml to be provided and also make the scope of scala-library
to be provided although it will be override in `ZEPPELIN_HOME/spark/pom.xml`
### What type of PR is it?
[Bug Fix]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1384
### How should this be tested?
Tested manually.
Use the following command to build zeppelin and then run it on spark-1.6.2
```
dev/change_scala_version.sh 2.11
mvn -DskipTests -Drat.skip=true -Pscala-2.11 -Pspark-2.0 -Dspark.version=2.0.0
-Phadoop-2.6 -Pyarn -Ppyspark -Psparkr clean package
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <[email protected]>
Closes #1377 from zjffdu/ZEPPELIN-1384 and squashes the following commits:
eb07535 [Jeff Zhang] ZEPPELIN-1384. Spark interpreter binary compatibility to
scala 2.10 / 2.11 broken
(cherry picked from commit d93fb7361d73d1cdbf6a551b3892d4dd8c69656d)
Signed-off-by: Damien CORNEAU <[email protected]>
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/9a28d31f
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/9a28d31f
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/9a28d31f
Branch: refs/heads/branch-0.6
Commit: 9a28d31f78de45ecef639e813fb101326cf38107
Parents: 7b72163
Author: Jeff Zhang <[email protected]>
Authored: Mon Aug 29 09:58:29 2016 +0800
Committer: Damien CORNEAU <[email protected]>
Committed: Thu Sep 1 16:22:04 2016 +0900
----------------------------------------------------------------------
zeppelin-display/pom.xml | 2 ++
1 file changed, 2 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/9a28d31f/zeppelin-display/pom.xml
----------------------------------------------------------------------
diff --git a/zeppelin-display/pom.xml b/zeppelin-display/pom.xml
index 879570e..e9f3c7c 100644
--- a/zeppelin-display/pom.xml
+++ b/zeppelin-display/pom.xml
@@ -81,6 +81,7 @@
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.version}</version>
+ <scope>provided</scope>
</dependency>
<dependency>
@@ -99,6 +100,7 @@
<groupId>org.scala-lang.modules</groupId>
<artifactId>scala-xml_${scala.binary.version}</artifactId>
<version>1.0.2</version>
+ <scope>provided</scope>
</dependency>
</dependencies>
</profile>
