Repository: zeppelin Updated Branches: refs/heads/master df2e77de9 -> d93fb7361
ZEPPELIN-1384. Spark interpreter binary compatibility to scala 2.10 / 2.11 broken ### What is this PR for? As described in ZEPPELIN-1384, we may hit the following error when running zeppelin on spark 1.6 if we build zeppelin this way ``` dev/change_scala_version.sh 2.11 mvn -DskipTests -Drat.skip=true -Pscala-2.11 -Pspark-2.0 -Dspark.version=2.0.0 -Phadoop-2.6 -Pyarn -Ppyspark -Psparkr clean package ``` ``` java.lang.NoSuchMethodError: scala.runtime.VolatileByteRef.create(B)Lscala/runtime/VolatileByteRef; at scala.xml.MetaData$.iterate$1(MetaData.scala:39) at scala.xml.MetaData$.normalize(MetaData.scala:45) at scala.xml.Elem.<init>(Elem.scala:99) at org.apache.spark.ui.jobs.StagePage$$anonfun$26.apply(StagePage.scala:57) at org.apache.spark.ui.jobs.StagePage$$anonfun$26.apply(StagePage.scala:55) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.immutable.List.foreach(List.scala:318) at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) at scala.collection.AbstractTraversable.map(Traversable.scala:105) at org.apache.spark.ui.jobs.StagePage.<init>(StagePage.scala:55) at org.apache.spark.ui.jobs.StagesTab.<init>(StagesTab.scala:34) at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:57) at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:195) at org.apache.spark.ui.SparkUI$.createLiveUI(SparkUI.scala:146) at org.apache.spark.SparkContext.<init>(SparkContext.scala:473) at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_1(SparkInterpreter.java:440) at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:354) at org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:137) at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:743) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93) ``` The root cause is that scala-xml is removed from scala 2.11 to a separate library, so here we have class conflict of scala-xml api. In this PR, I make the scope of scala-xml to be provided and also make the scope of scala-library to be provided although it will be override in `ZEPPELIN_HOME/spark/pom.xml` ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-1384 ### How should this be tested? Tested manually. Use the following command to build zeppelin and then run it on spark-1.6.2 ``` dev/change_scala_version.sh 2.11 mvn -DskipTests -Drat.skip=true -Pscala-2.11 -Pspark-2.0 -Dspark.version=2.0.0 -Phadoop-2.6 -Pyarn -Ppyspark -Psparkr clean package ``` ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjf...@apache.org> Closes #1377 from zjffdu/ZEPPELIN-1384 and squashes the following commits: eb07535 [Jeff Zhang] ZEPPELIN-1384. Spark interpreter binary compatibility to scala 2.10 / 2.11 broken Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/d93fb736 Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/d93fb736 Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/d93fb736 Branch: refs/heads/master Commit: d93fb7361d73d1cdbf6a551b3892d4dd8c69656d Parents: df2e77d Author: Jeff Zhang <zjf...@apache.org> Authored: Mon Aug 29 09:58:29 2016 +0800 Committer: Damien CORNEAU <cornead...@gmail.com> Committed: Thu Sep 1 16:21:45 2016 +0900 ---------------------------------------------------------------------- zeppelin-display/pom.xml | 2 ++ 1 file changed, 2 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/d93fb736/zeppelin-display/pom.xml ---------------------------------------------------------------------- diff --git a/zeppelin-display/pom.xml b/zeppelin-display/pom.xml index c30b061..7631d4f 100644 --- a/zeppelin-display/pom.xml +++ b/zeppelin-display/pom.xml @@ -81,6 +81,7 @@ <groupId>org.scala-lang</groupId> <artifactId>scala-library</artifactId> <version>${scala.version}</version> + <scope>provided</scope> </dependency> <dependency> @@ -99,6 +100,7 @@ <groupId>org.scala-lang.modules</groupId> <artifactId>scala-xml_${scala.binary.version}</artifactId> <version>1.0.2</version> + <scope>provided</scope> </dependency> </dependencies> </profile>