Eugene Sapozhnikov created ZEPPELIN-301:
-------------------------------------------
Summary: z.load for "com.amazonaws:aws-java-sdk*:*" breaks spark
interpreter
Key: ZEPPELIN-301
URL: https://issues.apache.org/jira/browse/ZEPPELIN-301
Project: Zeppelin
Issue Type: Bug
Components: Core, Interpreters
Affects Versions: 0.6.0
Environment: Linux ip-10-228-34-138 3.4.62-53.42.amzn1.x86_64 #1 SMP
Fri Sep 20 07:23:24 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
Java(TM) SE Runtime Environment (build 1.7.0_21-b11)
Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)
Installed and build with:
git clone https://github.com/apache/incubator-zeppelin.git zeppelin
mvn clean package -Pspark-1.4 -Dhadoop.version=2.6.0 -Phadoop-2.6 -Pyarn
-DskipTests
No changes in zeppelin/conf directory
Reporter: Eugene Sapozhnikov
Priority: Blocker
z.load(...) breaks something in spark interpreter. I get
"error: type mismatch" when rdds are used after z.load()
Steps to reproduce:
1. Install zeppelin as described in environment.
2. Create notebook
3. Create paragraphs P1 and P2 execute them
P1:
import org.apache.spark.rdd.RDD
var inputLinesRDD_raw:RDD[String] = sc.emptyRDD[String]
-----------OUT--------
import org.apache.spark.rdd.RDD
inputLinesRDD_raw: org.apache.spark.rdd.RDD[String] = EmptyRDD[0] at emptyRDD
at <console>:24
------IT'S OKAY-------
P2:
z.load("com.amazonaws:aws-java-sdk-s3:1.10.17")
import org.apache.spark.rdd.RDD
var inputLinesRDD_raw:RDD[String] = sc.emptyRDD[String]
-----------OUT--------
res1: Iterable[String] = Wrappers(com.amazonaws:aws-java-sdk-s3:1.10.17,
com.amazonaws:aws-java-sdk-kms:1.10.17,
com.amazonaws:aws-java-sdk-core:1.10.17, commons-logging:commons-logging:1.1.3,
org.apache.httpcomponents:httpclient:4.3.6,
org.apache.httpcomponents:httpcore:4.3.3, commons-codec:commons-codec:1.6,
com.fasterxml.jackson.core:jackson-databind:2.5.3,
com.fasterxml.jackson.core:jackson-annotations:2.5.0,
com.fasterxml.jackson.core:jackson-core:2.5.3, joda-time:joda-time:2.8.1)
import org.apache.spark.rdd.RDD
<console>:26: error: type mismatch;
found : org.apache.spark.rdd.EmptyRDD[String]
required: org.apache.spark.rdd.RDD[String]
var inputLinesRDD_raw:RDD[String] = sc.emptyRDD[String]
^
------IT'S UNEXPECTED-------
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
