Felix Schüler created TOREE-375:
-----------------------------------
Summary: Incorrect fully qualified name for spark context
Key: TOREE-375
URL: https://issues.apache.org/jira/browse/TOREE-375
Project: TOREE
Issue Type: Bug
Environment: Jupyter Notebook with Toree latest master
(1a9c11f5f1381c15b691a716acd0e1f0432a9a35) and Spark 2.0.2, Scala 2.11
Reporter: Felix Schüler
When running below snippet in a cell I get a compile error for the MLContext
Constructor. Somehow the fully qualified name of the SparkContext gets messed
up.
The same does not happen when I start a Spark shell with the --jars command and
create the MLContext there.
Snippet (the systemml jar is build with the latest master of SystemML):
{code}
%addjar
file:///home/felix/repos/incubator-systemml/target/systemml-0.13.0-incubating-SNAPSHOT.jar
-f
import org.apache.sysml.api.mlcontext._
import org.apache.sysml.api.mlcontext.ScriptFactory._
val ml = new MLContext(sc)
Starting download from
file:///home/felix/repos/incubator-systemml/target/systemml-0.13.0-incubating-SNAPSHOT.jar
Finished download of systemml-0.13.0-incubating-SNAPSHOT.jar
Name: Compile Error
Message: <console>:25: error: overloaded method constructor MLContext with
alternatives:
(x$1:
org.apache.spark.api.java.JavaSparkContext)org.apache.sysml.api.mlcontext.MLContext
<and>
(x$1:
org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.SparkContext)org.apache.sysml.api.mlcontext.MLContext
cannot be applied to
(org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.SparkContext)
val ml = new MLContext(sc)
^
StackTrace:
{/code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
