Hello,
I’m installing Apache Toree in our JupyterHub. But I can’t execute the example
https://github.com/apache/incubator-toree/blob/master/etc/examples/notebooks/magic-tutorial.ipynb
well. The below 2 errors occur. When installing, I haven’t met an error.
Can you have any solution?
1st Scala error:
[cid:image001.png@01D2EF72.9D9B0C20]
%%dataframe
df
Out;
ERROR: Job aborted due to stage failure: Task 2 in stage 11.0 failed 1 times,
most recent failure: Lost task 2.0 in stage 11.0 (TID 28, localhost, executor
driver): java.lang.ClassCastException: $line53.$read$$iw$$iw$DFRecord cannot be
cast to $line53.$read$$iw$$iw$DFRecord
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
Source)
at
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at
org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:389)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at
scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59)
at
scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104)
at
scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48)
at
scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310)
at scala.collection.AbstractIterator.to(Iterator.scala:1336)
at
scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302)
at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1336)
at
scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1336)
at
org.apache.spark.rdd.RDD$$anonfun$take$1$$anonfun$29.apply(RDD.scala:1353)
at
org.apache.spark.rdd.RDD$$anonfun$take$1$$anonfun$29.apply(RDD.scala:1353)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1944)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1944)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:99)
at
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Driver stacktrace:
%%dataframe [arguments]
DATAFRAME_CODE
DATAFRAME_CODE can be any numbered lines of code, as long as the
last line is a reference to a variable which is a DataFrame.
Option Description
------ -----------
--limit The number of records to return
(default: 10)
--output The type of the output: html, csv,
json (default: html)
(default: 10)
json (default: html)
2nd PySpark error;
%%PySpark
from operator import add
print(sc.parallelize(range(1, 100)).reduce(add))
Out;
Name: org.apache.toree.interpreter.broker.BrokerException
Message: Traceback (most recent call last):
File
"/tmp/kernel-PySpark-2e158b54-f8dc-483e-bd42-749a84637c05/pyspark_runner.py",
line 189, in <module>
eval(compiled_code)
File "<string>", line 1, in <module>
File
"/tmp/kernel-PySpark-2e158b54-f8dc-483e-bd42-749a84637c05/pyspark_runner.py",
line 107, in __getattr__
return self._jvm_kernel.__getattribute__(name)
AttributeError: 'JavaObject' object has no attribute 'magics'
StackTrace:
org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163)
org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163)
scala.Option.foreach(Option.scala:257)
org.apache.toree.interpreter.broker.BrokerState.markFailure(BrokerState.scala:162)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
py4j.Gateway.invoke(Gateway.java:280)
py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
py4j.commands.CallCommand.execute(CallCommand.java:79)
py4j.GatewayConnection.run(GatewayConnection.java:214)
java.lang.Thread.run(Thread.java:745)
