Re: Please check these 2 errors

2017-06-27 Thread Luciano Resende 
What Toree version and what Spark version are you using?

On Tue, Jun 27, 2017 at 2:25 AM, Sihyoung Jurn  wrote:

> Hello,
>
>
>
> I’m installing Apache Toree in our JupyterHub. But I can’t execute the
> example https://github.com/apache/incubator-toree/blob/master/
> etc/examples/notebooks/magic-tutorial.ipynb well. The below 2 errors
> occur. When installing, I haven’t met an error.
>
>
>
> Can you have any solution?
>
>
>
>
>
> 1st Scala error:
>
> %%dataframe
>
> df
>
>
>
> Out;
>
> ERROR: Job aborted due to stage failure: Task 2 in stage 11.0 failed 1
> times, most recent failure: Lost task 2.0 in stage 11.0 (TID 28, localhost,
> executor driver): java.lang.ClassCastException:
> $line53.$read$$iw$$iw$DFRecord cannot be cast to
> $line53.$read$$iw$$iw$DFRecord
>
>at org.apache.spark.sql.catalyst.expressions.GeneratedClass$
> GeneratedIterator.processNext(Unknown Source)
>
>at org.apache.spark.sql.execution.BufferedRowIterator.
> hasNext(BufferedRowIterator.java:43)
>
>at org.apache.spark.sql.execution.WholeStageCodegenExec$$
> anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)
>
>at scala.collection.Iterator$$anon$11.hasNext(Iterator.
> scala:408)
>
>at scala.collection.Iterator$$anon$11.hasNext(Iterator.
> scala:408)
>
>at scala.collection.Iterator$$anon$11.hasNext(Iterator.
> scala:408)
>
>at scala.collection.Iterator$$anon$10.hasNext(Iterator.
> scala:389)
>
>at scala.collection.Iterator$class.foreach(Iterator.scala:893)
>
>at scala.collection.AbstractIterator.foreach(
> Iterator.scala:1336)
>
>at scala.collection.generic.Growable$class.$plus$plus$eq(
> Growable.scala:59)
>
>at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(
> ArrayBuffer.scala:104)
>
>at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(
> ArrayBuffer.scala:48)
>
>at scala.collection.TraversableOnce$class.to(
> TraversableOnce.scala:310)
>
>at scala.collection.AbstractIterator.to(Iterator.scala:1336)
>
>at scala.collection.TraversableOnce$class.
> toBuffer(TraversableOnce.scala:302)
>
>at scala.collection.AbstractIterator.toBuffer(
> Iterator.scala:1336)
>
>at scala.collection.TraversableOnce$class.toArray(
> TraversableOnce.scala:289)
>
>at scala.collection.AbstractIterator.toArray(
> Iterator.scala:1336)
>
>at org.apache.spark.rdd.RDD$$anonfun$take$1$$anonfun$29.
> apply(RDD.scala:1353)
>
>at org.apache.spark.rdd.RDD$$anonfun$take$1$$anonfun$29.
> apply(RDD.scala:1353)
>
>at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(
> SparkContext.scala:1944)
>
>at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(
> SparkContext.scala:1944)
>
>at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.
> scala:87)
>
>at org.apache.spark.scheduler.Task.run(Task.scala:99)
>
>at org.apache.spark.executor.Executor$TaskRunner.run(
> Executor.scala:282)
>
>at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
>
>at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
>
>at java.lang.Thread.run(Thread.java:745)
>
>
>
> Driver stacktrace:
>
> %%dataframe [arguments]
>
> DATAFRAME_CODE
>
>
>
> DATAFRAME_CODE can be any numbered lines of code, as long as the
>
> last line is a reference to a variable which is a DataFrame.
>
> OptionDescription
>
> -----
>
> --limit   The number of records to return
>
>   (default: 10)
>
> --output  The type of the output: html, csv,
>
>   json (default: html)
>
> (default: 10)
>
> json (default: html)
>
>
>
>
>
> 2nd PySpark error;
>
>
>
> %%PySpark
>
> from operator import add
>
> print(sc.parallelize(range(1, 100)).reduce(add))
>
> Out;
>
> Name: org.apache.toree.interpreter.broker.BrokerException
>
> Message: Traceback (most recent call last):
>
>   File 
> "/tmp/kernel-PySpark-2e158b54-f8dc-483e-bd42-749a84637c05/pyspark_runner.py",
> line 189, in 
>
> eval(compiled_code)
>
>   File "", line 1, in 
>
>   File 
> "/tmp/kernel-PySpark-2e158b54-f8dc-483e-bd42-749a84637c05/pyspark_runner.py",
> line 107, in __getattr__
>
> return self._jvm_kernel.__getattribute__(name)
>
> AttributeError: 'JavaObject' object has no attribute 'magics'
>
>
>
> StackTrace: org.apache.toree.interpreter.broker.BrokerState$$anonfun$
> markFailure$1.apply(BrokerState.scala:163)
>
> org.apache.toree.interpreter.broker.BrokerState$$anonfun$
> markFailure$1.apply(BrokerState.scala:163)
>
> scala.Option.foreach(Option.scala:257)
>
> org.apache.toree.interpreter.broker.BrokerState.
> markFailure(BrokerState.scala:162)
>
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>

Please check these 2 errors

2017-06-27 Thread Sihyoung Jurn 
Hello,

I’m installing Apache Toree in our JupyterHub. But I can’t execute the example 
https://github.com/apache/incubator-toree/blob/master/etc/examples/notebooks/magic-tutorial.ipynb
 well. The below 2 errors occur. When installing, I haven’t met an error.

Can you have any solution?


1st Scala error:
[cid:image001.png@01D2EF72.9D9B0C20]
%%dataframe
df


Out;
ERROR: Job aborted due to stage failure: Task 2 in stage 11.0 failed 1 times, 
most recent failure: Lost task 2.0 in stage 11.0 (TID 28, localhost, executor 
driver): java.lang.ClassCastException: $line53.$read$$iw$$iw$DFRecord cannot be 
cast to $line53.$read$$iw$$iw$DFRecord
   at 
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator.processNext(Unknown
 Source)
   at 
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
   at 
org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$8$$anon$1.hasNext(WholeStageCodegenExec.scala:377)
   at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
   at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
   at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)
   at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:389)
   at scala.collection.Iterator$class.foreach(Iterator.scala:893)
   at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
   at 
scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59)
   at 
scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104)
   at 
scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48)
   at 
scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310)
   at scala.collection.AbstractIterator.to(Iterator.scala:1336)
   at 
scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302)
   at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1336)
   at 
scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289)
   at scala.collection.AbstractIterator.toArray(Iterator.scala:1336)
   at 
org.apache.spark.rdd.RDD$$anonfun$take$1$$anonfun$29.apply(RDD.scala:1353)
   at 
org.apache.spark.rdd.RDD$$anonfun$take$1$$anonfun$29.apply(RDD.scala:1353)
   at 
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1944)
   at 
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1944)
   at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
   at org.apache.spark.scheduler.Task.run(Task.scala:99)
   at 
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
   at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   at java.lang.Thread.run(Thread.java:745)

Driver stacktrace:
%%dataframe [arguments]
DATAFRAME_CODE

DATAFRAME_CODE can be any numbered lines of code, as long as the
last line is a reference to a variable which is a DataFrame.
OptionDescription
-----
--limit   The number of records to return
  (default: 10)
--output  The type of the output: html, csv,
  json (default: html)
(default: 10)
json (default: html)


2nd PySpark error;

%%PySpark
from operator import add
print(sc.parallelize(range(1, 100)).reduce(add))

Out;
Name: org.apache.toree.interpreter.broker.BrokerException
Message: Traceback (most recent call last):
  File 
"/tmp/kernel-PySpark-2e158b54-f8dc-483e-bd42-749a84637c05/pyspark_runner.py", 
line 189, in 
eval(compiled_code)
  File "", line 1, in 
  File 
"/tmp/kernel-PySpark-2e158b54-f8dc-483e-bd42-749a84637c05/pyspark_runner.py", 
line 107, in __getattr__
return self._jvm_kernel.__getattribute__(name)
AttributeError: 'JavaObject' object has no attribute 'magics'

StackTrace: 
org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163)
org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163)
scala.Option.foreach(Option.scala:257)
org.apache.toree.interpreter.broker.BrokerState.markFailure(BrokerState.scala:162)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
py4j.Gateway.invoke(Gateway.java:280)
py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
py4j.commands.CallCommand.execute(CallCommand.java:79)