ASF GitHub Bot commented on TOREE-430:

GitHub user Myllyenko opened a pull request:


    Fix for TOREE-430

    To fix TOREE-430, REPL-generated classes must be made visible to a 
    The only working way I found is to set the `spark.repl.class.outputDir` 
system property (and subsequently the corresponding Spark configuration 
property) to a path to an output directory of a REPL compiler. To achieve that, 
I had to move `SparkIMain` initialisation before creation of `SparkContext`.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/Myllyenko/incubator-toree TOREE-430

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #129
commit 2de78a0c50237bd2166843fd89982022d45a41b0
Author: Igor Melnichenko <myllye...@ya.ru>
Date:   2017-08-04T11:11:13Z

    Fixed TOREE-430 by setting the spark.repl.class.outputDir system property 
to a path to an output directory of a REPL compiler in the Scala 2.10 
    (cherry picked from commit 72261a6eb9759b0f0cae89793fdac25976d10c48)
    # Conflicts:


> ClassNotFoundException with lambdas on CDH 5.10.0
> -------------------------------------------------
>                 Key: TOREE-430
>                 URL: https://issues.apache.org/jira/browse/TOREE-430
>             Project: TOREE
>          Issue Type: Bug
>          Components: Kernel
>    Affects Versions: 0.1.0
>         Environment: CDH 5.10.0 (Spark 1.6.0, Scala 2.10.5)
>            Reporter: Igor Melnichenko
> In the specified environment, Toree classloader fails to load classes defined 
> and compiled in REPL. For instance, the following Spark example fails with 
> ClassNotFoundException and works fine if the lambda is removed:
> {code:title=Example.scala|borderStyle=solid}
> val NUM_SAMPLES = 100
> val count = sc.parallelize(1 to NUM_SAMPLES).filter { _ =>
>   val x = math.random
>   val y = math.random
>   x*x + y*y < 1
> }.count()
> println(s"Pi is roughly ${4.0 * count / NUM_SAMPLES}")
> {code}
> Cause: a location of a directory used by a REPL compiler to store 
> runtime-compiled classes is different from a location where classloader tries 
> to find these classes.
> PR with fix will be opened soon.

This message was sent by Atlassian JIRA

Reply via email to