[jira] [Commented] (TOREE-336) Toree not working with Apache Spark 2.0.0
[ https://issues.apache.org/jira/browse/TOREE-336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867360#comment-15867360 ] Luciano Resende commented on TOREE-336: --- There should be only RELEASES published outside of the realms of Apache. Developers should be cloning and building for sources. > Toree not working with Apache Spark 2.0.0 > - > > Key: TOREE-336 > URL: https://issues.apache.org/jira/browse/TOREE-336 > Project: TOREE > Issue Type: Bug > Environment: OSX and ubuntu-14.04, both running scala 2.10.4 and > spark 2.0.0 >Reporter: Tianhui Li > Original Estimate: 168h > Remaining Estimate: 168h > > Following the instructions on > https://github.com/apache/incubator-toree/blob/master/README.md, I run > ``` > pip install --pre toree > jupyter toree install --spark-home=$SPARK_HOME > ``` > I'm able to build fine. But upon starting the server and a new scala (or any > other type of notebook), I an error (provided below). This seems related to > using scala 2.10 rather than 2.11 (see > http://stackoverflow.com/questions/29339005/run-main-0-java-lang-nosuchmethoderror-scala-collection-immutable-hashset-emp > and > http://stackoverflow.com/questions/30536759/running-a-spark-application-in-intellij-14-1-3). > Below is the error: > $ jupyter notebook > [I 12:11:59.464 NotebookApp] Serving notebooks from local directory: > /Users/tianhui > [I 12:11:59.464 NotebookApp] 0 active kernels > [I 12:11:59.465 NotebookApp] The Jupyter Notebook is running at: > http://localhost:/ > [I 12:11:59.465 NotebookApp] Use Control-C to stop this server and shut down > all kernels (twice to skip confirmation). > [I 12:12:06.847 NotebookApp] 302 GET / (::1) 0.47ms > [I 12:12:10.591 NotebookApp] Creating new notebook in > [I 12:12:11.600 NotebookApp] Kernel started: > 20ca2e71-781b-4208-ad88-bc04c1ca37d6 > Starting Spark Kernel with > SPARK_HOME=/usr/local/Cellar/apache-spark/2.0.0/libexec/ > 16/09/03 12:12:12 [INFO] o.a.t.Main$$anon$1 - Kernel version: > 0.1.0.dev9-incubating-SNAPSHOT > 16/09/03 12:12:12 [INFO] o.a.t.Main$$anon$1 - Scala version: Some(2.10.4) > 16/09/03 12:12:12 [INFO] o.a.t.Main$$anon$1 - ZeroMQ (JeroMQ) version: 3.2.2 > 16/09/03 12:12:12 [INFO] o.a.t.Main$$anon$1 - Initializing internal actor > system > Exception in thread "main" java.lang.NoSuchMethodError: > scala.collection.immutable.HashSet$.empty()Lscala/collection/immutable/HashSet; > at akka.actor.ActorCell$.(ActorCell.scala:336) > at akka.actor.ActorCell$.(ActorCell.scala) > at akka.actor.RootActorPath.$div(ActorPath.scala:185) > at akka.actor.LocalActorRefProvider.(ActorRefProvider.scala:465) > at akka.actor.LocalActorRefProvider.(ActorRefProvider.scala:453) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$2.apply(DynamicAccess.scala:78) > at scala.util.Try$.apply(Try.scala:192) > at > akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:73) > at > akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:84) > at > akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:84) > at scala.util.Success.flatMap(Try.scala:231) > at > akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:84) > at akka.actor.ActorSystemImpl.liftedTree1$1(ActorSystem.scala:585) > at akka.actor.ActorSystemImpl.(ActorSystem.scala:578) > at akka.actor.ActorSystem$.apply(ActorSystem.scala:142) > at akka.actor.ActorSystem$.apply(ActorSystem.scala:109) > at > org.apache.toree.boot.layer.StandardBareInitialization$class.createActorSystem(BareInitialization.scala:71) > at org.apache.toree.Main$$anon$1.createActorSystem(Main.scala:35) > at > org.apache.toree.boot.layer.StandardBareInitialization$class.initializeBare(BareInitialization.scala:60) > at org.apache.toree.Main$$anon$1.initializeBare(Main.scala:35) > at > org.apache.toree.boot.KernelBootstrap.initialize(KernelBootstrap.scala:70) > at org.apache.toree.Main$delayedInit$body.apply(Main.scala:40) > at scala.Function0$class.apply$mcV$sp(Function0.scala:34) > at > scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12) > at scala.App$$anonfun$main$1.apply(App.scala:76) > at scala.App$$anonfun$main$1.apply(App.scala:76) > at
[jira] [Resolved] (TOREE-384) Erratic behavior with Toree - Scala kernel
[ https://issues.apache.org/jira/browse/TOREE-384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luciano Resende resolved TOREE-384. --- Resolution: Fixed Assignee: Corey A Stubbs > Erratic behavior with Toree - Scala kernel > -- > > Key: TOREE-384 > URL: https://issues.apache.org/jira/browse/TOREE-384 > Project: TOREE > Issue Type: Bug >Affects Versions: 0.1.0 > Environment: RedHat7 >Reporter: Kun Liu >Assignee: Corey A Stubbs > Labels: compile-error, kernel, scala > Attachments: Screen Shot 2017-02-14 at 4.13.53 PM.png > > > When starting a Spark Scala kernel, it may throw a "compile error" even for a > simple statement like sc.version, and there is no StackTrace displayed. See > attachment. Also, the line number is always "18". > This error does not happen very often, and my workaround is to shutdown the > notebook kernel and restart it. This may because the kernel is still unstable. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TOREE-384) Erratic behavior with Toree - Scala kernel
[ https://issues.apache.org/jira/browse/TOREE-384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867071#comment-15867071 ] Luciano Resende commented on TOREE-384: --- I have seen this when the kernel replies idle and Scala Interpreter hasn't been initialized yet. The fix below fixes the problem: https://github.com/apache/incubator-toree/commit/e86ac46029a1f2cc588b7f4b16ccb037ae71003a I'll merge to Master > Erratic behavior with Toree - Scala kernel > -- > > Key: TOREE-384 > URL: https://issues.apache.org/jira/browse/TOREE-384 > Project: TOREE > Issue Type: Bug >Affects Versions: 0.1.0 > Environment: RedHat7 >Reporter: Kun Liu > Labels: compile-error, kernel, scala > Attachments: Screen Shot 2017-02-14 at 4.13.53 PM.png > > > When starting a Spark Scala kernel, it may throw a "compile error" even for a > simple statement like sc.version, and there is no StackTrace displayed. See > attachment. Also, the line number is always "18". > This error does not happen very often, and my workaround is to shutdown the > notebook kernel and restart it. This may because the kernel is still unstable. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (TOREE-383) Fix flaky tests
[ https://issues.apache.org/jira/browse/TOREE-383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky reassigned TOREE-383: --- Assignee: Jakob Odersky > Fix flaky tests > --- > > Key: TOREE-383 > URL: https://issues.apache.org/jira/browse/TOREE-383 > Project: TOREE > Issue Type: Sub-task >Reporter: Jakob Odersky >Assignee: Jakob Odersky >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (TOREE-385) Refactor travis build to be runnable as a container (as opposed to a vm)
Jakob Odersky created TOREE-385: --- Summary: Refactor travis build to be runnable as a container (as opposed to a vm) Key: TOREE-385 URL: https://issues.apache.org/jira/browse/TOREE-385 Project: TOREE Issue Type: Sub-task Reporter: Jakob Odersky Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (TOREE-383) Fix flaky tests
Jakob Odersky created TOREE-383: --- Summary: Fix flaky tests Key: TOREE-383 URL: https://issues.apache.org/jira/browse/TOREE-383 Project: TOREE Issue Type: Sub-task Reporter: Jakob Odersky Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (TOREE-384) Erratic behavior with Toree - Scala kernel
Kun Liu created TOREE-384: - Summary: Erratic behavior with Toree - Scala kernel Key: TOREE-384 URL: https://issues.apache.org/jira/browse/TOREE-384 Project: TOREE Issue Type: Bug Affects Versions: 0.1.0 Environment: RedHat7 Reporter: Kun Liu Attachments: Screen Shot 2017-02-14 at 4.13.53 PM.png When starting a Spark Scala kernel, it may throw a "compile error" even for a simple statement like sc.version, and there is no StackTrace displayed. See attachment. Also, the line number is always "18". This error does not happen very often, and my workaround is to shutdown the notebook kernel and restart it. This may because the kernel is still unstable. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (TOREE-382) Revamp the sbt build and consolidate dependencies
[ https://issues.apache.org/jira/browse/TOREE-382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky updated TOREE-382: Priority: Minor (was: Major) > Revamp the sbt build and consolidate dependencies > - > > Key: TOREE-382 > URL: https://issues.apache.org/jira/browse/TOREE-382 > Project: TOREE > Issue Type: Sub-task >Reporter: Jakob Odersky >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (TOREE-382) Revamp the sbt build and consolidate dependencies
Jakob Odersky created TOREE-382: --- Summary: Revamp the sbt build and consolidate dependencies Key: TOREE-382 URL: https://issues.apache.org/jira/browse/TOREE-382 Project: TOREE Issue Type: Sub-task Reporter: Jakob Odersky -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (TOREE-381) Revamp the build
Jakob Odersky created TOREE-381: --- Summary: Revamp the build Key: TOREE-381 URL: https://issues.apache.org/jira/browse/TOREE-381 Project: TOREE Issue Type: Improvement Reporter: Jakob Odersky Priority: Minor Toree's build is quite complex and has flaky tests. The complexity is partly inherent to the nature of the project, however it is also due to accumulated features over time that are no longer used. In an attempt to improve developer productivity, I propose a 3-part plan to increase build stability and speed: 1. Revamp the sbt build by consolidating dependencies and refactoring the configuration to the current best practices (specifically moving common behaviour out of project/common.scala and into build-wide settings and auto-plugins) 2. Fix the flaky tests that currenlty fail due to classloader issues. 3. Refactor the travis build configuration to be runnable as a container (this should greatly improve CI throughput and latency). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TOREE-336) Toree not working with Apache Spark 2.0.0
[ https://issues.apache.org/jira/browse/TOREE-336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866856#comment-15866856 ] Alex Emelynov commented on TOREE-336: - [~mcarre] thank you a lot, you saved my time ) [~luciano resende] +1 for push a new version to PyPy, why it's not yet done? > Toree not working with Apache Spark 2.0.0 > - > > Key: TOREE-336 > URL: https://issues.apache.org/jira/browse/TOREE-336 > Project: TOREE > Issue Type: Bug > Environment: OSX and ubuntu-14.04, both running scala 2.10.4 and > spark 2.0.0 >Reporter: Tianhui Li > Original Estimate: 168h > Remaining Estimate: 168h > > Following the instructions on > https://github.com/apache/incubator-toree/blob/master/README.md, I run > ``` > pip install --pre toree > jupyter toree install --spark-home=$SPARK_HOME > ``` > I'm able to build fine. But upon starting the server and a new scala (or any > other type of notebook), I an error (provided below). This seems related to > using scala 2.10 rather than 2.11 (see > http://stackoverflow.com/questions/29339005/run-main-0-java-lang-nosuchmethoderror-scala-collection-immutable-hashset-emp > and > http://stackoverflow.com/questions/30536759/running-a-spark-application-in-intellij-14-1-3). > Below is the error: > $ jupyter notebook > [I 12:11:59.464 NotebookApp] Serving notebooks from local directory: > /Users/tianhui > [I 12:11:59.464 NotebookApp] 0 active kernels > [I 12:11:59.465 NotebookApp] The Jupyter Notebook is running at: > http://localhost:/ > [I 12:11:59.465 NotebookApp] Use Control-C to stop this server and shut down > all kernels (twice to skip confirmation). > [I 12:12:06.847 NotebookApp] 302 GET / (::1) 0.47ms > [I 12:12:10.591 NotebookApp] Creating new notebook in > [I 12:12:11.600 NotebookApp] Kernel started: > 20ca2e71-781b-4208-ad88-bc04c1ca37d6 > Starting Spark Kernel with > SPARK_HOME=/usr/local/Cellar/apache-spark/2.0.0/libexec/ > 16/09/03 12:12:12 [INFO] o.a.t.Main$$anon$1 - Kernel version: > 0.1.0.dev9-incubating-SNAPSHOT > 16/09/03 12:12:12 [INFO] o.a.t.Main$$anon$1 - Scala version: Some(2.10.4) > 16/09/03 12:12:12 [INFO] o.a.t.Main$$anon$1 - ZeroMQ (JeroMQ) version: 3.2.2 > 16/09/03 12:12:12 [INFO] o.a.t.Main$$anon$1 - Initializing internal actor > system > Exception in thread "main" java.lang.NoSuchMethodError: > scala.collection.immutable.HashSet$.empty()Lscala/collection/immutable/HashSet; > at akka.actor.ActorCell$.(ActorCell.scala:336) > at akka.actor.ActorCell$.(ActorCell.scala) > at akka.actor.RootActorPath.$div(ActorPath.scala:185) > at akka.actor.LocalActorRefProvider.(ActorRefProvider.scala:465) > at akka.actor.LocalActorRefProvider.(ActorRefProvider.scala:453) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) > at > akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$2.apply(DynamicAccess.scala:78) > at scala.util.Try$.apply(Try.scala:192) > at > akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:73) > at > akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:84) > at > akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:84) > at scala.util.Success.flatMap(Try.scala:231) > at > akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:84) > at akka.actor.ActorSystemImpl.liftedTree1$1(ActorSystem.scala:585) > at akka.actor.ActorSystemImpl.(ActorSystem.scala:578) > at akka.actor.ActorSystem$.apply(ActorSystem.scala:142) > at akka.actor.ActorSystem$.apply(ActorSystem.scala:109) > at > org.apache.toree.boot.layer.StandardBareInitialization$class.createActorSystem(BareInitialization.scala:71) > at org.apache.toree.Main$$anon$1.createActorSystem(Main.scala:35) > at > org.apache.toree.boot.layer.StandardBareInitialization$class.initializeBare(BareInitialization.scala:60) > at org.apache.toree.Main$$anon$1.initializeBare(Main.scala:35) > at > org.apache.toree.boot.KernelBootstrap.initialize(KernelBootstrap.scala:70) > at org.apache.toree.Main$delayedInit$body.apply(Main.scala:40) > at scala.Function0$class.apply$mcV$sp(Function0.scala:34) > at > scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12) > at scala.App$$anonfun$main$1.apply(App.scala:76) > at scala.App$$anonfun$main$1.apply(App.scala:76) > at
