I am able to build Toree using Scala 2.12.2 and Spark 2.2. Here are the steps:
cd $TOREE_HOME # location where you clone github.com/apache/incubator-toree APACHE_SPARK_VERSION=2.2.0 BASE_VERSION=0.2.0 IS_SNAPSHOT=False make clean release pip install $TOREE_HOME/dist/toree-pip/toree-*.tar.gz jupyter toree install --user --spark_home=$SPARK_HOME --kernel_name="Spark 2.2.0" --interpreters=Scala,SQL --debug Note that to be able to do `make clean release`, you need `gpg` to be setup. If you don't want to setup `gpg`, you can use `make clean dist` and then find the tar.gz under $TOREE_HOME/dist folder to install using pip/pip3. Thanks, - Sanjay On Thu, Feb 15, 2018 at 11:13 AM, Luciano Resende <luckbr1...@gmail.com> wrote: > Toree is not officially supporting Scala 2.12 yet. > You would have to make sure Toree gets built with that version of Scala and > that things are working properly. > > Would Scala 2.11.11 suffice your requirements? > > > On Wed, Feb 14, 2018 at 9:54 AM, Vannson, Raphael < > raphael.vann...@thinkbiganalytics.com> wrote: > > > Hello, > > > > I am trying to use Toree to develop in Scala with a Jupyter notebook. > > Our setup is Amazon EMR, Spark 2.2.0. My goal is to connect the Toree > > kernel to the Spark components already coming with EMR and to setup > master > > = yarn-client. > > > > I have installed Jupyterhub and Toree, was able to access Jupyter and > > create a new notebook using the Toree kernel but the kernel crashes. > > Below are the steps I have taken to setup everything and extract of the > > error message thrown by Jupyter / Toree. > > > > My guess is that Spark 2.2.0 is not supported with Toree and that there > is > > a Scala version mismatch. > > > > Can you confirm / let me know if there is a way to use Toree with Spark > > 2.2.0 on EMR? > > > > Let me know if you have any questions. > > > > Many thanks, > > Raphael > > > > > > sudo su - > > > > # Python 3.4 and pip3 > > curl -O https://bootstrap.pypa.io/get-pip.py > > /usr/bin/python3.4 get-pip.py > > > > > > # Nodejs, npm and http-proxy > > yum install nodejs npm --enablerepo=epel > > npm install -g configurable-http-proxy > > > > # Jupyter > > pip3 install notebook > > pip3 install jupyterhub > > > > > > # Spark / scala kernel > > mv /usr/bin/java /usr/bin/java.ori > > ln -s /usr/lib/jvm/java-1.8.0-openjdk.x86_64/bin/java /usr/bin/java > > > > wget http://downloads.typesafe.com/scala/2.12.4/scala-2.12.4.rpm > > rpm -ivh scala-2.12.4.rpm > > scala -version > > rm -f scala-2.12.4.rpm > > > > # environment variables > > vi /etc/profile > > insert ===>>> > > ##################################################################### > > export SPARK_HOME="/usr/lib/spark" > > echo "SPARK_HOME=$SPARK_HOME" > > > > export JAVA_HOME="/usr/lib/jvm/java-1.8.0-openjdk.x86_64" > > echo "JAVA_HOME=$JAVA_HOME" > > > > export SCALA_HOME="/usr/share/scala" > > echo "SCALA_HOME=$SCALA_HOME" > > > > export YARN_CONF_DIR="/etc/hadoop/conf" > > echo "YARN_CONF_DIR=$YARN_CONF_DIR" > > > > export HADOOP_CONF_DIR="/etc/hadoop/conf" > > echo "HADOOP_CONF_DIR=$HADOOP_CONF_DIR" > > <<<=== > > source /etc/profile > > > > # install Toree > > pip3 install toree > > jupyter toree install --spark_home=$SPARK_HOME --interpreters=Scala,SQL > > --spark_opts='--master yarn' --python=/usr/bin/python3 > > > > # Start jupyterhub > > logout > > jupyterhub > > > > > > ===> Can access Jupyter, open new Toree notebook but kernel fails with > > this error: > > (odd thing is scala version reported is 2.10.4, not installed 2.12.4) > > > > 18/02/13 05:36:53 INFO Main$$anon$1: Kernel version: 0.1.0-incubating > > 18/02/13 05:36:53 INFO Main$$anon$1: Scala version: Some(2.10.4) > > 18/02/13 05:36:53 INFO Main$$anon$1: ZeroMQ (JeroMQ) version: 3.2.5 > > 18/02/13 05:36:53 INFO Main$$anon$1: Initializing internal actor system > > Exception in thread "main" java.lang.NoSuchMethodError: > > scala.collection.immutable.HashSet$.empty()Lscala/ > > collection/immutable/HashSet; > > at akka.actor.ActorCell$.<init>(ActorCell.scala:336) > > at akka.actor.ActorCell$.<clinit>(ActorCell.scala) > > > > > > > -- > Luciano Resende > http://twitter.com/lresende1975 > http://lresende.blogspot.com/ >
