See if the information here helps https://issues.apache.org/jira/browse/TOREE-97
On Tue, May 3, 2016 at 7:21 PM, Vinayak Agrawal <[email protected]> wrote: > Hi, > > I am trying to setup my windows machine as a remote client by installing > Jupyter+toree to use Spark cluster which is on cloud(IBM Bluemix). > > I have installed jupyter+toree. I have also extracted Spark 1.6 on my > windows machine… > When I run Jupyter, its works with Python3 kernel, however the Toree Scala > Kernel fails to work. > > My objective is to make this Toree-Scala kernel to work with the Spark > Cloud instance. > Where do I need to change the configuration?(my assumption is local > spark-default.conf) > Note: My Spark Cloud instance is configured to use Yarn > > > My local spark-defaul.conf looks like > > # Default system properties included when running spark-submit. > # This is useful for setting default environmental settings. > > # Example: > # spark.master spark://master:7077 > # spark.eventLog.enabled true > # spark.eventLog.dir hdfs://namenode:8021/directory > # spark.serializer > org.apache.spark.serializer.KryoSerializer > # spark.driver.memory 5g > # spark.executor.extraJavaOptions -XX:+PrintGCDetails -Dkey=value > -Dnumbers="one two three" > > > > > > Spark Cloud instance spark-default.conf looks like : > > spark.yarn.applicationMaster.waitTries 10 > spark.yarn.submit.file.replication 3 > spark.history.kerberos.principal none > spark.yarn.historyServer.address xxxxxxxxxxxxx:18080 > spark.yarn.queue default > spark.yarn.scheduler.heartbeat.interval-ms 5000 > spark.history.kerberos.keytab none > spark.driver.extraJavaOptions -Diop.version={{iop_full_version}} > spark.eventLog.dir hdfs://xxxxxxxxxxxx/iop/apps/4.1.0.0/spark/logs/ > spark.history.ui.port 18080 > spark.yarn.preserve.staging.files false > spark.yarn.driver.memoryOverhead 384 > spark.yarn.containerLauncherMaxThreads 25 > spark.eventLog.enabled true > spark.yarn.max.executor.failures 3 > spark.yarn.jar hdfs://xxxxxxxxxxxx:8020/iop/apps/ > 4.1.0.0/spark/jars/spar > spark.yarn.am.extraJavaOptions -Diop.version={{iop_full_version}} > spark.yarn.executor.memoryOverhead 384 > > where xxxxxxxxx was the actual IP address. > > > > > -- > Vinayak Agrawal > > > "To Strive, To Seek, To Find and Not to Yield!" > ~Lord Alfred Tennyson >
