Hello Hyung, thanks for pointing out SPARK_HOME. Setting this did the trick.
I always used Git versions so I think I set SPARK_HOME in previous installations without remembering now. But it works now, thanks. - Jens > Am 29.12.2015 um 20:53 schrieb Hyung Sung Shim <hss...@nflabs.com>: > > Hello. > Maybe It's added from 0.5.5 version. > > 2015-12-30 4:42 GMT+09:00 Jens Rabe <rabe-j...@t-online.de > <mailto:rabe-j...@t-online.de>>: > Hello, > > is this new? In earlier Zeppelin versions I could just set the mentioned > options and it worked in yarn-client mode. > >> Am 29.12.2015 um 20:40 schrieb Hyung Sung Shim <hss...@nflabs.com >> <mailto:hss...@nflabs.com>>: >> >> Hello Jens Rabe. >> >> If you want to run zeppelin using spark-submit, you should set variable >> SPARK_HOME to zeppelin-env.sh. >> >> Thanks. >> >> >> >> 2015-12-30 4:18 GMT+09:00 Jens Rabe <rabe-j...@t-online.de >> <mailto:rabe-j...@t-online.de>>: >> Hello, >> >> I am trying to set up Zeppelin to use Spark on YARN. Spark on YARN itself >> works, I can use spark-submit and spark-shell. So I set up Zeppelin and my >> zeppelin-env.sh contains the following: >> >> #!/bin/bash >> >> export JAVA_HOME=/usr/lib/jvm/java-7-oracle >> export MASTER=yarn-client # Spark master url. eg. >> spark://master_addr:7077 <>. Leave empty if you want to use local mode. >> export ZEPPELIN_JAVA_OPTS="-Dspark.dynamicAllocation.enabled=true >> -Dspark.shuffle.service.enabled=true" # Additional jvm options. >> for example, export ZEPPELIN_JAVA_OPTS="-Dspark.executor.memory=8g >> -Dspark.cores.max=16" >> export ZEPPELIN_PORT=10080 >> export HADOOP_CONF_DIR=/opt/hadoop/etc/hadoop >> >> I double-checked that /opt/hadoop/etc/hadoop really contains the correct >> configuration files, and it does. zeppelin-env-sh is executable, too. But >> when I start Zeppelin and try to submit something, it tries to connect to a >> YARN RM at 127.0.0.1. It seems that it ignores HADOOP_CONF_DIR. >> >> Is this a bug or am I missing something? >> >> - Jens >> > >
