Hi,
"--master yarn --deploy-mode client" will be overridden when create spark
context by SparkInterpreter In zeppelin. You have to set those values in
interpreter setting page
Regards,
Jongyoul
On Fri, May 5, 2017 at 8:33 AM, Jianfeng (Jeff) Zhang <
jzh...@hortonworks.com> wrote:
>
> Could you try set yarn-client in interpreter setting page ?
>
>
> Best Regard,
> Jeff Zhang
>
>
> From: Yeshwanth Jagini <y...@yotabitesllc.com>
> Reply-To: "users@zeppelin.apache.org" <users@zeppelin.apache.org>
> Date: Friday, May 5, 2017 at 3:13 AM
> To: "users@zeppelin.apache.org" <users@zeppelin.apache.org>
> Subject: Unable to run Zeppelin Spark on YARN
>
> Hi we are running cloudera CDH 5.9.1 .
>
> while setting up zeppelin, i followed the documentation on website and
> specified following options
>
> export ZEPPELIN_JAVA_OPTS="-Dhadoop.version=2.6.0-cdh5.9.1"
> # Additional jvm options. for example, export
> ZEPPELIN_JAVA_OPTS="-Dspark.executor.memory=8g -Dspark.cores.max=16"
>
> export SPARK_HOME="/opt/cloudera/parcels/CDH-5.9.1-1.cdh5.9.1.p0.4/lib/spark"
> # (required) When it is defined, load it instead
> of Zeppelin embedded Spark libraries
> export SPARK_SUBMIT_OPTIONS="--master yarn --deploy-mode client"
> # (optional) extra options to pass to spark submit. eg)
> "--driver-memory 512M --executor-memory 1G".
> export SPARK_APP_NAME=Zeppelin # (optional) The
> name of spark application.
>
> export HADOOP_CONF_DIR=/etc/hadoop/conf #
> yarn-site.xml is located in configuration directory in HADOOP_CONF_DIR.
>
> export ZEPPELIN_IMPERSONATE_CMD='sudo -H -u ${ZEPPELIN_IMPERSONATE_USER}
> bash -c' # Optional, when user want to run interpreter as end web
> user. eg) 'sudo -H -u ${ZEPPELIN_IMPERSONATE_USER} bash -c '
>
> when running spark notebook, spark-submit is running in local mode and i
> cannot see the application in yarn resource manager.
> is there any other configuration i am missing?
>
>
> Thanks,
> Yeshwanth Jagini
>
--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net
