appleyuchi created ZEPPELIN-4838:
------------------------------------
Summary: where to set deploy mode for spark in zeppelin
Key: ZEPPELIN-4838
URL: https://issues.apache.org/jira/browse/ZEPPELIN-4838
Project: Zeppelin
Issue Type: Bug
Components: Interpreters
Affects Versions: 0.9.0
Reporter: appleyuchi
*Spark:*spark-3.0.0-preview2-bin-hadoop3.2
*Zeppelin:*zeppelin-0.9.0-preview1-bin-all
the log is as follows when I run some example from zeppelin:
java.lang.RuntimeException: master is set as yarn, but spark.submit.deployMode
is not specified
at
org.apache.zeppelin.interpreter.launcher.SparkInterpreterLauncher.getDeployMode(SparkInterpreterLauncher.java:345)
at
org.apache.zeppelin.interpreter.launcher.SparkInterpreterLauncher.buildEnvFromProperties(SparkInterpreterLauncher.java:76)
at
org.apache.zeppelin.interpreter.launcher.StandardInterpreterLauncher.launch(StandardInterpreterLauncher.java:87)
at
org.apache.zeppelin.interpreter.InterpreterSetting.createInterpreterProcess(InterpreterSetting.java:847)
at
org.apache.zeppelin.interpreter.ManagedInterpreterGroup.getOrCreateInterpreterProcess(ManagedInterpreterGroup.java:63)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getOrCreateInterpreterProcess(RemoteInterpreter.java:110)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:163)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:131)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:298)
at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:431)
at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:74)
at org.apache.zeppelin.scheduler.Job.run(Job.java:172)
at
org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:130)
at
org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:159)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
I don't know where to set
spark.submit.deployMode
I have tried in:
①spark-defaults.conf:
spark.submit.deployMode cluster
②$ZEPPELIN/conf/zeppelin-env.sh
export MASTER=yarn-cluster
but no luck
please help,thanks~!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
