Venkata Sudheer Kumar M created ZEPPELIN-3016:
-------------------------------------------------
Summary: Zeppelin Spark Interpreter User Impersonation is not
working
Key: ZEPPELIN-3016
URL: https://issues.apache.org/jira/browse/ZEPPELIN-3016
Project: Zeppelin
Issue Type: Bug
Components: Interpreters
Affects Versions: 0.7.3, 0.7.2, 0.7.1
Reporter: Venkata Sudheer Kumar M
Hadoop 2.7.3
Spark 2.1.1
Zeppelin (0.7.2 and 0.7.3)
Working:
Install and configure zeppelin 0.7.2/0.7.3 with Spark 2.1.1 with HDP 2.6.2
(Hadoop 2.7.3) and enable node labels from YARN ( * spark-am-worker-nodes* )
along with Preemption and Map spark to launch Application master only on these
node-labeled yarn nodes using *spark.yarn.am.nodeLabelExpression* and enable
zeppelin -spark interpreter to use *Per User* and *Isolated per user * with
this i'm able to launch the job (run *_sc.version _*from *_%pyspark_*) with the
default user *zeppelin*
Not Working:
Follow the same process as above and also enable the *User Impersonation*
setting for Spark Interpreter then (sc.version) it is failing with:
{color:#d04437}
java.lang.NullPointerException
at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
at
org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_2(SparkInterpreter.java:385)
at
org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:374)
at
org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:146)
at
org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:819)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
at
org.apache.zeppelin.spark.PySparkInterpreter.getSparkInterpreter(PySparkInterpreter.java:566)
at
org.apache.zeppelin.spark.PySparkInterpreter.createGatewayServerAndStartScript(PySparkInterpreter.java:209)
at
org.apache.zeppelin.spark.PySparkInterpreter.open(PySparkInterpreter.java:163)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:493)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at
org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){color}
Not sure why impersonation is even though Zeppelin Shell interpreter is able to
pick up my user and also i have even tried wit the below from zeppelin-env.sh
but it is still the same:
export ZEPPELIN_IMPERSONATE_CMD='sudo -H -u ${ZEPPELIN_IMPERSONATE_USER} bash
-c '
export ZEPPELIN_IMPERSONATE_SPARK_PROXY_USER=true
is there any known workaround for this or any related information will help.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
