Hi Tim,
Yes, I have referred the thread on user impersonation from Tuesday, July
10, 2018 (
https://www.mail-archive.com/user@livy.incubator.apache.org/msg00294.html )
Yes, I have enabled hive context.
Please find below configurations set in livy.conf .
livy.keystore = /path/to/server.jks
livy.keystore.password = password
livy.server.port = 8998
livy.spark.master = yarn
livy.spark.deploy-mode = cluster
livy.impersonation.enabled = true
livy.repl.enable-hive-context = true
livy.server.access-control.enabled = false
livy.server.auth.type = kerberos
livy.server.auth.kerberos.keytab=/path/to/http/keytab
livy.server.auth.kerberos.principal=HTTP/
livy.server.launch.kerberos.keytab= /path/to/my_proxy_user/keytab
livy.server.launch.kerberos.principal=my_proxy_user/
livy.superusers = my_proxy_user
Whenever I try to create a batch, the batch is successfully executing.
However, yarn diagnostics is throwing the above YarnException.
When I try to create a session, the session is finally ending up in a dead
state. with the above-mentioned Yarn Exception in the Yarn Diagnostics.
Thanks,
Vamsi
On Tue, 21 Aug 2018 at 00:49, Harsch, Tim wrote:
> Vamsi,
>
> It may be helpful if you read the thread on user impersonation from
> Tuesday, July 10, 2018.
>
>
>
> Do you have hive sql enabled? What related configurations do you have set
> in livy.conf ? Are you using kerberos as well?
>
>
>
>
>
> *From: *srungarapu vamsi
> *Reply-To: *"user@livy.incubator.apache.org" <
> user@livy.incubator.apache.org>
> *Date: *Monday, August 20, 2018 at 3:36 AM
> *To: *"user@livy.incubator.apache.org"
> *Subject: *Impersonation Error
>
>
>
> [External Email]
> --
>
> Hi,
>
>
>
> We have a secure kerberized CDH cluster.
>
> I am trying to test impersonation through livy rest sever.
>
> I have created the proxy user which can impersonate other users in the
> while running the spark jobs.
>
> Here is my core-site.xml for the proxy user "my_proxy_user":
>
>
>
>
>
> hadoop.proxyuser.my_proxy_user.hosts
>
> *
>
>
>
>
>
> hadoop.proxyuser. my_proxy_user.groups
>
> my_proxy_user_grp
>
>
>
>
>
> hadoop.proxyuser. my_proxy_user.users
>
> *
>
>
>
>
>
>
>
> With these configurations, whenever i am trying to create a session or
> batch, i am getting the following error.
>
>
>
> org.apache.hadoop.yarn.exceptions.YarnException: User my_proxy_user does not
> have privilage to see this attempt appattempt_1533784624146_0323_-01
>
> at
> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getApplicationAttemptReport(ClientRMService.java:375)
>
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getApplicationAttemptReport(ApplicationClientProtocolPBServiceImpl.java:355)
>
> at
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:425)
>
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
>
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
>
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2217)
>
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2213)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at javax.security.auth.Subject.doAs(Subject.java:422)
>
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
>
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2211)
>
>
>
> I am able to run the spark jobs by passsing the --proxyUser flag.
>
> Also, i am successfully able to run yarn jobs with HADOOP_PROXY_USER.
>
>
>
> But, on trying to create a session or batch, i am not getting in stderr of
> the session/batch logs. But "Yarn Diagnostics" show the above error.
>
>
>
> Can you please help me in solving this issue.
>
> --
>
> /Vamsi
>
--
/Vamsi
