Kent Yao created SPARK-20060:
--------------------------------
Summary: Spark On Non-Yarn Mode with Kerberized HDFS ProxyUser
Fails Talking to Hive MetaStore
Key: SPARK-20060
URL: https://issues.apache.org/jira/browse/SPARK-20060
Project: Spark
Issue Type: Bug
Components: Deploy, Spark Submit
Affects Versions: 2.2.0
Reporter: Kent Yao
For **Spark on non-Yarn** mode on a kerberized hdfs, we don't obtain
credentials from hive metastore, hdfs, etc and just use the local kinited user
to connecting them. But if we specify the --proxy-user argument on non-yarn
mode, such as local, standalone, after we simply use `UGI.createProxyUser` to
get a proxy ugi as the effective user and wrap the code in doAs, the proxy ugi
fails to talk to hive metastore cause by no crendentials. Thus, we need to
obtain credentials via the real user and add them to the proxy ugi.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
