GitHub user LantaoJin opened a pull request:
https://github.com/apache/spark/pull/21343
[SPARK-24292][SQL] Proxy user cannot connect to HiveMetastore in locaâ¦
â¦l mode
## What changes were proposed in this pull request?
[#17335](https://github.com/apache/spark/pull/17335) has fixed the similar
issue in yarn mode. Actually, in local mode we still face this problem.
If user want to create a table. Local mode is much easier than launching an
AM in yarn.
bin/spark-sql --master local --proxy-user x_user
But it doesn't work:
> Caused by: MetaException(message:Could not connect to meta store using
any of the URIs provided. Most recent failure:
org.apache.thrift.transport.TTransportException: GSS initiate failed
at
org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
at
org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at
org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:420)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:236)
at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
at
org.apache.spark.sql.hive.client.HiveClientImpl.(HiveClientImpl.scala:192)
## How was this patch tested?
Run the command below to see if it works.
bin/spark-sql --master local --proxy-user x_user
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/LantaoJin/spark SPARK-24292
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21343.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21343
commit 5a9863274aee0c00fe8702437925d856ccca2183
Author: Alan Jin
Date: 2018-05-16T07:21:14Z
[SPARK-24292][SQL] Proxy user cannot connect to HiveMetastore in local mode
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
