[jira] [Commented] (SPARK-19250) In security cluster, spark beeline connect to hive metastore failed
[
https://issues.apache.org/jira/browse/SPARK-19250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16493254#comment-16493254
]
zuotingbing commented on SPARK-19250:
-
Same problem in spark 2.2.1 . But We add kinit for Kerberos before start the
thrift server , beeline works well in spark 2.0.2
> In security cluster, spark beeline connect to hive metastore failed
> ---
>
> Key: SPARK-19250
> URL: https://issues.apache.org/jira/browse/SPARK-19250
> Project: Spark
> Issue Type: Bug
>Reporter: meiyoula
>Priority: Major
> Labels: security-issue
>
> 1. starting thriftserver in security mode, set hive.metastore.uris to hive
> metastore uri, also hive is in security mode.
> 2. when use beeline to create table, it can't connect to hive metastore
> successfully, occurs "Failed to find any Kerberos tgt".
> {quote}
> 2017-01-17 16:25:53,618 | ERROR | [pool-25-thread-1] | SASL negotiation
> failure |
> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:315)
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
> at
> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
> at
> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1738)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:513)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:249)
> at
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1533)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3119)
> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3138)
> at
> org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:791)
> at
> org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:755)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1461)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1014)
> at
> org.apache.hadoop.hive.ql.metadata.Table.getEmptyTable(Table.java:177)
> at org.apache.hadoop.hive.ql.metadata.Table.(Table.java:119)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.org$apache$spark$sql$hive$client$HiveClientImpl$$toHiveTable(HiveClientImpl.scala:803)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createTable$1.apply$mcV$sp(HiveClientImpl.scala:430)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createTable$1.apply(HiveClientImpl.scala:430)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createTable$1.apply(HiveClientImpl.scala:430)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$withHiveState$1.apply(HiveClientImpl.scala:284)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:231)
>
[jira] [Commented] (SPARK-19250) In security cluster, spark beeline connect to hive metastore failed
[
https://issues.apache.org/jira/browse/SPARK-19250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270005#comment-16270005
]
Reid Chan commented on SPARK-19250:
---
Hi [~meiyoula], may i ask how do you solve it?
> In security cluster, spark beeline connect to hive metastore failed
> ---
>
> Key: SPARK-19250
> URL: https://issues.apache.org/jira/browse/SPARK-19250
> Project: Spark
> Issue Type: Bug
>Reporter: meiyoula
> Labels: security-issue
>
> 1. starting thriftserver in security mode, set hive.metastore.uris to hive
> metastore uri, also hive is in security mode.
> 2. when use beeline to create table, it can't connect to hive metastore
> successfully, occurs "Failed to find any Kerberos tgt".
> {quote}
> 2017-01-17 16:25:53,618 | ERROR | [pool-25-thread-1] | SASL negotiation
> failure |
> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:315)
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
> at
> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
> at
> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1738)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:513)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:249)
> at
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1533)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3119)
> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3138)
> at
> org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:791)
> at
> org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:755)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1461)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1014)
> at
> org.apache.hadoop.hive.ql.metadata.Table.getEmptyTable(Table.java:177)
> at org.apache.hadoop.hive.ql.metadata.Table.(Table.java:119)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.org$apache$spark$sql$hive$client$HiveClientImpl$$toHiveTable(HiveClientImpl.scala:803)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createTable$1.apply$mcV$sp(HiveClientImpl.scala:430)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createTable$1.apply(HiveClientImpl.scala:430)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createTable$1.apply(HiveClientImpl.scala:430)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$withHiveState$1.apply(HiveClientImpl.scala:284)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:231)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:230)
> at
>
[jira] [Commented] (SPARK-19250) In security cluster, spark beeline connect to hive metastore failed
[
https://issues.apache.org/jira/browse/SPARK-19250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268545#comment-16268545
]
Reid Chan commented on SPARK-19250:
---
I encounter the same problem, spark version is 2.2.1, hive metastore version is
0.14.0, command is pretty simple,
{{code}}
create table zepdb.test_tablename (
idint,
query string,
name string
);
{{code}}
> In security cluster, spark beeline connect to hive metastore failed
> ---
>
> Key: SPARK-19250
> URL: https://issues.apache.org/jira/browse/SPARK-19250
> Project: Spark
> Issue Type: Bug
>Reporter: meiyoula
> Labels: security-issue
>
> 1. starting thriftserver in security mode, set hive.metastore.uris to hive
> metastore uri, also hive is in security mode.
> 2. when use beeline to create table, it can't connect to hive metastore
> successfully, occurs "Failed to find any Kerberos tgt".
> {quote}
> 2017-01-17 16:25:53,618 | ERROR | [pool-25-thread-1] | SASL negotiation
> failure |
> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:315)
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
> at
> org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
> at
> org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
> at
> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1738)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:513)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:249)
> at
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1533)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3119)
> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3138)
> at
> org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:791)
> at
> org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:755)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1461)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1014)
> at
> org.apache.hadoop.hive.ql.metadata.Table.getEmptyTable(Table.java:177)
> at org.apache.hadoop.hive.ql.metadata.Table.(Table.java:119)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.org$apache$spark$sql$hive$client$HiveClientImpl$$toHiveTable(HiveClientImpl.scala:803)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createTable$1.apply$mcV$sp(HiveClientImpl.scala:430)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createTable$1.apply(HiveClientImpl.scala:430)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createTable$1.apply(HiveClientImpl.scala:430)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$withHiveState$1.apply(HiveClientImpl.scala:284)
>
[jira] [Commented] (SPARK-19250) In security cluster, spark beeline connect to hive metastore failed
[ https://issues.apache.org/jira/browse/SPARK-19250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827320#comment-15827320 ] meiyoula commented on SPARK-19250: -- [~rxin] I think you may understand this bug. I find u set hive.metastore.uris=“” to thriftsever.hiveConf in commit https://github.com/apache/spark/commit/054f991c4350af1350af7a4109ee77f4a34822f0#diff-709404b0d3defeff035ef0c4f5a960e5. But when it set to local metastore, beeline openSession will not obtain token and connect remote metastore will failed. Can u have a look and give som ideas? Thanks! > In security cluster, spark beeline connect to hive metastore failed > --- > > Key: SPARK-19250 > URL: https://issues.apache.org/jira/browse/SPARK-19250 > Project: Spark > Issue Type: Bug >Reporter: meiyoula > Labels: security-issue > > 1. starting thriftserver in security mode, set hive.metastore.uris to hive > metastore uri, also hive is in security mode. > 2. when use beeline to create table, it can't connect to hive metastore > successfully, occurs "Failed to find any Kerberos tgt". > Reason： > When open hivemetastore client, first check if has token, because the > hive.metastore.uris has been set to local, so it don't obtain token; secondly > use tgt to auth, but current user is a proxyuser. So open metastore client > failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-19250) In security cluster, spark beeline connect to hive metastore failed
[ https://issues.apache.org/jira/browse/SPARK-19250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826521#comment-15826521 ] Marcelo Vanzin commented on SPARK-19250: Can't say whether they're related. You haven't posted the commands used to reproduce the problem, nor the error you see. (I'm also really not familiar with Spark's thrift server.) > In security cluster, spark beeline connect to hive metastore failed > --- > > Key: SPARK-19250 > URL: https://issues.apache.org/jira/browse/SPARK-19250 > Project: Spark > Issue Type: Bug >Reporter: meiyoula > > 1. starting thriftserver in security mode, set hive.metastore.uris to hive > metastore uri, also hive is in security mode. > 2. when use beeline to create table, it can't connect to hive metastore > successfully, occurs "Failed to find any Kerberos tgt". > Reason： > When open hivemetastore client, first check if has token, because the > hive.metastore.uris has been set to local, so it don't obtain token; secondly > use tgt to auth, but current user is a proxyuser. So open metastore client > failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-19250) In security cluster, spark beeline connect to hive metastore failed
[ https://issues.apache.org/jira/browse/SPARK-19250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15824896#comment-15824896 ] meiyoula commented on SPARK-19250: -- [~vanzin] I found SPARK-13478 is a similar bug, you has resolved it. Can you see this bug, and give some idea on it? Thanks! > In security cluster, spark beeline connect to hive metastore failed > --- > > Key: SPARK-19250 > URL: https://issues.apache.org/jira/browse/SPARK-19250 > Project: Spark > Issue Type: Bug >Reporter: meiyoula > > 1. starting thriftserver in security mode, set hive.metastore.uris to hive > metastore uri, also hive is in security mode. > 2. when use beeline to create table, it can't connect to hive metastore > successfully, occurs "Failed to find any Kerberos tgt". > Reason： > When open hivemetastore client, first check if has token, because the > hive.metastore.uris has been set to local, so it don't obtain token; secondly > use tgt to auth, but current user is a proxyuser. So open metastore client > failed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
