[
https://issues.apache.org/jira/browse/HADOOP-9957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767678#comment-13767678
]
Aimee Cheng commented on HADOOP-9957:
-------------------------------------
I see. Actually the reason for that ticket is that we found a strange problem
that though we use checkTGTAndReloginFromKeytab() every time when we do the
hbase access, but we still met "SASL authentication failed" problem after about
1-2 days running, you can see the error log in below. So we want that we can
force it to relogin without checking TGT when we met "SASL authentication
failed" problem. While now we replace the exception handler to be login again,
but this problem still exists. Sorry for creating ticket first without
checking carefully, I'll ask for help in hbase community.
{quote}
java.lang.RuntimeException: SASL authentication failed. The most likely cause
is missing or invalid credentials. Consider 'kinit'.
at
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$1.run(SecureClient.java:242)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37)
at org.apache.hadoop.hbase.security.User.call(User.java:590)
at org.apache.hadoop.hbase.security.User.access$700(User.java:51)
at
org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:444)
at
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.handleSaslConnectionFailure(SecureClient.java:203)
at
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:291)
at
org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1124)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:974)
at
org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEngine.java:104)
at $Proxy7.getClosestRowBefore(Unknown Source)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1016)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:882)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:984)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:886)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:843)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1533)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1418)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:918)
at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:774)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:749)
at
org.apache.hadoop.hbase.client.HTablePool$PooledHTable.put(HTablePool.java:394)
at
com.yahoo.slingstone.event.pipeline.dao.hbase.HBaseDAO.write(HBaseDAO.java:177)
at
com.yahoo.slingstone.event.pipeline.dao.hbase.CommonHbaseDAO.write(CommonHbaseDAO.java:91)
at
com.yahoo.slingstone.event.pipeline.storm.bolt.HBaseStorageBolt.doPersistentOperation(HBaseStorageBolt.java:181)
at
com.yahoo.slingstone.event.pipeline.storm.bolt.HBaseStorageBolt.execute(HBaseStorageBolt.java:105)
at
com.yahoo.slingstone.event.pipeline.batch.CommonBolt.execute(CommonBolt.java:36)
at
backtype.storm.daemon.executor$eval3836$fn__3837$tuple_action_fn__3839.invoke(executor.clj:566)
at
backtype.storm.daemon.executor$mk_task_receiver$fn__3760.invoke(executor.clj:345)
at
backtype.storm.disruptor$clojure_handler$reify__1583.onEvent(disruptor.clj:43)
at
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:84)
at
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:58)
at
backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62)
at
backtype.storm.daemon.executor$eval3836$fn__3837$fn__3846$fn__3893.invoke(executor.clj:658)
at backtype.storm.util$async_loop$fn__357.invoke(util.clj:377)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:722)
Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Failed to find
any Kerberos tgt)]
at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at
org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:156)
at
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:177)
at
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(SecureClient.java:85)
at
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:284)
at
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:281)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1212)
at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37)
at org.apache.hadoop.hbase.security.User.call(User.java:590)
at org.apache.hadoop.hbase.security.User.access$700(User.java:51)
at
org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:444)
at
org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstreams(SecureClient.java:280)
... 30 more
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed
to find any Kerberos tgt)
at
sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at
sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
at
sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at
sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
... 46 more
{quote}
> UserGroupInformation.checkTGTAndReloginFromKeytab() do the same thing as
> method reloginFromKeytab()
> ---------------------------------------------------------------------------------------------------
>
> Key: HADOOP-9957
> URL: https://issues.apache.org/jira/browse/HADOOP-9957
> Project: Hadoop Common
> Issue Type: Wish
> Components: security
> Reporter: Aimee Cheng
>
> The methods checkTGTAndReloginFromKeytab() and reloginFromKeytab() in
> UserGroupInformation actually are do the same things. Now reloginFromKeytab()
> will check the TGT expire time, if fresh, then will not relogin, just as what
> checkTGTAndReloginFromKeytab() does. I suggest maybe we can still let
> reloginFromKeytab() not check the TGT and provide a way to let develop can
> control when to relogin. While maybe we can just remove the
> checkTGTAndReloginFromKeytab() method.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
