Nick Allen created METRON-1266:
----------------------------------
Summary: Profiler - No valid credentials provided
Key: METRON-1266
URL: https://issues.apache.org/jira/browse/METRON-1266
Project: Metron
Issue Type: Bug
Affects Versions: 0.4.1
Reporter: Nick Allen
Assignee: Nick Allen
Fix For: Next + 1
When running the Profiler on a cluster that has multiple nodes and is secured
by Kerberos, it was observed that the HBaseBolt was unable to write to HBase.
The Storm worker running the HBaseBolt logged the following exception. This
does not occur all the time and does not occur in all environments.
{code}
2017-10-19 14:51:00.146 o.a.h.h.i.AbstractRpcClient [ERROR] SASL authentication
failed. The most likely cause is missing or invalid credentials. Consider
'kinit'.
javax.security.sasl.SaslException: GSS initiate failed
at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
~[?:1.8.0_144]
at
org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
~[stormjar.jar:?]
at
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:609)
~[stormjar.jar:?]
at
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:154)
[stormjar.jar:?]
at
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:735)
~[stormjar.jar:?]
at
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:732)
~[stormjar.jar:?]
at java.security.AccessController.doPrivileged(Native Method)
~[?:1.8.0_144]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_144]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
~[stormjar.jar:?]
at
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:732)
[stormjar.jar:?]
at
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:885)
[stormjar.jar:?]
at
org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:854)
[stormjar.jar:?]
at
org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1180)
[stormjar.jar:?]
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
[stormjar.jar:?]
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
[stormjar.jar:?]
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:32651)
[stormjar.jar:?]
at
org.apache.hadoop.hbase.client.ClientSmallScanner$SmallScannerCallable.call(ClientSmallScanner.java:201)
[stormjar.jar:?]
at
org.apache.hadoop.hbase.client.ClientSmallScanner$SmallScannerCallable.call(ClientSmallScanner.java:180)
[stormjar.jar:?]
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
[stormjar.jar:?]
at
org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:369)
[stormjar.jar:?]
at
org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:343)
[stormjar.jar:?]
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
[stormjar.jar:?]
at
org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:64)
[stormjar.jar:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_144]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_144]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]
Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism
level: Failed to find any Kerberos tgt)
at
sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
~[?:1.8.0_144]
at
sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
~[?:1.8.0_144]
at
sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
~[?:1.8.0_144]
at
sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
~[?:1.8.0_144]
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
~[?:1.8.0_144]
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
~[?:1.8.0_144]
at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192)
~[?:1.8.0_144]
... 25 more}}
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
