[
https://issues.apache.org/jira/browse/HIVE-18959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399090#comment-16399090
]
slim bouguerra commented on HIVE-18959:
---------------------------------------
possible exception stack
{code}
ERROR [KerberosHttpClient-0 ()]
org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon: Thread
Thread[KerberosHttpClient-0,5,main] threw an Exception. Shutting down now...
java.lang.reflect.UndeclaredThrowableException: null at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1887)
~[hadoop-common-2.7.3.2.6.5.0-88.jar:?]
at
org.apache.hadoop.hive.druid.security.KerberosHttpClient.inner_go(KerberosHttpClient.java:105)
~[hive-druid-handler-2.1.0.2.6.5.0-88.jar:2.1.0.2.6.5.0-88]
at
org.apache.hadoop.hive.druid.security.KerberosHttpClient.access$100(KerberosHttpClient.java:50)
~[hive-druid-handler-2.1.0.2.6.5.0-88.jar:2.1.0.2.6.5.0-88]
at
org.apache.hadoop.hive.druid.security.KerberosHttpClient$2.onSuccess(KerberosHttpClient.java:144)
~[hive-druid-handler-2.1.0.2.6.5.0-88.jar:2.1.0.2.6.5.0-88]
at
org.apache.hadoop.hive.druid.security.KerberosHttpClient$2.onSuccess(KerberosHttpClient.java:134)
~[hive-druid-handler-2.1.0.2.6.5.0-88.jar:2.1.0.2.6.5.0-88]
at
org.apache.hive.druid.com.google.common.util.concurrent.Futures$4.run(Futures.java:1181)
~[hive-druid-handler-2.1.0.2.6.5.0-88.jar:2.1.0.2.6.5.0-88]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[?:1.8.0_151]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
~[?:1.8.0_151]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
Caused by:
org.apache.hadoop.security.authentication.client.AuthenticationException:
GSSException: No valid credentials provided (Mechanism level: Failed to find
any Kerberos tgt)
at
org.apache.hadoop.hive.druid.security.DruidKerberosUtil.kerberosChallenge(DruidKerberosUtil.java:82)
~[hive-druid-handler-2.1.0.2.6.5.0-88.jar:2.1.0.2.6.5.0-88]
at
org.apache.hadoop.hive.druid.security.KerberosHttpClient$1.run(KerberosHttpClient.java:110)
~[hive-druid-handler-2.1.0.2.6.5.0-88.jar:2.1.0.2.6.5.0-88]
at
org.apache.hadoop.hive.druid.security.KerberosHttpClient$1.run(KerberosHttpClient.java:106)
~[hive-druid-handler-2.1.0.2.6.5.0-88.jar:2.1.0.2.6.5.0-88]
at java.security.AccessController.doPrivileged(Native Method)
~[?:1.8.0_151]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_151]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
~[hadoop-common-2.7.3.2.6.5.0-88.jar:?]
... 8 more
Caused by: org.ietf.jgss.GSSException: No valid credentials provided (Mechanism
level: Failed to find any Kerberos tgt)
at
sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
~[?:1.8.0_151]
at
sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
~[?:1.8.0_151]
at
sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
~[?:1.8.0_151]
at
sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
~[?:1.8.0_151]
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
~[?:1.8.0_151]
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
~[?:1.8.0_151]
at
org.apache.hadoop.hive.druid.security.DruidKerberosUtil.kerberosChallenge(DruidKerberosUtil.java:75)
~[hive-druid-handler-2.1.0.2.6.5.0-88.jar:2.1.0.2.6.5.0-88]
at
org.apache.hadoop.hive.druid.security.KerberosHttpClient$1.run(KerberosHttpClient.java:110)
~[hive-druid-handler-2.1.0.2.6.5.0-88.jar:2.1.0.2.6.5.0-88]
at
org.apache.hadoop.hive.druid.security.KerberosHttpClient$1.run(KerberosHttpClient.java:106)
~[hive-druid-handler-2.1.0.2.6.5.0-88.jar:2.1.0.2.6.5.0-88]
at java.security.AccessController.doPrivileged(Native Method)
~[?:1.8.0_151]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_151]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
~[hadoop-common-2.7.3.2.6.5.0-88.jar:?]
... 8 more
{code}
> Avoid creating extra pool of threads within LLAP
> ------------------------------------------------
>
> Key: HIVE-18959
> URL: https://issues.apache.org/jira/browse/HIVE-18959
> Project: Hive
> Issue Type: Task
> Components: Druid integration
> Environment: Kerberos Cluster
> Reporter: slim bouguerra
> Assignee: slim bouguerra
> Priority: Major
> Fix For: 3.0.0
>
>
> The current Druid-Kerberos-Http client is using an external single threaded
> pool to handle retry auth calls (eg when a cookie expire or other transient
> auth issues).
> First, this is not buying us anything since all the Druid Task is executed as
> one synchronous task.
> Second, this can cause a major issue if an exception occurs that leads to
> shutting down the LLAP main thread.
> Thus to fix this we should avoid using an external thread pool and handle
> retrying in a synchronous way.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
