Rushabh S Shah created HADOOP-14265:
---------------------------------------
Summary: AuthenticatedURL swallows the exception received from
server.
Key: HADOOP-14265
URL: https://issues.apache.org/jira/browse/HADOOP-14265
Project: Hadoop Common
Issue Type: Bug
Reporter: Rushabh S Shah
Assignee: Rushabh S Shah
While debugging some issue with kms server, we found out that AuthenticatedURL
swallows the original exception from server and constructed
{{AuthenticationException}} with response code and response message.
Below is the stack trace which didn't help in figuring out why the
getDelegationTokens call failed.
Due to lack of info logs on the kms server side also, this made the debugging
even harder.
{noformat}
2017-03-23 16:32:10,364 ERROR [HiveServer2-Background-Pool: Thread-17795] []
exec.Task (TezTask.java:execute(197)) - Failed to execute tez graph.
java.io.IOException: java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.crypto.key.kms.KMSClientProvider.addDelegationTokens(KMSClientProvider.java:1042)
at
org.apache.hadoop.crypto.key.KeyProviderDelegationTokenExtension.addDelegationTokens(KeyProviderDelegationTokenExtension.java:110)
at
org.apache.hadoop.hdfs.DistributedFileSystem.addDelegationTokens(DistributedFileSystem.java:2444)
at
org.apache.tez.common.security.TokenCache.obtainTokensForFileSystemsInternal(TokenCache.java:107)
at
org.apache.tez.common.security.TokenCache.obtainTokensForFileSystemsInternal(TokenCache.java:86)
at
org.apache.tez.common.security.TokenCache.obtainTokensForFileSystems(TokenCache.java:76)
at
org.apache.tez.client.TezClientUtils.addLocalResources(TezClientUtils.java:301)
at
org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:180)
at
org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:911)
...
Caused by: java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1954)
at
org.apache.hadoop.crypto.key.kms.KMSClientProvider.addDelegationTokens(KMSClientProvider.java:1023)
... 31 more
Caused by:
org.apache.hadoop.security.authentication.client.AuthenticationException:
Authentication failed, status: 403, message: Forbidden
at
org.apache.hadoop.security.authentication.client.AuthenticatedURL.extractToken(AuthenticatedURL.java:275)
at
org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:77)
at
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:132)
at
org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:214)
at
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:132)
at
org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.doDelegationTokenOperation(DelegationTokenAuthenticator.java:298)
at
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.getDelegationToken(DelegationTokenAuthenticator.java:170)
at
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.getDelegationToken(DelegationTokenAuthenticatedURL.java:377)
at
org.apache.hadoop.crypto.key.kms.KMSClientProvider$5.run(KMSClientProvider.java:1028)
at
org.apache.hadoop.crypto.key.kms.KMSClientProvider$5.run(KMSClientProvider.java:1023)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1936)
... 32 more
{noformat}
Following is the relevant chunk of code from branch-2.8 but the code in trunks
hasn't changed much.
{code:title=AuthenticatedURL.java|borderStyle=solid}
public static void extractToken(HttpURLConnection conn, Token token) throws
IOException, AuthenticationException {
int respCode = conn.getResponseCode();
if (notExpectedResponseCode) {
} else {
token.set(null);
throw new AuthenticationException("Authentication failed, status: " +
conn.getResponseCode() + ", message: " + conn.getResponseMessage());
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
