[
https://issues.apache.org/jira/browse/HADOOP-13251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15347756#comment-15347756
]
Xiao Chen commented on HADOOP-13251:
------------------------------------
Thanks Andrew.
bq. Is it possible to do the special per-op logic in
doDelegationTokenOperation, e.g. by not passing a DT so we trigger the TGT
relogin?? We have the op enum there, so don't need to resort to parsing the
query string.
IMHO it's only possible with hacks. Below stack trace is how we get to the
{{authenticate}}. DT has to be set outside, since that's what [DTAuthURL
requires|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticatedURL.java#L387].
Then we have several options, but all feel hacky to me:
- force a kerberos authenticate in {{doDelegationTokenOperation}}, for the 2
ops.
- unset the dt on {{token}} before passing it into {{openConnection}}
- use some kind of cache / maybe thread local storage
{noformat}
at
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:130)
at
org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:216)
at
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.doDelegationTokenOperation(DelegationTokenAuthenticator.java:312)
at
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.renewDelegationToken(DelegationTokenAuthenticator.java:239)
at
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.renewDelegationToken(DelegationTokenAuthenticatedURL.java:415)
at
org.apache.hadoop.crypto.key.kms.KMSClientProvider$2.run(KMSClientProvider.java:921)
at
org.apache.hadoop.crypto.key.kms.KMSClientProvider$2.run(KMSClientProvider.java:918)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1755)
at
org.apache.hadoop.crypto.key.kms.KMSClientProvider.renewDelegationToken(KMSClientProvider.java:917)
at
org.apache.hadoop.crypto.key.kms.KMSClientProvider$KMSTokenRenewer.renew(KMSClientProvider.java:182)
at org.apache.hadoop.security.token.Token.renew(Token.java:417)
at
org.apache.hadoop.crypto.key.kms.server.TestKMS$14$1$1.run(TestKMS.java:1858)
at
org.apache.hadoop.crypto.key.kms.server.TestKMS$14$1$1.run(TestKMS.java:1824)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1755)
at
org.apache.hadoop.crypto.key.kms.server.TestKMS$14$1.run(TestKMS.java:1824)
at
org.apache.hadoop.crypto.key.kms.server.TestKMS$14$1.run(TestKMS.java:1777)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1755)
at
org.apache.hadoop.crypto.key.kms.server.TestKMS.doAs(TestKMS.java:265)
at
org.apache.hadoop.crypto.key.kms.server.TestKMS.access$100(TestKMS.java:73)
at
org.apache.hadoop.crypto.key.kms.server.TestKMS$14.call(TestKMS.java:1777)
at
org.apache.hadoop.crypto.key.kms.server.TestKMS$14.call(TestKMS.java:1769)
at
org.apache.hadoop.crypto.key.kms.server.TestKMS.runServer(TestKMS.java:133)
at
org.apache.hadoop.crypto.key.kms.server.TestKMS.runServer(TestKMS.java:115)
at
org.apache.hadoop.crypto.key.kms.server.TestKMS.testDelegationTokensOps(TestKMS.java:1769)
at
org.apache.hadoop.crypto.key.kms.server.TestKMS.testDelegationTokensOpsSimple(TestKMS.java:1744)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{noformat}
bq. If we do need to parse, we should really use a library. A URL query string
is an unordered KV map, so this current manual parsing is brittle.
I think it's not brittle, since it was just looking for op=, so don't care
about the order etc. But it is a ugly, I changed it to look for query string
instead. What do you think?
> DelegationTokenAuthenticationHandler should detect actual renewer when renew
> token
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-13251
> URL: https://issues.apache.org/jira/browse/HADOOP-13251
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
> Affects Versions: 2.8.0
> Reporter: Xiao Chen
> Assignee: Xiao Chen
> Attachments: HADOOP-13251.01.patch, HADOOP-13251.02.patch,
> HADOOP-13251.03.patch, HADOOP-13251.04.patch, HADOOP-13251.05.patch,
> HADOOP-13251.06.patch, HADOOP-13251.07.patch, HADOOP-13251.innocent.patch
>
>
> Turns out KMS delegation token renewal feature (HADOOP-13155) does not work
> well with client side impersonation.
> In a MR example, an end user (UGI:user) gets all kinds of DTs (with
> renewer=yarn), and pass them to Yarn. Yarn's resource manager (UGI:yarn) then
> renews these DTs as long as the MR jobs are running. But currently, the token
> is used at the kms server side to decide the renewer, in which case is always
> the token's owner. This ends up rejecting the renew request due to renewer
> mismatch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
