[
https://issues.apache.org/jira/browse/HDFS-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Liu updated HDFS-7205:
-------------------------
Description:
When submit MapReduce job in security mode, we need to collect delegation
tokens (i.e. delegation token for NameNode, KMS).
{{addDelegationTokens}} is invoked several times, currently dt for NN is got
only once if exists. But dt for KMS is got every time, we should fix this.
Running in real cluster:
{quote}
14/10/08 02:30:52 INFO security.TokenCache: Got dt for
hdfs://hnode1.sh.intel.com:9000; Kind: HDFS_DELEGATION_TOKEN, Service:
10.239.47.8:9000, Ident: (HDFS_DELEGATION_TOKEN token 13 for user)
14/10/08 02:30:52 WARN token.Token: Cannot find class for token kind kms-dt
14/10/08 02:30:52 INFO security.TokenCache: Got dt for
hdfs://hnode1.sh.intel.com:9000; Kind: kms-dt, Service: 10.239.47.8:16000,
Ident: 00 04 75 73 65 72 04 79 61 72 6e 00 8a 01 48 eb e0 d2 76 8a 01 49 0f ed
56 76 06 02
14/10/08 02:30:53 WARN token.Token: Cannot find class for token kind kms-dt
14/10/08 02:30:53 INFO security.TokenCache: Got dt for
hdfs://hnode1.sh.intel.com:9000; Kind: kms-dt, Service: 10.239.47.8:16000,
Ident: 00 04 75 73 65 72 04 79 61 72 6e 00 8a 01 48 eb e0 d3 85 8a 01 49 0f ed
57 85 07 02
14/10/08 02:30:53 WARN token.Token: Cannot find class for token kind kms-dt
14/10/08 02:30:53 INFO security.TokenCache: Got dt for
hdfs://hnode1.sh.intel.com:9000; Kind: kms-dt, Service: 10.239.47.8:16000,
Ident: 00 04 75 73 65 72 04 79 61 72 6e 00 8a 01 48 eb e0 d3 8e 8a 01 49 0f ed
57 8e 08 02
14/10/08 02:30:53 WARN token.Token: Cannot find class for token kind kms-dt
14/10/08 02:30:53 INFO security.TokenCache: Got dt for
hdfs://hnode1.sh.intel.com:9000; Kind: kms-dt, Service: 10.239.47.8:16000,
Ident: 00 04 75 73 65 72 04 79 61 72 6e 00 8a 01 48 eb e0 d4 ef 8a 01 49 0f ed
58 ef 09 02
{quote}
was:
When submit MapReduce job in security mode, we need to collect delegation
tokens (i.e. delegation token for NameNode, KMS).
{{addDelegationTokens}} may be invoked several times, currently dt for NN is
got only once if exists. But dt for KMS is got every time, we should fix this.
> Delegation token for KMS should only be got once if it already exists
> ---------------------------------------------------------------------
>
> Key: HDFS-7205
> URL: https://issues.apache.org/jira/browse/HDFS-7205
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: encryption
> Affects Versions: 2.6.0
> Reporter: Yi Liu
> Assignee: Yi Liu
>
> When submit MapReduce job in security mode, we need to collect delegation
> tokens (i.e. delegation token for NameNode, KMS).
> {{addDelegationTokens}} is invoked several times, currently dt for NN is got
> only once if exists. But dt for KMS is got every time, we should fix this.
> Running in real cluster:
> {quote}
> 14/10/08 02:30:52 INFO security.TokenCache: Got dt for
> hdfs://hnode1.sh.intel.com:9000; Kind: HDFS_DELEGATION_TOKEN, Service:
> 10.239.47.8:9000, Ident: (HDFS_DELEGATION_TOKEN token 13 for user)
> 14/10/08 02:30:52 WARN token.Token: Cannot find class for token kind kms-dt
> 14/10/08 02:30:52 INFO security.TokenCache: Got dt for
> hdfs://hnode1.sh.intel.com:9000; Kind: kms-dt, Service: 10.239.47.8:16000,
> Ident: 00 04 75 73 65 72 04 79 61 72 6e 00 8a 01 48 eb e0 d2 76 8a 01 49 0f
> ed 56 76 06 02
> 14/10/08 02:30:53 WARN token.Token: Cannot find class for token kind kms-dt
> 14/10/08 02:30:53 INFO security.TokenCache: Got dt for
> hdfs://hnode1.sh.intel.com:9000; Kind: kms-dt, Service: 10.239.47.8:16000,
> Ident: 00 04 75 73 65 72 04 79 61 72 6e 00 8a 01 48 eb e0 d3 85 8a 01 49 0f
> ed 57 85 07 02
> 14/10/08 02:30:53 WARN token.Token: Cannot find class for token kind kms-dt
> 14/10/08 02:30:53 INFO security.TokenCache: Got dt for
> hdfs://hnode1.sh.intel.com:9000; Kind: kms-dt, Service: 10.239.47.8:16000,
> Ident: 00 04 75 73 65 72 04 79 61 72 6e 00 8a 01 48 eb e0 d3 8e 8a 01 49 0f
> ed 57 8e 08 02
> 14/10/08 02:30:53 WARN token.Token: Cannot find class for token kind kms-dt
> 14/10/08 02:30:53 INFO security.TokenCache: Got dt for
> hdfs://hnode1.sh.intel.com:9000; Kind: kms-dt, Service: 10.239.47.8:16000,
> Ident: 00 04 75 73 65 72 04 79 61 72 6e 00 8a 01 48 eb e0 d4 ef 8a 01 49 0f
> ed 58 ef 09 02
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
