[
https://issues.apache.org/jira/browse/HADOOP-14445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649691#comment-16649691
]
Xiao Chen edited comment on HADOOP-14445 at 10/15/18 3:45 AM:
--
[~daryn] Do you mind another review? Sadly this needs an addendum for 2 things:
* {{DelegationTokenIssuer}} class was recursively
'org/apache/hadoop/security/token' package twice... sorry didn't catch this
during review
* It caused 2 test failures in TestEncryptionZones. Pre-commit smartly skipped
hadoop-hdfs (only ran hadoop-hdfs-client and hadoop-common), and it's caught
when I try to backport to CDH where a full unit test was carried out. Out of
the 2 failures, {{testDelegationToken}} needs to update the way it's mocked,
and {{addMockKmsToken}} (another test method) caused mockito to give up,
refusing to call the method on interface...
(For thoroughness, internal pre-commit also complained about API compat, saying
{{addDelegationTokens}} is removed from FileSystem and DistributedFileSystem;
it also noted the same method is added to DelegationTokenIssuer, but not able
to use the latter as a clue to cross off the former. So this part is clearly to
be overruled)
was (Author: xiaochen):
[~daryn] sadly this needs an addendum for 2 things:
* {{DelegationTokenIssuer}} class was recursively
'org/apache/hadoop/security/token' package twice... sorry didn't catch this
during review
* It caused 2 test failures in TestEncryptionZones. Pre-commit smartly skipped
hadoop-hdfs (only ran hadoop-hdfs-client and hadoop-common), and it's caught
when I try to backport to CDH where a full unit test was carried out. Out of
the 2 failures, {{testDelegationToken}} needs to update the way it's mocked,
and {{addMockKmsToken}} (another test method) caused mockito to give up,
refusing to call the method on interface...
(For thoroughness, internal pre-commit also complained about API compat, saying
{{addDelegationTokens}} is removed from FileSystem and DistributedFileSystem;
it also noted the same method is added to DelegationTokenIssuer, but not able
to use the latter as a clue to cross off the former. So this part is clearly to
be overruled)
> Use DelegationTokenIssuer to create KMS delegation tokens that can
> authenticate to all KMS instances
>
>
> Key: HADOOP-14445
> URL: https://issues.apache.org/jira/browse/HADOOP-14445
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 2.8.0, 3.0.0-alpha1
> Environment: CDH5.7.4, Kerberized, SSL, KMS-HA, at rest encryption
>Reporter: Wei-Chiu Chuang
>Assignee: Xiao Chen
>Priority: Major
> Fix For: 3.2.0, 3.0.4, 3.1.2
>
> Attachments: HADOOP-14445-branch-2.8.002.patch,
> HADOOP-14445-branch-2.8.patch, HADOOP-14445.002.patch,
> HADOOP-14445.003.patch, HADOOP-14445.004.patch, HADOOP-14445.05.patch,
> HADOOP-14445.06.patch, HADOOP-14445.07.patch, HADOOP-14445.08.patch,
> HADOOP-14445.09.patch, HADOOP-14445.10.patch, HADOOP-14445.11.patch,
> HADOOP-14445.12.patch, HADOOP-14445.13.patch, HADOOP-14445.14.patch,
> HADOOP-14445.15.patch, HADOOP-14445.16.patch, HADOOP-14445.17.patch,
> HADOOP-14445.18.patch, HADOOP-14445.19.patch, HADOOP-14445.20.patch,
> HADOOP-14445.addemdum.patch, HADOOP-14445.branch-2.000.precommit.patch,
> HADOOP-14445.branch-2.001.precommit.patch, HADOOP-14445.branch-2.01.patch,
> HADOOP-14445.branch-2.02.patch, HADOOP-14445.branch-2.03.patch,
> HADOOP-14445.branch-2.04.patch, HADOOP-14445.branch-2.05.patch,
> HADOOP-14445.branch-2.06.patch, HADOOP-14445.branch-2.8.003.patch,
> HADOOP-14445.branch-2.8.004.patch, HADOOP-14445.branch-2.8.005.patch,
> HADOOP-14445.branch-2.8.006.patch, HADOOP-14445.branch-2.8.revert.patch,
> HADOOP-14445.branch-3.0.001.patch, HADOOP-14445.compat.patch,
> HADOOP-14445.revert.patch
>
>
> As discovered in HADOOP-14441, KMS HA using LoadBalancingKMSClientProvider do
> not share delegation tokens. (a client uses KMS address/port as the key for
> delegation token)
> {code:title=DelegationTokenAuthenticatedURL#openConnection}
> if (!creds.getAllTokens().isEmpty()) {
> InetSocketAddress serviceAddr = new InetSocketAddress(url.getHost(),
> url.getPort());
> Text service = SecurityUtil.buildTokenService(serviceAddr);
> dToken = creds.getToken(service);
> {code}
> But KMS doc states:
> {quote}
> Delegation Tokens
> Similar to HTTP authentication, KMS uses Hadoop Authentication for delegation
> tokens too.
> Under HA, A KMS instance must verify the delegation token given by another
> KMS instance, by checking the shared secret used to sign the delegation
> token. To do this, all KMS instances must be able to retrieve the shared
>
