[ https://issues.apache.org/jira/browse/HADOOP-14445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649691#comment-16649691 ]
Xiao Chen edited comment on HADOOP-14445 at 10/15/18 3:45 AM: -------------------------------------------------------------- [~daryn] Do you mind another review? Sadly this needs an addendum for 2 things: * {{DelegationTokenIssuer}} class was recursively 'org/apache/hadoop/security/token' package twice... sorry didn't catch this during review * It caused 2 test failures in TestEncryptionZones. Pre-commit smartly skipped hadoop-hdfs (only ran hadoop-hdfs-client and hadoop-common), and it's caught when I try to backport to CDH where a full unit test was carried out. Out of the 2 failures, {{testDelegationToken}} needs to update the way it's mocked, and {{addMockKmsToken}} (another test method) caused mockito to give up, refusing to call the method on interface... (For thoroughness, internal pre-commit also complained about API compat, saying {{addDelegationTokens}} is removed from FileSystem and DistributedFileSystem; it also noted the same method is added to DelegationTokenIssuer, but not able to use the latter as a clue to cross off the former. So this part is clearly to be overruled) was (Author: xiaochen): [~daryn] sadly this needs an addendum for 2 things: * {{DelegationTokenIssuer}} class was recursively 'org/apache/hadoop/security/token' package twice... sorry didn't catch this during review * It caused 2 test failures in TestEncryptionZones. Pre-commit smartly skipped hadoop-hdfs (only ran hadoop-hdfs-client and hadoop-common), and it's caught when I try to backport to CDH where a full unit test was carried out. Out of the 2 failures, {{testDelegationToken}} needs to update the way it's mocked, and {{addMockKmsToken}} (another test method) caused mockito to give up, refusing to call the method on interface... (For thoroughness, internal pre-commit also complained about API compat, saying {{addDelegationTokens}} is removed from FileSystem and DistributedFileSystem; it also noted the same method is added to DelegationTokenIssuer, but not able to use the latter as a clue to cross off the former. So this part is clearly to be overruled) > Use DelegationTokenIssuer to create KMS delegation tokens that can > authenticate to all KMS instances > ---------------------------------------------------------------------------------------------------- > > Key: HADOOP-14445 > URL: https://issues.apache.org/jira/browse/HADOOP-14445 > Project: Hadoop Common > Issue Type: Bug > Components: kms > Affects Versions: 2.8.0, 3.0.0-alpha1 > Environment: CDH5.7.4, Kerberized, SSL, KMS-HA, at rest encryption > Reporter: Wei-Chiu Chuang > Assignee: Xiao Chen > Priority: Major > Fix For: 3.2.0, 3.0.4, 3.1.2 > > Attachments: HADOOP-14445-branch-2.8.002.patch, > HADOOP-14445-branch-2.8.patch, HADOOP-14445.002.patch, > HADOOP-14445.003.patch, HADOOP-14445.004.patch, HADOOP-14445.05.patch, > HADOOP-14445.06.patch, HADOOP-14445.07.patch, HADOOP-14445.08.patch, > HADOOP-14445.09.patch, HADOOP-14445.10.patch, HADOOP-14445.11.patch, > HADOOP-14445.12.patch, HADOOP-14445.13.patch, HADOOP-14445.14.patch, > HADOOP-14445.15.patch, HADOOP-14445.16.patch, HADOOP-14445.17.patch, > HADOOP-14445.18.patch, HADOOP-14445.19.patch, HADOOP-14445.20.patch, > HADOOP-14445.addemdum.patch, HADOOP-14445.branch-2.000.precommit.patch, > HADOOP-14445.branch-2.001.precommit.patch, HADOOP-14445.branch-2.01.patch, > HADOOP-14445.branch-2.02.patch, HADOOP-14445.branch-2.03.patch, > HADOOP-14445.branch-2.04.patch, HADOOP-14445.branch-2.05.patch, > HADOOP-14445.branch-2.06.patch, HADOOP-14445.branch-2.8.003.patch, > HADOOP-14445.branch-2.8.004.patch, HADOOP-14445.branch-2.8.005.patch, > HADOOP-14445.branch-2.8.006.patch, HADOOP-14445.branch-2.8.revert.patch, > HADOOP-14445.branch-3.0.001.patch, HADOOP-14445.compat.patch, > HADOOP-14445.revert.patch > > > As discovered in HADOOP-14441, KMS HA using LoadBalancingKMSClientProvider do > not share delegation tokens. (a client uses KMS address/port as the key for > delegation token) > {code:title=DelegationTokenAuthenticatedURL#openConnection} > if (!creds.getAllTokens().isEmpty()) { > InetSocketAddress serviceAddr = new InetSocketAddress(url.getHost(), > url.getPort()); > Text service = SecurityUtil.buildTokenService(serviceAddr); > dToken = creds.getToken(service); > {code} > But KMS doc states: > {quote} > Delegation Tokens > Similar to HTTP authentication, KMS uses Hadoop Authentication for delegation > tokens too. > Under HA, A KMS instance must verify the delegation token given by another > KMS instance, by checking the shared secret used to sign the delegation > token. To do this, all KMS instances must be able to retrieve the shared > secret from ZooKeeper. > {quote} > We should either update the KMS documentation, or fix this code to share > delegation tokens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org