[
https://issues.apache.org/jira/browse/HADOOP-14445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16320470#comment-16320470
]
Rushabh S Shah commented on HADOOP-14445:
-----------------------------------------
Attaching a new patch for trunk which fixes the incompatibility that were
discussed in previous comments and addressing the review comments made by Xiao
on the previous patch.
Added a bunch of test cases to verify compatibility.
bq. I think we can improve the comments with dtService to be clearer. Suggest
something along the lines of:
Addressed in latest patch.
bq. Now that we handle the port stuff nicely in unit tests,
fallbackDefaultPortForTesting and related logic can be removed.
Addressed in latest patch.
bq. On renew and cancel, suggest to add a debug log when keyProvider == null
for supportability.
Addressed in latest patch.
bq. Let's use HADOOP_SECURITY_KEY_PROVIDER_PATH instead of
KeyProviderFactory.KEY_PROVIDER_PATH.
Addressed in latest patch.
bq. When createKeyProviderForTests returns non-null value (before return kp),
add a info log, since this should only happen in tests
Addressed in latest patch.
bq. doKMSWithZKWithDelegationToken, do we need to loop through the tokens and
verify? After this fix, there would only be 1 kms-dt mapping to the entire url
right? IMO we should verify there's just 1 kms-dt.
Addressed in latest patch.
bq. doKMSWithZKWithDelegationToken, besides verifying renewal, we should also
verify some key operations.
This patch has nothing to do with key operations. Key shell commands don't use
delegation tokens. They use kerberos tickets.
This jira is only changing delegation token handling part. The existing key
shell tests are enough.
If you think existing tests are not enough, then please open a new jira to
cover that.
bq. Happy to see the compat test, thanks! We should also verify some key
operations too here.
Same comment as last one.
bq.HdfsKMSUtil: Looks like we can remove the not-used createKeyProvider method.
This method is getting called from
{{DFSUtil#createKeyProviderCryptoExtension}}. Namenode calls this method to
create key provider.
So we cannot remove this method.
[~xiaochen]: please review.
> Delegation tokens are not shared between KMS instances
> ------------------------------------------------------
>
> Key: HADOOP-14445
> URL: https://issues.apache.org/jira/browse/HADOOP-14445
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
> Affects Versions: 2.8.0, 3.0.0-alpha1
> Environment: CDH5.7.4, Kerberized, SSL, KMS-HA, at rest encryption
> Reporter: Wei-Chiu Chuang
> Assignee: Rushabh S Shah
> Attachments: HADOOP-14445-branch-2.8.002.patch,
> HADOOP-14445-branch-2.8.patch, HADOOP-14445.002.patch, HADOOP-14445.003.patch
>
>
> As discovered in HADOOP-14441, KMS HA using LoadBalancingKMSClientProvider do
> not share delegation tokens. (a client uses KMS address/port as the key for
> delegation token)
> {code:title=DelegationTokenAuthenticatedURL#openConnection}
> if (!creds.getAllTokens().isEmpty()) {
> InetSocketAddress serviceAddr = new InetSocketAddress(url.getHost(),
> url.getPort());
> Text service = SecurityUtil.buildTokenService(serviceAddr);
> dToken = creds.getToken(service);
> {code}
> But KMS doc states:
> {quote}
> Delegation Tokens
> Similar to HTTP authentication, KMS uses Hadoop Authentication for delegation
> tokens too.
> Under HA, A KMS instance must verify the delegation token given by another
> KMS instance, by checking the shared secret used to sign the delegation
> token. To do this, all KMS instances must be able to retrieve the shared
> secret from ZooKeeper.
> {quote}
> We should either update the KMS documentation, or fix this code to share
> delegation tokens.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
