[
https://issues.apache.org/jira/browse/HDFS-13682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16513312#comment-16513312
]
Xiao Chen commented on HDFS-13682:
----------------------------------
Took an easier route and debugged branch-2. It turns out HADOOP-9747 does have
some effects here - specifically at [this
method|https://github.com/apache/hadoop/commit/59cf7588779145ad5850ad63426743dfe03d8347#diff-e6a2371b73365b7ba7ff9a266b9aa138L724].
When this meets the KMSCP's morph-based-on-ugi logic, the ugi being used as
actual changed from loginUgi to currentUgi. (Also has a weird HTTP 400 somehow,
which is fixed if contentType is set).
Following this, I confirmed if we change {{KMSCP#getActualUgi}}'s check from
{{actualUgi.hasKerberosCredentials()}} to {{!actualUgi.isFromKeytab() &&
!actualUgi.isFromTicket()}} (and making {{UGI#isFromTicket}} public of course),
the test passes. This appears to be a more 'compatible' change.
IMO we should still consider explicitly doing the KMS call using the NN login
ugi, this applies to both the {{getMetadata}} call during createEZ and the
\{{generateEncryptedKey}} call from startFile. Reason being these calls are
internal to the NN, and the hdfs rpc caller isn't expected to really interact
with the KMS in this case.
> Cannot create encryption zone after KMS auth token expires
> ----------------------------------------------------------
>
> Key: HDFS-13682
> URL: https://issues.apache.org/jira/browse/HDFS-13682
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: encryption, namenode
> Affects Versions: 3.0.0
> Reporter: Xiao Chen
> Assignee: Xiao Chen
> Priority: Critical
> Attachments: HDFS-13682.dirty.repro.branch-2.patch,
> HDFS-13682.dirty.repro.patch
>
>
> Our internal testing reported this behavior recently.
> {noformat}
> [root@nightly6x-1 ~]# sudo -u hdfs /usr/bin/kinit -kt
> /cdep/keytabs/hdfs.keytab hdfs -l 30d -r 30d
> [root@nightly6x-1 ~]# sudo -u hdfs klist
> Ticket cache: FILE:/tmp/krb5cc_994
> Default principal: [email protected]
> Valid starting Expires Service principal
> 06/12/2018 03:24:09 07/12/2018 03:24:09
> krbtgt/[email protected]
> [root@nightly6x-1 ~]# sudo -u hdfs hdfs crypto -createZone -keyName key77
> -path /user/systest/ez
> RemoteException:
> org.apache.hadoop.security.authentication.client.AuthenticationException:
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)
> {noformat}
> Upon further investigation, it's due to the KMS client (cached in HDFS NN)
> cannot authenticate with the server after the authentication token (which is
> cached by KMSCP) expires, even if the HDFS client RPC has valid kerberos
> credentials.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
