[
https://issues.apache.org/jira/browse/HDFS-11393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15856158#comment-15856158
]
Daryn Sharp commented on HDFS-11393:
------------------------------------
We've seen the same in our initial testing. There's not a good way to
selectively obtain the token other than add/remove the kms conf setting.
That's certainly less than ideal since EZs are supposed to be transparent to
the user.
One impediment with the current model is that apis for obtaining tokens aren't
path based but filesystem based. If they were path based and clients correctly
premeditated all paths to be accessed, the token could be obtained if needed.
But... Even if a client premeditates it will access a given path, there's no
way to know that deeper in that path is an EZ. Symlinks pose a similar issue.
About the only real way to solve the problem (in the current security model) is
allowing the client to auth to the kms via the NN token. That would be a
massive change that creates a tight coupling with the services, requires shared
key distribution, more critical path moving parts, complicates the kms ability
to enforce its own ACLs, etc.
In the end, for us the job submission rate is so insignificant compared to all
other job generated load that it's an acceptable, albeit unnecessary, overhead.
> Hadoop KMS contacted by jobs which don’t use KMS encryption
> ------------------------------------------------------------
>
> Key: HDFS-11393
> URL: https://issues.apache.org/jira/browse/HDFS-11393
> Project: Hadoop HDFS
> Issue Type: Wish
> Environment: Hadoop 2.7.3, Spark 1.6.3 on Yarn, Oozie 4.2.3
> Cluster secured with Kerberos
> Reporter: Alexandre Linte
> Priority: Minor
>
> Hello,
> After few days of usage of Hadoop KMS in our pre-production platform, it was
> noticed that after restarting resourcemanagers, all Yarn jobs generated on
> the platform interrogated the KMS server, even if the didn't process
> encrypted information.
> {noformat}
> 2016-11-23 10:58:47,708 DEBUG AuthenticationFilter - Request
> [http://uabigkms01:16000/kms/v1/?op=GETDELEGATIONTOKEN&renewer=rm%2Fuabigrm01%40SANDBOX.HADOOP]
> triggering authentication
> 2016-11-23 10:58:47,735 DEBUG AuthenticationFilter - Request
> [http://uabigkms01:16000/kms/v1/?op=GETDELEGATIONTOKEN&renewer=rm%2Fuabigrm01%40SANDBOX.HADOOP]
> user xxxx authenticated
> {noformat}
> Indeed after research we see that KMS supports delegation token to
> authenticate to the Java KeyProvider by processes without Kerberos
> credentials.
> Is there a way to bypass Delegation Token on KMS and just contact KMS when
> jobs or user into HDFS use encrypted data ?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
