[
https://issues.apache.org/jira/browse/HADOOP-10433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13958858#comment-13958858
]
Larry McCay commented on HADOOP-10433:
--------------------------------------
I don't think that features are an issue at this point.
My larger concern is whether Hadoop common needs a KMS server offering of its
own or not.
Given that the keystore provider is not likely a scalable provider for a KMS
that will lead us down the path of requiring an appropriate DB to do it right.
This was my thinking as I starting prototyping on an earlier version of the
KeyProvider API as well.
It became reasonable to me that the KeyProvider API within Hadoop common was
sufficient to enable deployments to plugin any number of various KeyProvider
implementations. This might be:
1. direct to KMIP
2. to an OpenStack Barbican server
3. to a Knox (or wherever) KMS
This would allow Hadoop common to not have to take on the weight of another
server.
I'm not really arguing against putting it common here but recollecting my
thought process on the matter.
Here is a question that I have had trouble answering with regard to the server
and API being in common together...
Given the pluggable nature of the KeyProvider API what value does adding a full
KMS server with additional (and maybe redundant) pluggability to common provide?
I've had trouble reconciling that in my mind.
Now, take that same server implementation and move it out of common and it
easily makes sense to be able to have pluggability for the Hadoop KeyProvider
API, KMIP and others.
So, when I said "as long as it make sense to do so", I was really talking about
whether it made sense to have it colocated with the KeyProvider API in common.
I think the feature set is less of an issue.
> Key Management Server based on KeyProvider API
> ----------------------------------------------
>
> Key: HADOOP-10433
> URL: https://issues.apache.org/jira/browse/HADOOP-10433
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
> Affects Versions: 3.0.0
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Attachments: HADOOP-10433-v2.patch, HADOOP-10433-v3.patch,
> HADOOP-10433.patch, KMS-ALL-PATCHES-v2.patch, KMS-ALL-PATCHES-v3.patch,
> KMS-ALL-PATCHES.patch, KMS-doc.pdf
>
>
> (from HDFS-6134 proposal)
> Hadoop KMS is the gateway, for Hadoop and Hadoop clients, to the underlying
> KMS. It provides an interface that works with existing Hadoop security
> components (authenticatication, confidentiality).
> Hadoop KMS will be implemented leveraging the work being done in HADOOP-10141
> and HADOOP-10177.
> Hadoop KMS will provide an additional implementation of the Hadoop
> KeyProvider class. This implementation will be a client-server implementation.
> The client-server protocol will be secure:
> * Kerberos HTTP SPNEGO (authentication)
> * HTTPS for transport (confidentiality and integrity)
> * Hadoop ACLs (authorization)
> The Hadoop KMS implementation will not provide additional ACL to access
> encrypted files. For sophisticated access control requirements, HDFS ACLs
> (HDFS-4685) should be used.
> Basic key administration will be supported by the Hadoop KMS via the, already
> available, Hadoop KeyShell command line tool
> There are minor changes that must be done in Hadoop KeyProvider functionality:
> The KeyProvider contract, and the existing implementations, must be
> thread-safe
> KeyProvider API should have an API to generate the key material internally
> JavaKeyStoreProvider should use, if present, a password provided via
> configuration
> KeyProvider Option and Metadata should include a label (for easier
> cross-referencing)
> To avoid overloading the underlying KeyProvider implementation, the Hadoop
> KMS will cache keys using a TTL policy.
> Scalability and High Availability of the Hadoop KMS can achieved by running
> multiple instances behind a VIP/Load-Balancer. For High Availability, the
> underlying KeyProvider implementation used by the Hadoop KMS must be High
> Available.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
