[
https://issues.apache.org/jira/browse/HADOOP-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16804667#comment-16804667
]
Anu Engineer commented on HADOOP-16144:
---------------------------------------
Hi [~daryn],
This is a just the first draft of the KMSv2 based on Hadoop RPC. This is also
my first ever patch on KMS, please holler if I have done something colossally
stupid. I really appreciate the chance to write a KMS from scratch, Thanks.
Here overview and patterns (and rationale) used in this patch.
1. KeyManagementProtos.proto - Contains all our proto definitions. It follows a
pattern that we have used extensively in Ozone. That has one RPC call
(submitRequest) and a union of commands for both request and responses. This is
very useful when we want to add tracing, audit logging, and profiling, etc.
Single entry and exit point in the server makes it easy for the rest of the
code. Other than this protobuf file, all files other files are in
_??/hadoop-common-project/hadoop-kms/src/main/java/org/apache/hadoop/crypto/key/kms/v2??_
2. Under the v2 directory, you will find the *protocol directory*. The classes
under it follow the standard Hadoop RPC pattern. The client and Server side
stubs.
2. The server code is in *v2/server* -- I have broken up each function to
xxxOps.java file, so that is where the core logic is. For example, deleteKey
logic will be in DeleteKeyOps.java.
3. KMSv2Server.java -- all calls follow a standard pattern; they are
decodeRequest, call the main function and then call makeResponse.
4. The Server Dispatch functions in KMSV2ProtocolServerSideTranslatorPB decode
the request and get us to type safety. This is where we will add the Audit
functionality.
5. What is missing --
1. Tests -- I have just started working on it.
2. CLI Client -- I will get to that soon.
3. Genesis -- A Microbench marking tool
4. Kerberos -- I have commented out the Kerberos code for now.
5. ACL check -- this is kind of strange -- when I got the first calls to
createkey working, the providers were checking and enforcing Key ACLS. Looks
like in the current KMS code path we might be doing ACLs check across different
places. We might want to clean that up. I will get to that once this core code
is done.
6. At some point, I would like to introduce Ratis based provider, so Ozone can
run independently of ZooKeeper. That is long term.
7. KMS audit, it looks like a small change on this patch, I will probably add
that before the final patch.
+Again, Work under progress+, just posting a patch so that you are not left
wondering what the hell is this guy doing. I will get to tests tomorrow and
post a better patch over the weekend. Please think of this patch as a proposal
and I am very open to re-architecture based on new ideas, so feel free to
comment including future looking ideas. Since we are rewriting this from
scratch, we can accommodate any future looking ideas.
Thanks
Anu
[~jojochuang], [~xyao], [~elek] FYI.
> Create a Hadoop RPC based KMS client
> ------------------------------------
>
> Key: HADOOP-16144
> URL: https://issues.apache.org/jira/browse/HADOOP-16144
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: kms
> Reporter: Wei-Chiu Chuang
> Assignee: Anu Engineer
> Priority: Major
> Attachments: KMS.RPC.patch
>
>
> Create a new KMS client implementation that speaks Hadoop RPC.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
