[ https://issues.apache.org/jira/browse/HADOOP-16119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16775262#comment-16775262 ]
Wei-Chiu Chuang commented on HADOOP-16119: ------------------------------------------ Attached the design doc draft in the jira, please review: [^Design doc_ KMS v2.pdf] HADOOP-15977 (RPC support for TLS) will benefit this work greatly, but there's nothing that prevents the two work in parallel. I am taking a first crack at this project but feel free to contribute to the design doc and subtasks. > KMS on Hadoop RPC Engine > ------------------------ > > Key: HADOOP-16119 > URL: https://issues.apache.org/jira/browse/HADOOP-16119 > Project: Hadoop Common > Issue Type: New Feature > Reporter: Jonathan Eagles > Assignee: Wei-Chiu Chuang > Priority: Major > Attachments: Design doc_ KMS v2.pdf > > > Per discussion on common-dev and text copied here for ease of reference. > https://lists.apache.org/thread.html/0e2eeaf07b013f17fad6d362393f53d52041828feec53dcddff04808@%3Ccommon-dev.hadoop.apache.org%3E > {noformat} > Thanks all for the inputs, > To offer additional information (while Daryn is working on his stuff), > optimizing RPC encryption opens up another possibility: migrating KMS > service to use Hadoop RPC. > Today's KMS uses HTTPS + REST API, much like webhdfs. It has very > undesirable performance (a few thousand ops per second) compared to > NameNode. Unfortunately for each NameNode namespace operation you also need > to access KMS too. > Migrating KMS to Hadoop RPC greatly improves its performance (if > implemented correctly), and RPC encryption would be a prerequisite. So > please keep that in mind when discussing the Hadoop RPC encryption > improvements. Cloudera is very interested to help with the Hadoop RPC > encryption project because a lot of our customers are using at-rest > encryption, and some of them are starting to hit KMS performance limit. > This whole "migrating KMS to Hadoop RPC" was Daryn's idea. I heard this > idea in the meetup and I am very thrilled to see this happening because it > is a real issue bothering some of our customers, and I suspect it is the > right solution to address this tech debt. > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org