Thank you everyone for the feedback. It was very helpful. Cheers.
--------------- Saad Mufti On Fri, Aug 18, 2017 at 3:20 PM, Andrew Purtell <apurt...@apache.org> wrote: > The Hadoop KMS in 2.6 or 2.7 can be suitable for demos or prototypes but I > would advise against using it for more than that. Recently the KMS has seen > a number of security improvements. Because it is fairly self contained, you > can check out branch-2.8 or branch-2, build everything, extract the KMS, > and use that. > > For what it is worth at my employer we are considering HDFS at rest > encryption. We are building our own key management infrastructure, > incorporating various security and business requirements, and will > implement to the KMS on-wire API for providing key management services to > HDFS. > > > > > On Fri, Aug 18, 2017 at 10:25 AM, Saad Mufti <saad.mu...@gmail.com> wrote: > > > Hi, > > > > I'm looking for some guidance as our security team is requiring us to > > implement encryption of our HBase data at rest and in motion. I'm reading > > the docs and doing research and the choice seems to be between doing it > at > > the HBase level or the more general HDFS level. > > > > I am leaning towards HDFS level as there is some other data that is > derived > > from HBase in HDFS and it would be nice to have that encrypted as well. > > Once set up the encryption is supposed to transparent to clients. We're > > still at HBase 1.0 level, we're using a Cloudera 5.5 based distribution > but > > no commercial license. For reasons I won't go into upgrading is not an > > option in the short term and we need to implement encryption before that > > > > But I have a warning in a google groups somewhere (can't find it anymore) > > that warns that HDFS level encryption doesn't play well with HBase if on > > Hadoop 2.6.x, which we're at. Does anyone know the specific issue, or if > > there is a specific ticket I can look at to see if our Hadoop distro > > includes that fix? > > > > Also, out of the box the Key Management Server included in Hadoop is > based > > on a simple file based Java Keystore and there are warnings that it is > not > > suitable for production environments. Cloudera has their own proprietary > > KMS but we don't have a license to it. Can anyone share what groups that > > use pure open source distros are using as their KMS when implementing > > encryption in production environments? > > > > Thanks in advance for any guidance you can provide. > > > > ---- > > Saad > > > > > > -- > Best regards, > Andrew > > Words like orphans lost among the crosstalk, meaning torn from truth's > decrepit hands > - A23, Crosstalk >
