[
https://issues.apache.org/jira/browse/HBASE-7544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603596#comment-13603596
]
Andrew Purtell commented on HBASE-7544:
---------------------------------------
bq. w.r.t. the compilation error
A 'mvn -Dcrypto -Dhadoop.profile=2.0 -DskipTests clean install' to get
everything in place in the local Maven cache followed by a 'mvn -Dcrypto
-Dhadoop.profile=2.0 test -Dtest=TestHFileEncryption' works for me here.
(I am using a locally patched version of Hadoop branch-2 with crypto support.
You can get it at
https://github.com/intel-hadoop/hadoop-common-rhino/tree/branch-2. Be sure to
compile Hadoop with -Pnative and include the directory holding the newly built
libhadoop.so in LD_LIBRARY_PATH or TestHFileEncryption won't pass.)
bq. In hbase-common/pom.xml
Yes that should be -Dcrypto. The 'crypto' or 'nocrypto' profiles are activated
depending on if that is defined or not. If you have some thoughts on how this
could be done better with Maven that would be great. Unfortunately a separate
module isn't feasible because of the changes in hbase-server. Originally I
didn't do any of this Maven hacking, I just used reflection in Encryption.java,
but I don't want to do it that way because reflection is brittle and slow. I
also need to use a different constructor for the WALReader in
SequenceFileLogReader. (This is because HBase uses the deprecated Hadoop 1
style constructors for SequenceFile and the crypto support for SequenceFile,
when using those constructors, requires a crypto context at instantiation.) So
at a minimum I had to separate out a crypto enabled SequenceFileLogReader from
a stock SequenceFileLogReader. This difference would be an excellent candidate
for a hadoop-compat module as soon as there's a Hadoop version suitable for
targeting.
> Transparent table/CF encryption
> -------------------------------
>
> Key: HBASE-7544
> URL: https://issues.apache.org/jira/browse/HBASE-7544
> Project: HBase
> Issue Type: New Feature
> Components: HFile, io
> Affects Versions: 0.96.0
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Attachments: 7544.patch, 7544.patch, 7544.pdf
>
>
> Introduce transparent encryption of HBase on disk data.
> Depends on a separate contribution of an encryption codec framework to Hadoop
> core and an AES-NI (native code) codec. This is work done in the context of
> MAPREDUCE-4491 but I'd gather there will be additional JIRAs for common and
> HDFS parts of it.
> Requirements:
> - Transparent encryption at the CF or table level
> - Protect against all data leakage from files at rest
> - Two-tier key architecture for consistency with best practices for this
> feature in the RDBMS world
> - Built-in key management
> - Flexible and non-intrusive key rotation
> - Mechanisms not exposed to or modifiable by users
> - Hardware security module integration (via Java KeyStore)
> - HBCK support for transparently encrypted files (+ plugin architecture for
> HBCK)
> Additional goals:
> - Shell support for administrative functions
> - Avoid performance impact for the null crypto codec case
> - Play nicely with other changes underway: in HFile, block coding, etc.
> We're aiming for rough parity with Oracle's transparent tablespace encryption
> feature, described in
> http://www.oracle.com/technetwork/database/owp-security-advanced-security-11gr-133411.pdf
> as
> {quote}
> “Transparent Data Encryption uses a 2-tier key architecture for flexible and
> non-intrusive key rotation and least operational and performance impact: Each
> application table with at least one encrypted column has its own table key,
> which is applied to all encrypted columns in that table. Equally, each
> encrypted tablespace has its own tablespace key. Table keys are stored in the
> data dictionary of the database, while tablespace keys are stored in the
> header of the tablespace and additionally, the header of each underlying OS
> file that makes up the tablespace. Each of these keys is encrypted with the
> TDE master encryption key, which is stored outside of the database in an
> external security module: either the Oracle Wallet (a PKCS#12 formatted file
> that is encrypted using a passphrase supplied either by the designated
> security administrator or DBA during setup), or a Hardware Security Module
> (HSM) device for higher assurance […]”
> {quote}
> Further design details forthcoming in a design document and patch as soon as
> we have all of the clearances in place.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
