[
https://issues.apache.org/jira/browse/HBASE-25263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230669#comment-17230669
]
Mate Szalay-Beko commented on HBASE-25263:
------------------------------------------
These are very good points, thanks!
{quote}... each HFile has all of the necessary metadata for supporting its
decryption stored within it ...
{quote}
Yes, I didn't think it through deep enough before asking the question. You are
right and this also means that we can change the derivation function without
causing problems with backward compatibility. We can even bulkload HFiles /
move snapshots to other cluster and all the "old" hfiles will be readable by
the "new" cluster (assuming the master encryption key is the same / still
known).
{quote}If our older key derivation function has been deemed insufficient then
all of the derived DEKs should probably also be treated as insufficient. The
guidance we give there is compaction will rewrite the files with newly derived
DEKs. Major compaction will rewrite most files, but not all, so there is scope
for further improvement here... We may want an unconditional major compaction
to ensure all DEKs are rotated.
{quote}
This is a valid point. Also an unconditional major compaction could be useful
for other things too (like changing HFile compression maybe?). I can check the
code and create a follow-up ticket later.
BTW in the documentation
([http://hbase.apache.org/book.html#hbase.encryption.server)] we only recommend
major compaction, we don't mention that this won't necessary rewrite all the
HFiles (I might need to update this):
{quote}Rotate the Data Key
To rotate the data key, first change the ColumnFamily key in the column
descriptor, then trigger a major compaction. When compaction is complete, all
HFiles will be re-encrypted using the new data key. Until the compaction
completes, the old HFiles will still be readable using the old key.
{quote}
Also a note for me: I should also update the documentation mentioning the key
derivation function we use in the HBase shell (and also mention that the users
will have a better control over the encryption key if they provide it from Java
code and not from the shell)
> Change encryption key generation algorithm used in the HBase shell
> ------------------------------------------------------------------
>
> Key: HBASE-25263
> URL: https://issues.apache.org/jira/browse/HBASE-25263
> Project: HBase
> Issue Type: Improvement
> Components: encryption, shell
> Reporter: Mate Szalay-Beko
> Assignee: Mate Szalay-Beko
> Priority: Major
>
> This ticket is a follow-up of HBASE-25181, where several issues were
> discussed on the PR:
> 1. Currently we use {{PBKDF2WithHmacSHA1}} key generation algorithm to
> generate a secret key for HFile / WalFile encryption, when the user is
> defining a string encryption key in the hbase shell. This algorithm is not
> secure enough and not allowed in certain environments (e.g. on FIPS compliant
> clusters). Our plan is to change it to e.g. {{PBKDF2WithHmacSHA384}}. If
> this would break backward compatibility, then we should make this algorithm
> configurable.
> Smaller issues:
> 2. In {{EncryptionUtil.createEncryptionContext}} the various encryption
> config checks should throw IllegalStateExceptions instead of
> RuntimeExceptions.
> 3. Test cases in {{TestEncryptionTest.java}} should be broken down into
> smaller tests.
> 4. {{TestEncryptionDisabled.java}} should use {{ExpectedException}} JUnit
> rule to validate exceptions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
