[
https://issues.apache.org/jira/browse/HDFS-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Chen updated HDFS-10899:
-----------------------------
Attachment: HDFS-10899.12.patch
Thanks for the review [~jojochuang], patch 12 to address all comments.
- Updated cancel to cancel any pending futures. Do not think we need to wait
for the ongoing operation (aka if the updater is processing a completed future)
to stop .
- {{ReencryptHandler#startThreads/stopThreads}} renamed for readability.
- Also updated docs about my snapshot point above.
bq. did you notice any significant NameNode pause during re-encryption other
than GC pauses?
As clarified offline, didn't see any RPC hang, since we're not holding locks
when contacting KMS, which appears to be the bottleneck.
Re-encryption does have to take the read/write locks when iterating the EZ /
updating xattrs, respectively. But since we're not holding the locks
continuously, and throttling will be done based on KMS latency, my gut feeling
is this will not significantly block NN (or pause).
Will try to test with some jobs running during re-encryption to verify - after
the throttling code is done.
> Add functionality to re-encrypt EDEKs
> -------------------------------------
>
> Key: HDFS-10899
> URL: https://issues.apache.org/jira/browse/HDFS-10899
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: encryption, kms
> Reporter: Xiao Chen
> Assignee: Xiao Chen
> Attachments: editsStored, HDFS-10899.01.patch, HDFS-10899.02.patch,
> HDFS-10899.03.patch, HDFS-10899.04.patch, HDFS-10899.05.patch,
> HDFS-10899.06.patch, HDFS-10899.07.patch, HDFS-10899.08.patch,
> HDFS-10899.09.patch, HDFS-10899.10.patch, HDFS-10899.10.wip.patch,
> HDFS-10899.11.patch, HDFS-10899.12.patch, HDFS-10899.wip.2.patch,
> HDFS-10899.wip.patch, Re-encrypt edek design doc.pdf, Re-encrypt edek design
> doc V2.pdf
>
>
> Currently when an encryption zone (EZ) key is rotated, it only takes effect
> on new EDEKs. We should provide a way to re-encrypt EDEKs after the EZ key
> rotation, for improved security.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
