steveloughran commented on code in PR #6140:
URL: https://github.com/apache/hadoop/pull/6140#discussion_r1372139869
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEncryptionDSSEKMSUserDefinedKey.java:
##########
@@ -39,10 +42,12 @@ protected Configuration createConfiguration() {
Configuration c = new Configuration();
String kmsKey = S3AUtils.getS3EncryptionKey(getTestBucketName(c), c);
// skip the test if DSSE-KMS or KMS key not set.
- if (StringUtils.isBlank(kmsKey)) {
- skip(S3_ENCRYPTION_KEY + " is not set for " +
- DSSE_KMS.getMethod());
+ try {
+ skipIfEncryptionNotSet(c, DSSE_KMS);
+ } catch (IOException e) {
+ throw new RuntimeException(e);
Review Comment:
if this was production i'd ask for an UncheckedIOException(); not so worried
here
##########
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/encryption.md:
##########
@@ -316,6 +320,79 @@ metadata. Since only one encryption key can be provided
at a time, S3A will not
pass the correct encryption key to decrypt the data.
+### <a name="dsse-kms"></a> DSSE-KMS: Dual-layer Server-Encryption with KMS
Managed Encryption Keys
+
+By providing a dual-layer server-side encryption mechanism using AWS Key
Management Service
+(AWS KMS) keys, known as DSSE-KMS, two layers of encryption are applied to
objects upon their
+upload to Amazon S3. DSSE-KMS simplifies the process of meeting compliance
requirements that
+mandate the implementation of multiple layers of encryption for data while
maintaining complete
+control over the encryption keys.
+
+
+When uploading data encrypted with SSE-KMS, the sequence is as follows:
+
+1. The S3A client must declare a specific CMK in the property
`fs.s3a.encryption.key`, or leave
+ it blank to use the default configured for that region.
+
+2. The S3A client uploads all the data as normal, now including encryption
information.
+
+3. The S3 service encrypts the data with a symmetric key unique to the new
object.
+
+4. The S3 service retrieves the chosen CMK key from the KMS service, and, if
the user has
+ the right to use it, uses it to provide dual-layer encryption for the data.
+
+
+When downloading DSSE-KMS encrypted data, the sequence is as follows
+
+1. The S3A client issues an HTTP GET request to read the data.
+
+2. S3 sees that the data was encrypted with DSSE-KMS, and looks up the
specific key in the
+ KMS service.
+
+3. If and only if the requesting user has been granted permission to use the
CMS key does
+ the KMS service provide S3 with the key.
+
+4. As a result, S3 will only decode the data if the user has been granted
access to the key.
+
+
Review Comment:
can you add a "further reading" link to the aws docs on this; its a
complicated topic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
