[
https://issues.apache.org/jira/browse/HADOOP-19013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17840901#comment-17840901
]
ASF GitHub Bot commented on HADOOP-19013:
-----------------------------------------
steveloughran commented on code in PR #6646:
URL: https://github.com/apache/hadoop/pull/6646#discussion_r1534430622
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/EncryptionTestUtils.java:
##########
@@ -111,4 +119,27 @@ public static void assertEncrypted(S3AFileSystem fs,
}
}
+ /**
+ * Assert that a path is encrypted with right encryption settings.
+ * @param fs filesystem.
+ * @param path path
+ * @param algorithm encryption algorithm.
+ * @param kmsKey full kms key if present.
+ * @throws IOException any IOE.
+ */
+ public static void validateEncryptionFileAttributes(S3AFileSystem fs,
+ Path path,
+ String algorithm,
+ Optional<String> kmsKey)
throws IOException {
+ Map<String, byte[]> xAttrs = fs.getXAttrs(path);
+
Assertions.assertThat(HeaderProcessing.decodeBytes(xAttrs.get(XA_SERVER_SIDE_ENCRYPTION)))
Review Comment:
assert that the .get isn't null(), you can use .extracting to chain
```
assertThat(xAttrs.get(XA_SERVER_SIDE_ENCRYPTION))
.describedAs(...)
.isNotNull()
.extracting(HeaderProcessing::decodeBytes())
.isEqualTo(...)
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/EncryptionTestUtils.java:
##########
@@ -111,4 +119,27 @@ public static void assertEncrypted(S3AFileSystem fs,
}
}
+ /**
+ * Assert that a path is encrypted with right encryption settings.
+ * @param fs filesystem.
+ * @param path path
+ * @param algorithm encryption algorithm.
+ * @param kmsKey full kms key if present.
+ * @throws IOException any IOE.
+ */
+ public static void validateEncryptionFileAttributes(S3AFileSystem fs,
+ Path path,
+ String algorithm,
+ Optional<String> kmsKey)
throws IOException {
+ Map<String, byte[]> xAttrs = fs.getXAttrs(path);
+
Assertions.assertThat(HeaderProcessing.decodeBytes(xAttrs.get(XA_SERVER_SIDE_ENCRYPTION)))
+ .describedAs("Server side encryption algorithm must match")
+ .isEqualTo(algorithm);
+ Assertions.assertThat(xAttrs.containsKey(XA_ENCRYPTION_KEY_ID))
Review Comment:
there's a specific assertion on a map containing a value
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEncryptionWithDefaultS3Settings.java:
##########
@@ -97,6 +104,21 @@ protected void assertEncrypted(Path path) throws
IOException {
EncryptionTestUtils.assertEncrypted(fs, path, SSE_KMS, kmsKey);
}
+ @Test
+ public void testEncryptionFileAttributes() throws Exception {
+ Path path = path(createFilename(1024));
Review Comment:
add a describe() for the logs
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AEncryptionSSEKMSDefaultKey.java:
##########
@@ -19,12 +19,20 @@
package org.apache.hadoop.fs.s3a;
import java.io.IOException;
+import java.util.Optional;
+import org.apache.hadoop.fs.contract.ContractTestUtils;
Review Comment:
wrong location
> fs.getXattrs(path) for S3FS doesn't have
> x-amz-server-side-encryption-aws-kms-key-id header.
> --------------------------------------------------------------------------------------------
>
> Key: HADOOP-19013
> URL: https://issues.apache.org/jira/browse/HADOOP-19013
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.6
> Reporter: Mukund Thakur
> Assignee: Mukund Thakur
> Priority: Major
> Labels: pull-request-available
>
> Once a path while uploading has been encrypted with SSE-KMS with a key id and
> then later when we try to read the attributes of the same file, it doesn't
> contain the key id information as an attribute. should we add it?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
