[
https://issues.apache.org/jira/browse/HADOOP-18708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774100#comment-17774100
]
ASF GitHub Bot commented on HADOOP-18708:
-----------------------------------------
steveloughran commented on code in PR #6164:
URL: https://github.com/apache/hadoop/pull/6164#discussion_r1355135994
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ClientFactory.java:
##########
@@ -82,6 +83,31 @@ S3Client createS3Client(URI uri,
S3AsyncClient createS3AsyncClient(URI uri,
S3ClientCreationParameters parameters) throws IOException;
+
+ /**
+ * Creates a new {@link software.amazon.encryption.s3.S3EncryptionClient}.
+ * Used when client side encryption is enabled.
+ *
+ * @param s3AsyncClient The asynchronous S3 client, will be used for
cryptographic operations.
+ * @param s3Client The synchronous S3 client, will be used for non
cryptographic operations.
+ * @param cseMaterials cse key and type to use
+ * @return S3EncryptionClient
+ */
+ S3Client createS3EncryptionClient(S3AsyncClient s3AsyncClient, S3Client
s3Client,
Review Comment:
1. have an EncryptionClientCreationParameters with CSEMaterials part of it
2. can we actually use a subclass of the default factory which does support
encryption.
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java:
##########
@@ -994,11 +996,25 @@ private void bindAWSClient(URI name, boolean dtEnabled)
throws IOException {
.withMultipartCopyEnabled(isMultipartCopyEnabled)
.withMultipartThreshold(multiPartThreshold)
.withTransferManagerExecutor(unboundedThreadPool)
- .withRegion(region);
+ .withRegion(region)
+ .withClientSideEncryptionEnabled(isCSEEnabled);
S3ClientFactory clientFactory =
ReflectionUtils.newInstance(s3ClientFactoryClass, conf);
s3Client = clientFactory.createS3Client(getUri(), parameters);
createS3AsyncClient(clientFactory, parameters);
+
Review Comment:
given its a new jar, any way we can isolate this stuff?
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AConfiguration.java:
##########
@@ -575,6 +579,12 @@ public void testS3SpecificSignerOverride() throws
Exception {
.describedAs("Custom STS signer not called").isTrue();
}
+ private void unsetClientSideConfiguration(Configuration conf) {
Review Comment:
change name. e.g unsetEncryption
##########
hadoop-tools/hadoop-aws/pom.xml:
##########
@@ -508,6 +508,11 @@
<artifactId>bundle</artifactId>
<scope>compile</scope>
</dependency>
+ <dependency>
+ <groupId>software.amazon.encryption.s3</groupId>
+ <artifactId>amazon-s3-encryption-client-java</artifactId>
+ <scope>compile</scope>
Review Comment:
we need this to be scoped as provided, with matching changes in the code
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ClientFactory.java:
##########
@@ -82,6 +83,31 @@ S3Client createS3Client(URI uri,
S3AsyncClient createS3AsyncClient(URI uri,
S3ClientCreationParameters parameters) throws IOException;
+
+ /**
+ * Creates a new {@link software.amazon.encryption.s3.S3EncryptionClient}.
+ * Used when client side encryption is enabled.
+ *
+ * @param s3AsyncClient The asynchronous S3 client, will be used for
cryptographic operations.
+ * @param s3Client The synchronous S3 client, will be used for non
cryptographic operations.
+ * @param cseMaterials cse key and type to use
+ * @return S3EncryptionClient
+ */
+ S3Client createS3EncryptionClient(S3AsyncClient s3AsyncClient, S3Client
s3Client,
+ CSEMaterials cseMaterials);
Review Comment:
add default throws new UnsupportedOperationException(). i've already got a
separate implementation for HBoss and this patch breaks it right now
> AWS SDK V2 - Implement CSE
> --------------------------
>
> Key: HADOOP-18708
> URL: https://issues.apache.org/jira/browse/HADOOP-18708
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Ahmar Suhail
> Priority: Major
> Labels: pull-request-available
>
> S3 Encryption client for SDK V2 is now available, so add client side
> encryption back in.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
