[
https://issues.apache.org/jira/browse/HADOOP-19197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17853325#comment-17853325
]
ASF GitHub Bot commented on HADOOP-19197:
-----------------------------------------
raphaelazzolini opened a new pull request, #6874:
URL: https://github.com/apache/hadoop/pull/6874
Add the property fs.s3a.encryption.context that allow users to specify the
AWS KMS Encryption Context to be used in S3A.
The value of the encryption context is a key/value string that will be
Base64 encoded and set in the parameter ssekmsEncryptionContext from the S3
client.
Contributed by Raphael Azzolini
### Description of PR
This code change adds a new property to S3A: fs.s3a.encryption.context\
The property's value accepts a set of key/value attributes to be set on S3's
encryption context. The value of the property will be base64 encoded and set in
the parameter ssekmsEncryptionContext from the S3 client.
### How was this patch tested?
S3's head-object response doesn't contain the object encryption key.
Therefore, I enabled CloudTrails data logs in my bucket to verify that the
tests were passing the encryption context to the request.
I added this property to `auth-keys.xml`
```
<property>
<name>fs.s3a.encryption.context</name>
<value>
project=hadoop,
jira=HADOOP-19197,
component=fs/s3
</value>
</property>
```
Then I executed the following tests:
```
mvn clean verify -Dit.test=ITestS3AEncryption* -Dtest=none
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running
org.apache.hadoop.fs.s3a.ITestS3AEncryptionSSEKMSDefaultKeyWithEncryptionContext
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.10
s
> S3A: Support AWS KMS Encryption Context
> ---------------------------------------
>
> Key: HADOOP-19197
> URL: https://issues.apache.org/jira/browse/HADOOP-19197
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Raphael Azzolini
> Assignee: Viraj Jasani
> Priority: Major
>
> S3A properties allow users to choose the AWS KMS key
> ({_}fs.s3a.encryption.key{_}) and S3 encryption algorithm to be used
> (f{_}s.s3a.encryption.algorithm{_}). In addition to the AWS KMS Key, an
> encryption context can be used as non-secret data that adds additional
> integrity and authenticity to check the encrypted data. However, there is no
> option to specify the [AWS KMS Encryption
> Context|https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#encrypt_context]
> in S3A.
> In AWS SDK v2 the encryption context in S3 requests is set by the parameter
> [ssekmsEncryptionContext.|https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/CreateMultipartUploadRequest.Builder.html#ssekmsEncryptionContext(java.lang.String)]
> It receives a base64-encoded UTF-8 string holding JSON with the encryption
> context key-value pairs. The value of this parameter could be set by the user
> in a new property {_}*fs.s3a.encryption.context*{_}, and be stored in the
> [EncryptionSecrets|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/EncryptionSecrets.java]
> to later be used when setting the encryption parameters in
> [RequestFactoryImpl|https://github.com/apache/hadoop/blob/f92a8ab8ae54f11946412904973eb60404dee7ff/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/RequestFactoryImpl.java].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
