[
https://issues.apache.org/jira/browse/HADOOP-19197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17853304#comment-17853304
]
Viraj Jasani commented on HADOOP-19197:
---------------------------------------
How about we allow user to configure _fs.s3a.encryption.context_ similar to how
we allow for {_}fs.s3a.aws.credentials.provider.mapping{_}? i.e. key-value pair
of String values, let S3A take care of converting the key-value pairs to Base64
encoded JSON of String key-value pairs.
Given that the context is anyways sent in plain text (it's just Base64 encoded
JSON String, not a secret key), we can allow user to configure plain text
key-value pairs separate by "=" with {_}fs.s3a.encryption.context{_}.
Sample validation error when we pass anything other than Base64 encoded Json:
{code:java}
Caused by: software.amazon.awssdk.services.s3.model.S3Exception: The header
'x-amz-server-side-encryption-context' shall be Base64-encoded UTF-8 string
holding JSON which represents a string-string map (Service: S3, Status Code:
400, Request ID: SC3CA6BGC8B8RBRD, Extended Request ID:
8iCVA0qZsxlPXxkDpR49Gtah5LlcgTojtoHyvSEvdY25Kqow5/SPMtXIzuIKzgra16t5e23VQIc6iNle0FhcGw==){code}
> S3A: Support AWS KMS Encryption Context
> ---------------------------------------
>
> Key: HADOOP-19197
> URL: https://issues.apache.org/jira/browse/HADOOP-19197
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Raphael Azzolini
> Priority: Major
>
> S3A properties allow users to choose the AWS KMS key
> ({_}fs.s3a.encryption.key{_}) and S3 encryption algorithm to be used
> (f{_}s.s3a.encryption.algorithm{_}). In addition to the AWS KMS Key, an
> encryption context can be used as non-secret data that adds additional
> integrity and authenticity to check the encrypted data. However, there is no
> option to specify the [AWS KMS Encryption
> Context|https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#encrypt_context]
> in S3A.
> In AWS SDK v2 the encryption context in S3 requests is set by the parameter
> [ssekmsEncryptionContext.|https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/CreateMultipartUploadRequest.Builder.html#ssekmsEncryptionContext(java.lang.String)]
> It receives a base64-encoded UTF-8 string holding JSON with the encryption
> context key-value pairs. The value of this parameter could be set by the user
> in a new property {_}*fs.s3a.encryption.context*{_}, and be stored in the
> [EncryptionSecrets|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/EncryptionSecrets.java]
> to later be used when setting the encryption parameters in
> [RequestFactoryImpl|https://github.com/apache/hadoop/blob/f92a8ab8ae54f11946412904973eb60404dee7ff/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/RequestFactoryImpl.java].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
