[
https://issues.apache.org/jira/browse/HADOOP-16794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025960#comment-17025960
]
Mukund Thakur commented on HADOOP-16794:
----------------------------------------
Debugging details:
When a put operation is performed without -d parameter is used, it happens in
two steps. First is copying to a temporary file and then copying the temp file
to the right location. The is issue is durning the copy step as an extra header
"x-amz-server-side-encryption: aws:kms" is set.
When -d parameter is used, the file is uploaded directly and in that api call
this kms header is not set.
Solution: Removing the kms header from the ClonedObjectMetatData.
Steps for performing testing:
* Create a bucket in s3 console.
* Create a CMK in KMS console.
* Update the bucket encryption configurations to use SSE-KMS and select the
key created above.
* Compile the hadoop with the patch and generate new distribution.
* Run bin/hadoop fs -put <file> <bucket>
* Verify the encryption key set for the <file> in s3 console. It should match
with the CMK created in the KMS console.
Bucket used for testing :
[https://mthakur-data.s3.ap-south-1.amazonaws.com/file2]
> S3 Encryption is always using default region-specific AWS-managed KMS key
> -------------------------------------------------------------------------
>
> Key: HADOOP-16794
> URL: https://issues.apache.org/jira/browse/HADOOP-16794
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.2.1
> Reporter: Mukund Thakur
> Priority: Major
>
> When using (bucket-level) S3 Default Encryption with SSE-KMS and a CMK, all
> files uploaded via the HDFS {{FileSystem}} {{s3a://}} scheme receive the
> wrong encryption key, always falling back to the region-specific AWS-managed
> KMS key for S3, instead of retaining the custom CMK.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
