adoroszlai opened a new pull request #2912: URL: https://github.com/apache/ozone/pull/2912
## What changes were proposed in this pull request? S3 copy object fails for source key with special char. Noticed with `=`, as in `key=value`, which is used for Spark SQL partitioned tables. `put-object` creates the key with literal `=`: ``` Running command 'aws s3api --endpoint-url http://s3g:9878 put-object --bucket bucket-ozone-test-9766114691 --key ozone-test-7644827888/copyobject/key=value/f1 --body /tmp/copyfile ... Running command 'aws s3api --endpoint-url http://s3g:9878 list-objects --bucket bucket-ozone-test-9766114691 --prefix ozone-test-7644827888/copyobject/key=value/ ${output} = { "Contents": [ { "LastModified": "2021-12-11T14:19:26.002Z", "ETag": "2021-12-11T14:19:26.002Z", "StorageClass": "STANDARD", "Key": "ozone-test-7644827888/copyobject/key=value/f1", "Size": 29 } ] } ``` But the source key path for `copy-object` is passed to S3 Gateway via a HTTP header, which is URL-encoded. This mismatch results in the following error: ``` Running command 'aws s3api --endpoint-url http://s3g:9878 copy-object --bucket destbucket-85046 --key ozone-test-7644827888/copyobject/key=value/f1 --copy-source bucket-ozone-test-9766114691/ozone-test-7644827888/copyobject/key=value/f1 ${rc} = 255 ${output} = An error occurred (NoSuchKey) when calling the CopyObject operation: The specified key does not exist ... <Error> <Code>NoSuchKey</Code> <Message>The specified key does not exist</Message> <Resource>ozone-test-7644827888/copyobject/key%3Dvalue/f1</Resource> <RequestId/> </Error> ``` This patch fixes the error by URL-decoding the source key path. https://issues.apache.org/jira/browse/HDDS-6096 ## How was this patch tested? Changed unit and acceptance tests to cover `key=value` case. ``` Running command 'aws s3api --endpoint-url http://s3g:9878 copy-object --bucket destbucket-44546 --key ozone-test-2143247000/copyobject/key=value/f1 --copy-source bucket-ozone-test-8252066797/ozone-test-2143247000/copyobject/key=value/f1' ${rc} = 0 ${output} = { "CopyObjectResult": { "LastModified": "2021-12-11T12:21:04.632Z", "ETag": "923bc384-0308-4d4a-b7e8-ededf46a4808" } } ``` CI: https://github.com/adoroszlai/hadoop-ozone/runs/4492631771 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
