[
https://issues.apache.org/jira/browse/HDDS-5975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharat Viswanadham resolved HDDS-5975.
--------------------------------------
Fix Version/s: 1.3.0
Resolution: Fixed
> S3G: object key name should be encoded
> --------------------------------------
>
> Key: HDDS-5975
> URL: https://issues.apache.org/jira/browse/HDDS-5975
> Project: Apache Ozone
> Issue Type: Bug
> Components: S3
> Affects Versions: 1.1.0
> Reporter: Kohei Sugihara
> Assignee: Kohei Sugihara
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.3.0
>
>
> S3G will not work properly for some objects that is includes `+` and ` ` in a
> key/prefix name. They should be url-encoded.
> h1. Quick reproduce
> `aws s3 rm --recursive` will not work correctly because it confuse `+` and `
> `.
> {code:sh}
> # create files and sync them
> % ls special-files -l
> total 0
> -rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:39 '&=;:+ ,?'
> -rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:36 '<abc>'
> -rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:41 '\{^}%'
> -rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:41 '\{^}%`]>[~<#|'
> -rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:24 'a b c'
> -rw-r--r-- 1 ksugihara ksugihara 0 Nov 12 08:21 a+b+c
> % aws s3 --endpoint HOST sync special-files s3://should-be-deleted
> upload: special-files/&=;:+ ,? to s3://should-be-deleted/&=;:+ ,?
> upload: special-files/<abc> to s3://should-be-deleted/<abc>
> upload: special-files/a b c to s3://should-be-deleted/a b c
> upload: special-files/a+b+c to s3://should-be-deleted/a+b+c
> upload: special-files/\{^}%`]>[~<#| to s3://should-be-deleted/\{^}%`]>[~<#|
> upload: special-files/\{^}% to s3://should-be-deleted/\{^}%
> # two 'a b c' files in this bucket. '+' and ' ' should be identified
> % aws s3 --endpoint HOST ls s3://should-be-deleted/
> 2021-11-12 14:34:29 0 &=;: ,?
> 2021-11-12 14:34:29 0 <abc>
> 2021-11-12 14:34:29 0 \{^}%
> 2021-11-12 14:34:29 0 \{^}%`]>[~<#|
> 2021-11-12 14:34:29 0 a b c
> 2021-11-12 14:34:29 0 a b c
> # let's delete all files in the bucket
> % aws s3 --endpoint HOST rm --recursive s3://should-be-deleted/
> delete: s3://should-be-deleted/&=;: ,?
> delete: s3://should-be-deleted/<abc>
> delete: s3://should-be-deleted/a b c
> delete: s3://should-be-deleted/\{^}%
> delete: s3://should-be-deleted/a b c
> delete: s3://should-be-deleted/\{^}%`]>[~<#|
> # several objects still remain
> % aws s3 --endpoint HOST ls s3://should-be-deleted
> 2021-11-12 14:34:29 0 &=;: ,?
> 2021-11-12 14:34:29 0 a b c
> {code}
> {code:xml}
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/";>
> <Name>should-be-deleted</Name>
> <Prefix></Prefix>
> <Marker></Marker>
> <MaxKeys>1000</MaxKeys>
> <KeyCount>2</KeyCount>
> <Delimiter>/</Delimiter>
> <EncodingType>url</EncodingType>
> <IsTruncated>false</IsTruncated>
> <Contents>
> <Key>&=;:+ ,?</Key>
> <LastModified>2021-11-12T05:34:29.337Z</LastModified>
> <ETag>2021-11-12T05:34:29.337Z</ETag>
> <Size>0</Size>
> <StorageClass>STANDARD</StorageClass>
> </Contents>
> <Contents>
> <Key>a+b+c</Key>
> <LastModified>2021-11-12T05:34:29.610Z</LastModified>
> <ETag>2021-11-12T05:34:29.610Z</ETag>
> <Size>0</Size>
> <StorageClass>STANDARD</StorageClass>
> </Contents>
> </ListBucketResult>
> {code}
> h1. References
> * [https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html] –
> a guideline for object key name & special handlings
> * https://issues.apache.org/jira/browse/HDDS-3161 – plus (`+`) is allowed
> here
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
