[
https://issues.apache.org/jira/browse/HDDS-742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Elek, Marton updated HDDS-742:
------------------------------
Description:
In s3 gateway the GET bucket endpoint is already implemented. It can return
with the available objects based on a given prefix.
([https://docs.aws.amazon.com/AmazonS3/latest/API/v2-RESTBucketGET.html)]
As it's defined the Delimiter parameter is used to reduce the response with
returning only the first-level keys and prefixes (aka directories)
{code:java}
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/";>
<Name>example-bucket</Name>
<Prefix></Prefix>
<KeyCount>2</KeyCount>
<MaxKeys>1000</MaxKeys>
<Delimiter>/</Delimiter>
<IsTruncated>false</IsTruncated>
<Contents>
<Key>sample.jpg</Key>
<LastModified>2011-02-26T01:56:20.000Z</LastModified>
<ETag>"bf1d737a4d46a19f3bced6905cc8b902"</ETag>
<Size>142863</Size>
<StorageClass>STANDARD</StorageClass>
</Contents>
<CommonPrefixes>
<Prefix>photos/</Prefix>
</CommonPrefixes>
</ListBucketResult>{code}
Here we can have multiple additional objects with photos/ prefix but they are
not added to the response.
The main problem in the ozone s3 implementation is that the Delimiter parameter
*should be optional.* In case of the delimiter is missing we should always
return all the keys without any common prefix simplification.
It requires for recursive directory listing which is used by s3a adapter.
was:
In s3 gateway the GET bucket endpoint is already implemented. It can return
with the available objects based on a given prefix.
([https://docs.aws.amazon.com/AmazonS3/latest/API/v2-RESTBucketGET.html)]
As it's defined the Delimiter parameter is used to reduce the response with
returning only the first-level keys and prefixes (aka directories)
{code:java}
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/";>
<Name>example-bucket</Name>
<Prefix></Prefix>
<KeyCount>2</KeyCount>
<MaxKeys>1000</MaxKeys>
<Delimiter>/</Delimiter>
<IsTruncated>false</IsTruncated>
<Contents>
<Key>sample.jpg</Key>
<LastModified>2011-02-26T01:56:20.000Z</LastModified>
<ETag>"bf1d737a4d46a19f3bced6905cc8b902"</ETag>
<Size>142863</Size>
<StorageClass>STANDARD</StorageClass>
</Contents>
<CommonPrefixes>
<Prefix>photos/</Prefix>
</CommonPrefixes>
</ListBucketResult>{code}
Here we can have multiple additional objects with photos/ prefix but they are
not added to the response.
The main problem in the ozone s3 implementation is that the Delimiter parameter
*should be optional.* In case of the delimiter is missing we always return with
all the keys without and common prefix simplification.
It requires for recursive directory listing which is used by s3a adapter.
> Handle object list requests (GET bucket) without prefix parameter
> -----------------------------------------------------------------
>
> Key: HDDS-742
> URL: https://issues.apache.org/jira/browse/HDDS-742
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: S3
> Reporter: Elek, Marton
> Assignee: Elek, Marton
> Priority: Major
> Attachments: HDDS-742.001.patch
>
>
> In s3 gateway the GET bucket endpoint is already implemented. It can return
> with the available objects based on a given prefix.
> ([https://docs.aws.amazon.com/AmazonS3/latest/API/v2-RESTBucketGET.html)]
> As it's defined the Delimiter parameter is used to reduce the response with
> returning only the first-level keys and prefixes (aka directories)
> {code:java}
> <ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/";>
> <Name>example-bucket</Name>
> <Prefix></Prefix>
> <KeyCount>2</KeyCount>
> <MaxKeys>1000</MaxKeys>
> <Delimiter>/</Delimiter>
> <IsTruncated>false</IsTruncated>
> <Contents>
> <Key>sample.jpg</Key>
> <LastModified>2011-02-26T01:56:20.000Z</LastModified>
> <ETag>"bf1d737a4d46a19f3bced6905cc8b902"</ETag>
> <Size>142863</Size>
> <StorageClass>STANDARD</StorageClass>
> </Contents>
> <CommonPrefixes>
> <Prefix>photos/</Prefix>
> </CommonPrefixes>
> </ListBucketResult>{code}
> Here we can have multiple additional objects with photos/ prefix but they are
> not added to the response.
>
> The main problem in the ozone s3 implementation is that the Delimiter
> parameter *should be optional.* In case of the delimiter is missing we should
> always return all the keys without any common prefix simplification.
> It requires for recursive directory listing which is used by s3a adapter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
