[
https://issues.apache.org/jira/browse/HDDS-11532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Andika updated HDDS-11532:
-------------------------------
Description:
[https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListMultipartUploads.html]
!image-2024-10-05-15-26-03-339.png|width=492,height=69!
ListMultipartUploads result enforces sorting order based on key and time (only
for general purpose bucket).
Currently, Ozone "sorts" the result based on the upload ID since the DB key for
multipartInfoTable is "/\{VOLUME_NAME}/\{BUCKET_NAME}/\{KEY_NAME}/\{OBJECT_ID}"
(for LEGACY and OBS bucket). Since the DB key for multipart uploads with the
same key has the same prefix "/\{VOLUME_NAME}/\{BUCKET_NAME}/\{KEY_NAME}/", the
key-based sorting should be handled (assuming DB keys only contains ASCII
characters, see the note at the bottom). Therefore, to enforce the time-based
sorting, we can sort the multipart uploads with the same key based on the
initiate time. We can also choose to only support this for LEGACY / OBS bucket.
Note: Currently the StringCodec uses UTF-8 which might be encoded to different
number of bytes depending of the character, and since RocksDB only cares about
bytes, these might cause some unexpected sorting order. However, AFAIK for
ASCII characters, UTF-8 will encoding is always 1 byte, so if the DB key are
always ASCII, the key name sorting might hold.
was:
[https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListMultipartUploads.html]
!image-2024-10-05-15-26-03-339.png|width=492,height=69!
ListMultipartUploads result enforces sorting order based on key and time (only
for general purpose bucket).
Currently, Ozone "sorts" the result based on the upload ID since the DB key for
multipartInfoTable is "/\{VOLUME_NAME}/\{BUCKET_NAME}/\{KEY_NAME}/\{OBJECT_ID}"
(for LEGACY and OBS bucket). Since the DB key for multipart uploads with the
same key has the same prefix "/\{VOLUME_NAME}/\{BUCKET_NAME}/\{KEY_NAME}/", the
key-based sorting should be handled (assuming DB keys only contains ASCII
characters, see the note at the bottom). Therefore, to enforce the time-based
sorting, we can sort the multipart uploads with the same key based on the
initiate time.
Note: Currently the StringCodec uses UTF-8 which might be encoded to different
number of bytes depending of the character, and since RocksDB only cares about
bytes, these might cause some unexpected sorting order. However, AFAIK for
ASCII characters, UTF-8 will encoding is always 1 byte, so if the DB key are
always ASCII, the key name sorting might hold.
> Sort multipart uploads on ListMultipartUploads response
> -------------------------------------------------------
>
> Key: HDDS-11532
> URL: https://issues.apache.org/jira/browse/HDDS-11532
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: s3gateway
> Reporter: Ivan Andika
> Priority: Minor
> Attachments: image-2024-10-05-15-26-03-339.png
>
>
> [https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListMultipartUploads.html]
> !image-2024-10-05-15-26-03-339.png|width=492,height=69!
> ListMultipartUploads result enforces sorting order based on key and time
> (only for general purpose bucket).
> Currently, Ozone "sorts" the result based on the upload ID since the DB key
> for multipartInfoTable is
> "/\{VOLUME_NAME}/\{BUCKET_NAME}/\{KEY_NAME}/\{OBJECT_ID}" (for LEGACY and OBS
> bucket). Since the DB key for multipart uploads with the same key has the
> same prefix "/\{VOLUME_NAME}/\{BUCKET_NAME}/\{KEY_NAME}/", the key-based
> sorting should be handled (assuming DB keys only contains ASCII characters,
> see the note at the bottom). Therefore, to enforce the time-based sorting, we
> can sort the multipart uploads with the same key based on the initiate time.
> We can also choose to only support this for LEGACY / OBS bucket.
> Note: Currently the StringCodec uses UTF-8 which might be encoded to
> different number of bytes depending of the character, and since RocksDB only
> cares about bytes, these might cause some unexpected sorting order. However,
> AFAIK for ASCII characters, UTF-8 will encoding is always 1 byte, so if the
> DB key are always ASCII, the key name sorting might hold.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
