chungen0126 opened a new pull request, #11862: URL: https://github.com/apache/ozone/pull/11862
## What changes were proposed in this pull request? OmMetadataManagerImpl#getMultipartUploadKeys turns key-marker and upload-id-marker into a RocksDB seek key. The way the seek key was computed made ListMultipartUploads return wrong results in the two cases below. ### Problem 1: key-marker sorts before the prefix Create multipart uploads without completing them: dir1/dir2/file1, dir1/file2, dir1bh/file1, dir1bha/file1. aws s3api list-multipart-uploads --prefix dir1bh --key-marker dir1/dir2/file1 Actual: no uploads are returned. Expected: dir1bh/file1 and dir1bha/file1, as both are under the prefix and after the key-marker. ### Problem 2: the seek key skipped too much With key-marker only: 1. Create multipart uploads without completing them: `a`, `a/x`, `ab`, `b`. 2. `aws s3api list-multipart-uploads --key-marker a` Actual: only `b` is returned. Expected: `a/x`, `ab` and `b`, as they are all after key-marker `a`. With key-marker and upload-id-marker: 1. Create two multipart uploads for key `a` and one for key `b`, without completing them. 2. `aws s3api list-multipart-uploads --key-marker a --upload-id-marker 0` Actual: only the upload of `b` is returned. Expected: both uploads of `a` and the upload of `b`, as the upload IDs of `a` (e.g. `01a18bcc-...`) are lexicographically greater than `0`. ### Difference from AWS S3 Ozone lists multipart uploads in the order of their DB keys (/volume/bucket/key/uploadId), while AWS S3 lists them in the order of key names. / sorts after characters like . and -, so key a.txt is listed before key a in Ozone. This ordering is not introduced by this PR, but this PR makes both markers consistently mean a position in Ozone's order. Example with one upload for each of the keys a, a.txt, a/x, ab, b: Ozone: a.txt, a, a/x, ab, b AWS S3: a, a.txt, a/x, ab, b Difference: a and a.txt are swapped. The same uploads are listed. The difference above is visible to S3 clients and needs to be documented. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-17563 ## How was this patch tested? https://github.com/chungen0126/ozone/actions/runs/41021897804 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
