[
https://issues.apache.org/jira/browse/HDDS-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18057251#comment-18057251
]
Ivan Andika edited comment on HDDS-9095 at 2/9/26 5:50 AM:
-----------------------------------------------------------
[~ashishkr] We can use S3 SDK to listMultipartUploads on the bucket to list all
the incomplete multipart uploads and then abort all the multipart uploads.
Let me know if this works. This behavior is the same with AWS S3 (i.e. user
need to clear incomplete multipart uploads before the bucket can be deleted).
> Since old mpu keys will be orphan in just MultiPartInfoTable and not in
> openKeyTable
The MPU keys also have corresponding openKeyTable entry with format
/\{vol}/\{bucket}/\{key}{*}/\{uploadId}{*} (see HDDS-9098). In the future, we
can remove the openKeyTable entry.
was (Author: JIRAUSER298977):
[~ashishkr] We can use S3 SDK to listMultipartUploads on the bucket to list all
the incomplete multipart uploads and then abort all the multipart uploads.
Let me know if this works.
> Since old mpu keys will be orphan in just MultiPartInfoTable and not in
> openKeyTable
The MPU keys also have corresponding openKeyTable entry with format
/\{vol}/\{bucket}/\{key}{*}/\{uploadId}{*} (see HDDS-9098). In the future, we
can remove the openKeyTable entry.
> Cleanup expired incomplete MPUs
> -------------------------------
>
> Key: HDDS-9095
> URL: https://issues.apache.org/jira/browse/HDDS-9095
> Project: Apache Ozone
> Issue Type: Improvement
> Components: OM
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
> Fix For: 1.4.0
>
>
> In our cluster, there are around few thousands MPU keys in
> multipartInfoTable, with some of them being few months old.
> The reason is that these MPU keys are already initiated, and possibly
> committed few parts, but was not completed / aborted by the user. These
> spaces can be freed.
> Similar to the cleanup service OM open key table (HDDS-4120), we can
> implement clean up on MultipartInfoTable (and related open keys in
> OpenKeyTable) using a background job. -However, instead of using a new OM
> request / response, we can reuse the OM MPU abort request / response instead,
> which already handles the cleanup (i.e. expired inflight MPU can be aborted
> after a defined expiry threshold).-
>
> This is similar to {{AbortIncompleteMultipartUpload}} lifecycle
> configurations in AWS S3
> ([https://aws.amazon.com/blogs/aws-cloud-financial-management/discovering-and-deleting-incomplete-multipart-uploads-to-lower-amazon-s3-costs/]).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
