[
https://issues.apache.org/jira/browse/HDDS-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18057264#comment-18057264
]
Ashish Kumar edited comment on HDDS-9095 at 2/9/26 7:29 AM:
------------------------------------------------------------
[~ivanandika] No the abort is failing as entry in openKeyTable is missing. As
in old behaviour before this implementation only keys were getting deleted from
openKeyTable. So those orphan MPUs are existing in MultiPartInfoTable and
cannot be aborted.
{code:java}
[OM StateMachine ApplyTransaction Thread -
0]-org.apache.hadoop.ozone.om.request.s3.multipart.S3MultipartUploadAbortRequest:
Abort Multipart request is failed for KeyName
1232bd558f424f69a156a320b7db6b5c.pdf in VolumeName/Bucket s3v/test
NO_SUCH_MULTIPART_UPLOAD_ERROR
org.apache.hadoop.ozone.om.exceptions.OMException: Abort Multipart Upload
Failed: volume: s3vbucket: test: abc.pdf
at
org.apache.hadoop.ozone.om.request.s3.multipart.S3MultipartUploadAbortRequest.validateAndUpdateCache(S3MultipartUploadAbortRequest.java:157)
{code}
was (Author: JIRAUSER304344):
No the abort is failing as entry in openKeyTable is missing. As in old
behaviour before this implementation only keys were getting deleted from
openKeyTable. So those orphan MPUs are existing in MultiPartInfoTable and
cannot be aborted.
{code:java}
[OM StateMachine ApplyTransaction Thread -
0]-org.apache.hadoop.ozone.om.request.s3.multipart.S3MultipartUploadAbortRequest:
Abort Multipart request is failed for KeyName
1232bd558f424f69a156a320b7db6b5c.pdf in VolumeName/Bucket s3v/test
NO_SUCH_MULTIPART_UPLOAD_ERROR
org.apache.hadoop.ozone.om.exceptions.OMException: Abort Multipart Upload
Failed: volume: s3vbucket: test: abc.pdf
at
org.apache.hadoop.ozone.om.request.s3.multipart.S3MultipartUploadAbortRequest.validateAndUpdateCache(S3MultipartUploadAbortRequest.java:157)
{code}
> Cleanup expired incomplete MPUs
> -------------------------------
>
> Key: HDDS-9095
> URL: https://issues.apache.org/jira/browse/HDDS-9095
> Project: Apache Ozone
> Issue Type: Improvement
> Components: OM
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
> Fix For: 1.4.0
>
>
> In our cluster, there are around few thousands MPU keys in
> multipartInfoTable, with some of them being few months old.
> The reason is that these MPU keys are already initiated, and possibly
> committed few parts, but was not completed / aborted by the user. These
> spaces can be freed.
> Similar to the cleanup service OM open key table (HDDS-4120), we can
> implement clean up on MultipartInfoTable (and related open keys in
> OpenKeyTable) using a background job. -However, instead of using a new OM
> request / response, we can reuse the OM MPU abort request / response instead,
> which already handles the cleanup (i.e. expired inflight MPU can be aborted
> after a defined expiry threshold).-
>
> This is similar to {{AbortIncompleteMultipartUpload}} lifecycle
> configurations in AWS S3
> ([https://aws.amazon.com/blogs/aws-cloud-financial-management/discovering-and-deleting-incomplete-multipart-uploads-to-lower-amazon-s3-costs/]).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
