[
https://issues.apache.org/jira/browse/HDDS-9095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18057268#comment-18057268
]
Ivan Andika edited comment on HDDS-9095 at 2/9/26 7:43 AM:
-----------------------------------------------------------
> No the abort is failing as entry in openKeyTable is missing
HDDS-9098 should exclude the MPU open key from getting deleted.
> As in old behaviour before this implementation only keys were getting deleted
> from openKeyTable
So the situation is there is a very old MPU key before HDDS-9098 where the
OpenKeyCleanupService deleted the openKeyTable entry (although
multipartInfoTable entry still exists)?
In that case, we probably can do two options
* We might need to push a fix to not throw an exception when the openKeyTable
entry does not exist
[https://github.com/apache/ozone/blob/917ea25ba6197e894c8734589ee16913b3e8a848/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartUploadAbortRequest.java#L153]
* We simply use RocksDB tool to remove the multipartInfoTable entry
(essentially aborting it).
Let me know what you think.
was (Author: JIRAUSER298977):
> No the abort is failing as entry in openKeyTable is missing
HDDS-9098 should exclude the MPU open key from getting deleted.
> As in old behaviour before this implementation only keys were getting deleted
> from openKeyTable
So the situation is there is a very old MPU key before HDDS-9098 where the
OpenKeyCleanupService deleted the openKeyTable entry (although
multipartInfoTable entry still exists)?
In that case, we probably can do two options
* We might need to push a fix to not throw an exception when the openKeyTable
entry does not exist
[https://github.com/apache/ozone/blob/917ea25ba6197e894c8734589ee16913b3e8a848/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartUploadAbortRequest.java#L153]
* We simply use RocksDB tool to remove the multipartInfoTable entry
(essentially aborting it).
Let me know what you think.
> Cleanup expired incomplete MPUs
> -------------------------------
>
> Key: HDDS-9095
> URL: https://issues.apache.org/jira/browse/HDDS-9095
> Project: Apache Ozone
> Issue Type: Improvement
> Components: OM
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
> Fix For: 1.4.0
>
>
> In our cluster, there are around few thousands MPU keys in
> multipartInfoTable, with some of them being few months old.
> The reason is that these MPU keys are already initiated, and possibly
> committed few parts, but was not completed / aborted by the user. These
> spaces can be freed.
> Similar to the cleanup service OM open key table (HDDS-4120), we can
> implement clean up on MultipartInfoTable (and related open keys in
> OpenKeyTable) using a background job. -However, instead of using a new OM
> request / response, we can reuse the OM MPU abort request / response instead,
> which already handles the cleanup (i.e. expired inflight MPU can be aborted
> after a defined expiry threshold).-
>
> This is similar to {{AbortIncompleteMultipartUpload}} lifecycle
> configurations in AWS S3
> ([https://aws.amazon.com/blogs/aws-cloud-financial-management/discovering-and-deleting-incomplete-multipart-uploads-to-lower-amazon-s3-costs/]).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
