[
https://issues.apache.org/jira/browse/HDDS-11256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17870066#comment-17870066
]
Ivan Andika commented on HDDS-11256:
------------------------------------
{quote}The description sort of implies this already, but our implementation of
delete recovery would likely be tied to the protocol being used
{quote}
I see. This seems to be a valid direction the community might tackle the issue.
Although, I think this might require OM to know which type of client sends the
deletion (e.g. to handle deletion though S3 for FSO bucket).
Additionally, I think S3 object versioning has not been fully implemented in OM
due to various reasons (e.g. OM DB management) and seems that S3 object
versioning is enabled per bucket instead of per cluster. The OM key trash
feature currently aims to enable it for the whole OM service.
I'm putting this proposal since some community members are interested in it.
I'm perfectly fine if the community decides to go another direction for this
issue.
> OM Key Trash Feature
> --------------------
>
> Key: HDDS-11256
> URL: https://issues.apache.org/jira/browse/HDDS-11256
> Project: Apache Ozone
> Issue Type: New Feature
> Components: OM
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
>
> Context: Currently Ozone supports Trash feature with similar implementation
> as HDFS trash feature. However, this only protects large scale deletion
> through Hadoop client (e.g. "hadoop fs -rm -r"). However, other deletions
> using -skipTrash or through S3 are not protected by this trash.
> We are currently working on the implementation in our internal cluster. We
> started with idea of DN block trash (similar to HDFS-12996), but realized
> that this requires changes in all Ozone components and the complexity will be
> very high. The final design implements an OM-based solution that resembles
> HDDS-2416 (Ozone Trash Feature):
> * There is a separate "Trash Table" that hold the deleted keys
> * There will be a background service that checks the "Trash Table" for
> deleted keys older than a certain expiry threshold and move them to the
> deletedTable for normal deletion (a trash cleanup service)
> * In the event of large accidental deletions, an admin can call a "recover"
> request which will query the trash tables and return it back to the original
> keys
> However, there are also some planned implementation differences that can be
> covered in a design document, which includes things like:
> * Another table which stores the modificationTime as RDB key for faster DB
> traversal
> * Recovery request starts a stateful background service on OM (similar to
> ContainerBalancer) to handle large amount of keys
> * Enabling the trash on runtime using a new OM request which uses Ratis
> transaction to ensure consistency of the OM DB. This will set a flag in OM
> metaTable that is used by OM to decide whether to use normal OM deletion or a
> "trash" deletion
> * Runtime reconfigurability of trash cleanup service parameters
> If the community members are interested and see the need for this feature, I
> can come up with a more complete design document.
> As I understand, Ozone already supports snapshot which is able to protect
> against accidental deletion, thus there might be a lot of overlaps. Due to
> these overlaps, it might not make sense to have this trash feature.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
