[
https://issues.apache.org/jira/browse/HDDS-11256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869861#comment-17869861
]
Ivan Andika commented on HDDS-11256:
------------------------------------
cc: [~Nicholas Niu] since I heard from [~Symious] that you are interested in
this feature.
> OM Key Trash Feature
> --------------------
>
> Key: HDDS-11256
> URL: https://issues.apache.org/jira/browse/HDDS-11256
> Project: Apache Ozone
> Issue Type: New Feature
> Components: OM
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
>
> Context: Currently Ozone supports Trash feature with similar implementation
> as HDFS trash feature. However, this only protects large scale deletion
> through Hadoop client (e.g. "hadoop fs -rm -r"). However, other deletions
> using -skipTrash or through S3 are not protected by this trash.
> We are currently working on the implementation in our internal cluster. We
> started with idea of DN block trash (similar to HDFS-12996), but realized
> that this requires changes in all Ozone components and the complexity will be
> very high. The final design implements an OM-based solution that resembles
> HDDS-2416 (Ozone Trash Feature):
> * There is a separate "Trash Table" that hold the deleted keys
> * There will be a background service that checks the "Trash Table" for
> deleted keys older than a certain expiry threshold and move them to the
> deletedTable for normal deletion (a trash cleanup service)
> * In the event of large accidental deletions, an admin can call a "recover"
> request which will query the trash tables and return it back to the original
> keys
> However, there are also some planned implementation differences that can be
> covered in a design document, which includes things like:
> * Another table which stores the modificationTime as RDB key for faster DB
> traversal
> * Recovery request starts a stateful background service on OM (similar to
> ContainerBalancer) to handle large amount of keys
> * Enabling the trash on runtime using a new OM request which uses Ratis
> transaction to ensure consistency of the OM DB. This will set a flag in OM
> metaTable that is used by OM to decide whether to use normal OM deletion or a
> "trash" deletion
> * Runtime reconfigurability of trash cleanup service parameters
> If the community members are interested and see the need for this feature, I
> can come up with a more complete design document.
> As I understand, Ozone already supports snapshot which is able to protect
> against accidental deletion, thus there might be a lot of overlaps. Due to
> these overlaps, it might not make sense to have this trash feature.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
