Ivan Andika created HDDS-11256:
----------------------------------
Summary: OM Key Trash Feature
Key: HDDS-11256
URL: https://issues.apache.org/jira/browse/HDDS-11256
Project: Apache Ozone
Issue Type: New Feature
Reporter: Ivan Andika
Assignee: Ivan Andika
Context: Currently Ozone supports Trash feature with similar implementation as
HDFS trash feature. However, this only protects large scale deletion through
Hadoop client (e.g. "hadoop fs -rm -r"). However, other deletions using
-skipTrash or through S3 are not protected by this trash.
We are currently working on the implementation in our internal cluster. We
started with idea of DN block trash (similar to HDFS-12996), but realized that
this requires changes in all Ozone components and the complexity will be very
high. The final design implements an OM-based solution that resembles HDDS-2416
(Ozone Trash Feature):
* There is a separate "Trash Table" that hold the deleted keys
* There will be a background service that checks the "Trash Table" for deleted
keys older than a certain expiry threshold and move them to the deletedTable
for normal deletion (a trash cleanup service)
* In the event of large accidental deletions, an admin can call a "recover"
request which will query the trash tables and return it back to the original
keys
However, there are also some planned implementation differences that can be
covered in a design document, which includes things like:
* Another table which stores the modificationTime as RDB key for faster DB
traversal
* Recovery request starts a stateful background service on OM (similar to
ContainerBalancer) to handle large amount of keys
* Enabling the trash on runtime using a new OM request which uses Ratis
transaction to ensure consistency of the OM DB. This will set a flag in OM
metaTable that is used by OM to decide whether to use normal OM deletion or a
"trash" deletion
* Runtime reconfigurability of trash cleanup service parameters
If the community members are interested and see the need for this feature, I
can come up with a more complete design document.
As I understand, Ozone already supports snapshot which is able to protect
against accidental deletion, thus there might be a lot of overlaps. Due to
these overlaps, it might not make sense to have this trash feature.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
