[
https://issues.apache.org/jira/browse/HDFS-12996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16321337#comment-16321337
]
Hanisha Koneru commented on HDFS-12996:
---------------------------------------
Thanks for the review, [~shahrs87].
bq. Suppose user1 and user2 deleted some of their directories (lets say dir1
and dir2 respectively). If user1 wants to recover its directory, then we will
recover dir2 as well ?
Yes. In the current design, recovery is done by rolling back to an earlier
image. We could separately build a more fine-grained recovery mechanism on top
of the replica trash.
bq. Many of our clients(lets say user1) use /tmp/<userId> to store their
intermediate task output (to work around quota problems). After a job
completes, they delete this space and use the same location to store next job
output. In the meantime if some other user(lets say user2) wants to recover
their mistakenly deleted directory then we will go back in time for user1 which
might corrupt user1's output directory.
True. This again would be a trade-off between recovering the deleted data and
undoing operations performed after the delete operation. Only an administrator
can make this call.
The goal of this feature is to provide a safe-guard to recover from
catastrophic mistakes where it is acceptable to lose a few recent changes to
recover deleted data.
bq. Also the design looks very similar to Checkpointing/Snapshots.
--> Didn't get what you mean by checkpointing in this context. If you take
frequent rolling snapshots e.g. hourly snapshots of root directory from a cron
job, then you don't need this feature and you can recover deleted files from a
recent snapshot. However very few clusters are setup for this.
> DataNode Replica Trash
> ----------------------
>
> Key: HDFS-12996
> URL: https://issues.apache.org/jira/browse/HDFS-12996
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Hanisha Koneru
> Assignee: Hanisha Koneru
> Attachments: DataNode_Replica_Trash_Design_Doc.pdf
>
>
> DataNode Replica Trash will allow administrators to recover from a recent
> delete request that resulted in catastrophic loss of user data. This is
> achieved by placing all invalidated blocks in a replica trash on the datanode
> before completely purging them from the system. The design doc is attached
> here.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
