[
https://issues.apache.org/jira/browse/HDFS-5907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898467#comment-13898467
]
Arpit Agarwal edited comment on HDFS-5907 at 2/11/14 11:12 PM:
---------------------------------------------------------------
Preliminary patch, needs test cases.
I was able to test delete/finalize manually. Was not able to test rollback yet.
The patch adds a trash directory under {{BlockPoolSliceStorage}}. The trash
directory location is {{SD/current/BP-.../trash/}} and its sub-directory layout
mirrors that of {{SD/current/BP-.../current/}}. This allows keeping memory of
where to restore the file on rollback.
An alternative is to just rename the file under its current location to
indicate it is a deleted block but this makes finalize and rollback expensive
as we may have to potentially iterate over millions of block files.
The DN detects whether a rolling upgrade is in progress or not by looking for
the presence of {{RollingUpgradeStatus}} in {{HeartbeatResponse}}.
If a block is deleted for a BPID undergoing rolling upgrade then it is moved to
a trash directory instead. On finalize the trash directory is emptied. On
rollback, files under trash/ are restored to their corresponding locations
under current/.
was (Author: arpitagarwal):
Preliminary patch, needs test cases.
I was able to test delete/finalize manually. Was not able to test rollback yet.
The patch adds a trash directory under {{BlockPoolSliceStorage}}. The trash
directory location is {{SD/current/BP-.../trash/}} and its sub-directory layout
mirrors that of {{SD/current/BP-.../current/}}. This allows keeping memory of
where to restore the file on rollback.
An alternative is to just rename the file under its current location to
indicate it is a deleted block but this makes rollback expensive as we may have
to potentially iterate over millions of block files.
The DN detects whether a rolling upgrade is in progress or not by looking for
the presence of {{RollingUpgradeStatus}} in {{HeartbeatResponse}}.
If a block is deleted for a BPID undergoing rolling upgrade then it is moved to
a trash directory instead. On finalize the trash directory is emptied. On
rollback, files under trash/ are restored to their corresponding locations
under current/.
> Handle block deletion requests during rolling upgrades
> ------------------------------------------------------
>
> Key: HDFS-5907
> URL: https://issues.apache.org/jira/browse/HDFS-5907
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Affects Versions: HDFS-5535 (Rolling upgrades)
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Attachments: HDFS-5907.01.patch
>
>
> Jira to track the following:
> # During a rolling upgrade the DataNode should handle block deletions by
> creating hard links to the block files.
> # Files must be restored during a rollback.
> # Cleanup hard links during finalize.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
