[
https://issues.apache.org/jira/browse/HDFS-5907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arpit Agarwal updated HDFS-5907:
--------------------------------
Attachment: HDFS-5907.04.patch
Thanks for the detailed review [~sureshms]. Addressed everything not mentioned
below and also feedback from [~vinayrpet].
{quote}
Why is on finalize the blocks in trash are not being deleted?
{quote}
They are deleted when NN signals finalize by the absence of
{{RollingUpgradeStatus}} in the {{HeartbeatResponse}}. This triggers purge via
{{BPOfferService#signalRollingUpgrade}}.
{quote}
Sorry I missed the logic of ENUM_WITH_ROLLING_UPGRAE_OPTION and its been a
while, but I also forgot about why we added DFS_DATANODE_STARTUP_KEY in the
first place.
{quote}
The logic for {{ENUM_WITH_ROLLING_UPGRADE_OPTION}} is required because the
{{enum.toString}} was changed separately to include
{{RollingUpgradeStartupOption}} and we need to have a corresponding parse
method. This is the static {{#getEnum}}.
{quote}
Also the logic is not quite right. restoreDirectory is always going to be null
right. Checking for null seems redundant and also you can just rename the trash
to getRestoreDirectory(child).
{quote}
Good catch, the assignment to null should have been outside the for loop, this
is to avoid recreating it if multiple times when restoring more than one child
in the same parent. Fixed it.
{quote}
Unit tests required for patterns added to BlockPoolSliceStorage, and methods
getTrashDirectory, getRestoreDirectory. Also these methods might be easier to
test, if it accepts string
{quote}
I've added a number of unit tests in this version of the patch. Still working
on an E2E test for DN finalize/rollback.
> Handle block deletion requests during rolling upgrades
> ------------------------------------------------------
>
> Key: HDFS-5907
> URL: https://issues.apache.org/jira/browse/HDFS-5907
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Affects Versions: HDFS-5535 (Rolling upgrades)
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Attachments: HDFS-5907.01.patch, HDFS-5907.02.patch,
> HDFS-5907.04.patch
>
>
> DN changes when a rolling upgrade is in progress:
> # DataNode should handle block deletions by moving block files to 'trash'.
> # Block files should be restored to their original locations during a
> rollback.
> # Purge trash when the rolling upgrade is finalized.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
