[
https://issues.apache.org/jira/browse/HADOOP-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12593251#action_12593251
]
dhruba borthakur commented on HADOOP-2656:
------------------------------------------
Here is a Test Plan for testing Distributed Upgrades for Block Generation Stamp
Test Cases
T1 Install build version 0.17 and create a new HDFS cluster of size 500 nodes..
Run random writer, sort, and then sort validation. Then shutdown cluster.
Install version 0.18 build on the cluster. Start cluster with the -upgrade
option.
Expected behavior: The Namenode should trigger an upgrade on all
Datanodes. Then the Namenode should exit safe mode automatically. Run sort
validation again to verify that data is intact.
T2 Start a DFS Upgrade and restart Namenode during the upgrade.
Expected Behaviour: Upgrade should proceed normally with appropriate
messages in Namenode log
T3 Start a DFS upgrade and restart some of the Datanodes while the upgrade is
occurring.
Expected behaviour: Upgrade should proceed normally with appropriate
messages in Datenode logs.
T4 Start an upgrade and stop two Datanodes.
Expected Behavior: Upgrade should complete successfully because at least
one replica of every block should still be available.
T4 Start an upgrade and stop half of the Datanodes.
Expected Behavior: Upgrade should stall because at least one replica of
every block would not be available. Verify that useful message is displayed in
the Namenode logs.
T5 Start a DFS upgrade and let it complete successfully. Then issue the admin
command to finalize the upgrade.
Excepted Behavior: The Namenode should be able to finalize the cluster.
T5 Same as T4. Once the upgrade completes successfully, bring up the two
Datanodes that were shut down earlier.
Expected behavior: These two Datanodes should start their own upgrades.
Note that the Namenode could have already replicated these blocks.
T6 Same at T1 but with more aggressive periodic block scanner by setting
dfs.datanode.scan.period.hours to 1.
Expected Behavior: Upgrade should succeed. Look at the Datanode log file
to verify that no bad blocks were found by the periodic block scanner.
T7 Same as T1. Before starting the upgrade, log into a Datanode and delete a
particular blk_xxx.meta file. Then start the DFS upgrade.
Expected Behavior: Look at the Namenode log to detect that the upgrade
failed on that Datanode.
> Support for upgrading existing cluster to facilitate appends to HDFS files
> --------------------------------------------------------------------------
>
> Key: HADOOP-2656
> URL: https://issues.apache.org/jira/browse/HADOOP-2656
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.18.0
>
> Attachments: upgradeGenStamp.patch, upgradeGenStamp4.patch,
> upgradeGenStamp5.patch, upgradeGenStamp6.patch, upgradeGenStamp7.patch,
> upgradeGenStamp8.patch, upgradeGenStamp9.patch, upgradeGenStamp9.patch
>
>
> HADOOP-1700 describes the design for supporting appends to HDFS files. This
> design requires a distributed-upgrade to existing cluster installations. The
> design specifies that the DataNode persist the 8-byte BlockGenerationStamp in
> the block metadata file. The upgrade code will introduce this new field in
> the block metadata file and initialize this value to 0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
