[
https://issues.apache.org/jira/browse/HDFS-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated HDFS-2001:
------------------------------
Attachment: hdfs-2001.txt
This patch goes on top of HDFS-1991 and HDFS-1994.
It removes the previouscheckpoint/lastcheckpoint directories and updates
TestDFSStorageStateRecovery to match that.
I currently commented out a couple of the test cases in TestSaveNamespace which
need to still be updated and improved on this branch.
I also added a new test case for the scenario when the 2NN fails two
checkpoints in a row. Previous to this patch, there was a bug with that
situation, but this patch addresses it.
With this patch, I'm able to run a stress test with two 2NNs checkpointing as
fast as they can for a while without issues. I'll let it loop for a few hours
tonight.
> HDFS-1073: Kill previous.checkpoint, lastcheckpoint.tmp directories
> -------------------------------------------------------------------
>
> Key: HDFS-2001
> URL: https://issues.apache.org/jira/browse/HDFS-2001
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: Edit log branch (HDFS-1073)
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Fix For: Edit log branch (HDFS-1073)
>
> Attachments: hdfs-2001.txt
>
>
> In the old design for edits/image storage, the secondary namenode does a
> complicated dance of moving "current/" to "lastcheckpoint.tmp", checkpointing
> into "current/", then moving "lastcheckpoint.tmp" back to
> "previous.checkpoint". The idea here was so that there would always be one
> directory with a valid set of storage files.
> In the HDFS-1073 design, this complicated dance isn't necessary. If a
> checkpoint fails, we can just rm that single fsimage_N.ckpt file and still be
> left with a valid storage directory.
> So, we can just let the 2NN keep a single current/ dir around for all
> checkpoints and eliminate the complexity of the dance.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
