[
https://issues.apache.org/jira/browse/HDFS-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Kelly updated HDFS-1725:
-----------------------------
Attachment: HDFS-1725.diff
Added new patch which addresses Todds comments. Also caught up with HDFS-1073
run-commit-test passes with this patch the same as they do with HDFS-1073. The
following tests have failures.
org.apache.hadoop.cli.TestHDFSCLI
org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
org.apache.hadoop.hdfs.TestHDFSServerPorts
org.apache.hadoop.server.namenode.TestBackupNode
> Set storage directories only at FSImage construction (was Cleanup FSImage
> construction)
> ---------------------------------------------------------------------------------------
>
> Key: HDFS-1725
> URL: https://issues.apache.org/jira/browse/HDFS-1725
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: Edit log branch (HDFS-1073)
>
> Attachments: HDFS-1725-review-guide.pdf, HDFS-1725.diff,
> HDFS-1725.diff, HDFS-1725.diff, HDFS-1725.diff, HDFS-1725.diff,
> HDFS-1725.diff, HDFS-1725.diff, HDFS-1725.diff, HDFS-1725.patch
>
>
> HDFS-1580 proposes extending FSEditLog to allow it to use editlog streams
> which are not backed by StorageDirectory. Currently, to set the the
> directories used for edits, NNStorage#setStorageDirectory is called with a
> list of URIs as the second argument. NNStorage takes this list or URIs, takes
> all file:/// URIs and adds them to its StorageDirectory list. Then, when
> opened, FSEditLog will request a list of StorageDirectories from NNStorage
> and create a list of EditLogOutputStreams based on these.
> This approach cannot work with HDFS-1580. NNStorage exists solely to deal
> with filesystem based storage. As such, only StorageDirectories can be
> retrieved from NNStorage by FSEditLog. So, FSEditLog should get the URI from
> some place other than NNStorage. This presents a further problem, in that,
> NNStorage#setStorageDirectories is the current way of setting the URIs for
> images and edits. This call can happen at any time, so the directories in
> NNStorage can change at any time. If FSEditLog is to get its URIs from
> elsewhere, this opens up the risk of the filesystem directories in NNStorage
> and filesystem URIs being out of sync.
> A solution to this is to stipulate that the URIs for NNStorage are set only
> once, on construction. All proper uses of NNStorage#setStorageDirectories are
> being called just after construction of the image in any case. All other
> cases are using NNStorage#setStorageDirectories not to set the storage
> directories, but for the side effects of this call. This guide explains these
> other cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
