[
https://issues.apache.org/jira/browse/HDFS-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015224#comment-13015224
]
Ivan Kelly commented on HDFS-1725:
----------------------------------
Just to clarify the issue I'm trying to address with this JIRA while it's fresh
in my mind (just hit the same problem again on HDFS-1580). In HDFS-1580 I'm
trying to add the ability to have a editlog output stream which is not backed
by a StorageDirectory. Currently images and edits are both stored in a
StorageDirectory.
Currently when FSImage and FSEditLog are created, the image and edit URIs may
or may not be set. If they are not set on construction,
NNStorage.setStorageDirectories is called to set them. This isn't a problem
because FSEditLog only uses StorageDirectory backed output streams and as such
can get the list of locations to write to from storage, which is exactly what
it does. On FSEditLog.open() is iterates through all EDITS type directories in
the storage object, and creates a stream for it.
Obviously this will not work for anything but StorageDirectories and it's
precisely this problem which I was trying to address here. If
NNStorage.setStorageDirectories() is used, you have to go to NNStorage for the
location list when calling FSEditLog.open as the list could change at any time.
The solution I proposed was to mandate that you always set the
StorageDirectories on construction. This means that FSEditLog can get the list
on construction from FSImage and never has to query NNStorage. The only other
solution I can think of is to add a setStorageDirectories call to FSImage which
calls the same call on NNStorage and sets the list for FSEditLog. This would
work, but does seem as elegant as just setting them on construction.
> Cleanup FSImage construction
> ----------------------------
>
> Key: HDFS-1725
> URL: https://issues.apache.org/jira/browse/HDFS-1725
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 0.23.0
>
> Attachments: HDFS-1725.diff, HDFS-1725.diff, HDFS-1725.diff,
> HDFS-1725.diff, HDFS-1725.diff, HDFS-1725.diff
>
>
> FSImage construction is messy. Sometimes the storagedirectories in use are
> set straight away, sometimes they are not. This makes it hard for anything
> under FSImage (i.e. FSEditLog) to make assumptions about what it can use.
> Therefore, this patch makes FSImage set the storage directories in use during
> construction, and never allows them to change. If you want to change
> storagedirectories you create a new image.
> Also, all the construction code should be the same with the only difference
> being the parameters passed. When not passed, these should get sensible
> defaults.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
