[
https://issues.apache.org/jira/browse/HDFS-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13012534#comment-13012534
]
Todd Lipcon commented on HDFS-1538:
-----------------------------------
bq. Why not do latestNameSD.read at the end of doRecovery() and get rid of
getStorageDirectoryForProperties() altogether?
The "VERSION" files are one thing that will need to be discussed/attacked later
in this branch. There are currently some things in there that are
image-specific (eg checkpoint time) that don't really make a ton of sense when
multiple images may be present in a directory, and the image/edit files
themselves encode their txids. I will be opening a JIRA to discuss this soon.
The particular reason for this call being distinct from the latestNameSD is
that we might roll edit logs several times without actually saving a
checkpoint. This may cause the VERSION file to get updated in one of the edits
directories rather than one of the image directories.
I'm still not 100% sure that this makes sense, though. Can I leave this as is,
but add a "TODO" note to come back to this before merge? We will sweep for
TODO/FIXME/etc before the project is deemed merge-able.
bq. The previous command takes care of editsVersion!=LAYOUT_VERSION, and
imageVersion!=LAYOUT_VERSION shouldn't cause issues anyhow, as we always have
to be able to load from old layout versions
Yes, I think that's my opinion as well - ie that we should feel safe to leave
an old version in place without a re-save. This is similar in logic to the
recent comments on HDFS-1780. I will change this line to a TODO for further
discussion as well.
bq. I think it would be clearer for loadFSImage to return the last txid it
loaded and then pass that as a parameter to loadEdits(List<File> editLogs, long
fromTxid).
Who, then, should set storage.lastCheckpointTxId? To me, this makes sense -
storage.lastCheckpointTxId is the txid
of the last checkpoint (ie fsimage_X) that was either loaded or saved. Hence,
the loading of edit logs should always start at lastCheckpointTxId + 1.
bq. Is ImageLoadPlan the best name for this given that it loads Edits also. Why
not just LoadPlan?
Will fix.
bq. You should create the logger with FSImageOldStorageInspector so that it's
clear which inspector is logging
Will fix.
> Refactor more startup and image loading code out of FSImage
> -----------------------------------------------------------
>
> Key: HDFS-1538
> URL: https://issues.apache.org/jira/browse/HDFS-1538
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: Edit log branch (HDFS-1073)
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Fix For: Edit log branch (HDFS-1073)
>
> Attachments: hdfs-1538-1.txt, hdfs-1538-2.txt, hdfs-1538-on-1521.txt,
> hdfs-1538-on-branch.txt, hdfs-1538.txt
>
>
> For HDFS-1073, we need to be able to continue to load images in the old
> "fsimage/edits/edits.new" layout for the purposes of upgrade. But that code
> will be only for backwards compatibility, and we want to be able to switch to
> new code for the new layout. This subtask is to separate out much of that
> code into an interface which we can implement for both the old and new
> layouts.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
