Hi Martin,
"fsimage" is the file that contains all name space information.
Name-node stores this information in memory and periodically checkpoints
it to disk.
"edits" contains a log of name space operations that occurred since the
last checkpoint.
So that if the name-node fails or otherwise exits between two
checkpoints the
name-space could be restored by merging the latest image and the edits.
Property "dfs.name.dir" specifies the directory where fsimage, edits and
other name-node
files are stored.
You can specify multiple entries (comma delimited) in "dfs.name.dir".
Then all name-space
files will be duplicated in these directories. This is the right way to
backup the name-space
image as a local copy.
Periodic checkpointing is performed by the secondary name-node. The side
effect of the
checkpoint is that it removes contents of the edits file. Yes you are
right the edits file can
grow big that is why periodic checkpointing is important.
Does that answer your questions?
--Konstantin
Doug Cutting wrote:
KrzyCube wrote:
I found that "File[] editFiles" in FSEditLog.java , then i trace the
call stack and found that it can be configured as multi-case of
"dfs.name.dir" . Is this means the NameNode data can be split into
pieces or
just set replication as the number of the strings of dirs that
configured ?
Is that right the way to backup several copys of the
editlog+fsimage in
local filesystem ?
According to the documentation for dfs.name.dir:
http://lucene.apache.org/hadoop/hadoop-default.html#dfs.name.dir
Determines where on the local filesystem the DFS name node should
store the name table. If this is a comma-delimited list of directories
then the name table is replicated in all of the directories, for
redundancy.
Note that as of release 0.11 there's support for a secondary namenode,
started with 'bin/hadoop secondarynamenode'.
http://issues.apache.org/jira/browse/HADOOP-227#action_12458332
This documentation should probably be added to
http://lucene.apache.org/hadoop/hdfs_design.html, or at least to the
wiki...
Doug
