Re: Secondary namenode fsimage concept

[shanmuganathan.r]

Hi Kai,

      There is no datas stored  in the secondarynamenode related to the Hadoop 
cluster . Am I correct?
If it correct means If we run the secondaryname node in separate machine then 
fetching , merging and transferring time is increased if the cluster has large 
data in the namenode fsimage file . At the time if fail over occurs , then how 
can we recover the nearly one hour changes in the HDFS file ? (default check 
point time is one hour)

Thanks R.Shanmuganathan  






---- On Thu, 06 Oct 2011 12:20:28 +0530 Kai Voigt<k...@123.org> wrote 
---- 


Hi, 
 
the secondary namenode only fetches the two files when a checkpointing is 
needed. 
 
Kai 
 
Am 06.10.2011 um 08:45 schrieb shanmuganathan.r: 
 
> Hi Kai, 
> 
> In the Second part I meant 
> 
> 
> Is the secondary namenode also contain the FSImage file or the two 
files(FSImage and EdiltLog) are transferred from the namenode at the checkpoint 
time. 
> 
> 
> Thanks 
> Shanmuganathan 
> 
> 
> 
> 
> 
> ---- On Thu, 06 Oct 2011 11:37:50 +0530 Kai 
Voigt<k...@123.org> wrote ---- 
> 
> 
> Hi, 
> 
> you're correct when saying the namenode hosts the fsimage file and the 
edits log file. 
> 
> The fsimage file contains a snapshot of the HDFS metadata (a filename to 
blocks list mapping). Whenever there is a change to HDFS, it will be appended 
to the edits file. Think of it as a database transaction log, where changes 
will not be applied to the datafile, but appended to a log. 
> 
> To prevent the edits file growing infinitely, the secondary namenode 
periodically pulls these two files, and the namenode starts writing changes to 
a new edits file. Then, the secondary namenode merges the changes from the 
edits file with the old snapshot from the fsimage file and creates an updated 
fsimage file. This updated fsimage file is then copied to the namenode. 
> 
> Then, the entire cycle starts again. To answer your question: The namenode 
has both files, even if the secondary namenode is running on a different 
machine. 
> 
> Kai 
> 
> Am 06.10.2011 um 07:57 schrieb shanmuganathan.r: 
> 
> > 
> > Hi All, 
> > 
> > I have a doubt in hadoop secondary namenode concept . Please 
correct if the following statements are wrong . 
> > 
> > 
> > The namenode hosts the fsimage and edit log files. The secondary 
namenode hosts the fsimage file only. At the time of checkpoint the edit log 
file is transferred to the secondary namenode and the both files are merged, 
Then the updated fsimage file is transferred to the namenode . Is it correct? 
> > 
> > 
> > If we run the secondary namenode in separate machine , then both 
machines contain the fsimage file . Namenode only contains the editlog file. Is 
it true? 
> > 
> > 
> > 
> > Thanks R.Shanmuganathan 
> > 
> > 
> > 
> > 
> > 
> > 
> 
> -- 
> Kai Voigt 
> k...@123.org 
> 
> 
> 
> 
> 
> 
> 
 
-- 
Kai Voigt 
k...@123.org