Sorry to hijack but after following this thread, I had a related question to the secondary location of dfs.name.dir.
Is the approach outlined below the preferred/suggested way to do this? Is this people mean when they say, "stick it on NFS" ? Thanks! On May 17, 2010, at 11:14 PM, Todd Lipcon wrote: > On Mon, May 17, 2010 at 5:10 PM, jiang licht <[email protected]> wrote: > >> I am considering to use a machine to save a >> redundant copy of HDFS metadata through setting dfs.name.dir in >> hdfs-site.xml like this (as in YDN): >> >> <property> >> <name>dfs.name.dir</name> >> <value>/home/hadoop/dfs/name,/mnt/namenode-backup</value> >> <final>true</final> >> </property> >> >> where the two folders are on different machines so that >> /mnt/namenode-backup keeps a copy of hdfs file system information and its >> machine can be used to replace the first machine that fails as namenode. >> >> So, my question is how big this hdfs metatdata will consume? I guess it is >> proportional to the hdfs capacity. What ratio is that or what size will be >> for 150TB hdfs? >> > > On the order of a few GB, max (you really need double the size of your > image, so it has tmp space when downloading a checkpoint or performing an > upgrade). But on any disk you can buy these days you'll have plenty of > space. > > -Todd > > > -- > Todd Lipcon > Software Engineer, Cloudera
