Whew, glad I asked. It might be useful for someone to update the wiki: http://wiki.apache.org/hadoop/FAQ#How_do_I_set_up_a_hadoop_node_to_use_multiple_volumes.3F
-Mike On Thu, Feb 10, 2011 at 12:43 PM, Harsh J <[email protected]> wrote: > DO NOT format your NameNode. Formatting a NameNode is equivalent to > formatting a FS -- you're bound lose it all. > > And while messing with NameNode, after bringing it down safely, ALWAYS > take a backup of the existing dfs.name.dir contents and preferably the > SNN checkpoint directory contents too (if you're running it). > > The RIGHT way to add new directories to the NameNode's dfs.name.dir is > by comma-separating them in the same value and NOT by adding two > properties - that is not how Hadoop's configuration operates. In your > case, bring NN down and edit conf as: > > > <property> > > <name>dfs.name.dir</name> > > <value>/mnt/hadoop/name,/public/hadoop/name</value> > > </property> > > Create the new directory by copying the existing one. Both must have > the SAME file and structure in them, like mirror copies of one > another. Ensure that this new location, apart from being symmetric in > content, is also symmetric in permissions. NameNode will require WRITE > permissions via its user on all locations configured. > > Having configured properly and ensured that both storage directories > mirror one another, launch your NameNode back up again (feel a little > paranoid and do check namenode logs for any issues -- in which case > your backup would be very essential as a requirement for recovery!). > > P.s. Hold on for a bit for a possible comment from another user before > getting into action. I've added extra directories this way, but I do > not know if this is "the" genuine way to do so - although it feels > right to me. > > On Thu, Feb 10, 2011 at 10:27 PM, mike anderson <[email protected]> > wrote: > > This should be a straightforward question, but better safe than sorry. > > > > I wanted to add a second name node directory (on an NFS as a backup), so > now > > my hdfs-site.xml contains: > > > > <property> > > <name>dfs.name.dir</name> > > <value>/mnt/hadoop/name</value> > > </property> > > <property> > > <name>dfs.name.dir</name> > > <value>/public/hadoop/name</value> > > </property> > > > > > > When I go to start DFS i'm getting the exception: > > > > org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: > Directory > > /public/hadoop/name is in an inconsistent state: storage directory does > not > > exist or is not accessible. > > > > > > After googling a bit, it seems like I want to do "bin/hadoop namenode > > -format" > > > > Is this right? As long as I shut down DFS before issuing the command I > > shouldn't lose any data? > > > > Thanks in advance, > > Mike > > > > > > -- > Harsh J > www.harshj.com >
