[ http://issues.apache.org/jira/browse/HADOOP-745?page=comments#action_12458028 ] dhruba borthakur commented on HADOOP-745: -----------------------------------------
Adding comments from an email thread: > On 12/8/06, Espen Amble Kolstad <[EMAIL PROTECTED]> wrote: > > Hi, > > > > I run hadoop-0.9-dev and my edits-file has become corrupt. When I > > try to start the namenode I get the following error: > > 2006-12-08 20:38:57,431 ERROR dfs.NameNode - > > java.io.FileNotFoundException: Parent path does not exist: > > /user/trank/dotno/segments/20061208154235/parse_data/part-00000 > > at > > org.apache.hadoop.dfs.FSDirectory$INode.addNode(FSDirectory.java:186) > > at > > org.apache.hadoop.dfs.FSDirectory.unprotectedMkdir(FSDirectory.java:714) > > at org.apache.hadoop.dfs.FSEditLog.loadFSEdits(FSEditLog.java:254) > > at org.apache.hadoop.dfs.FSImage.loadFSImage(FSImage.java:191) > > at > > org.apache.hadoop.dfs.FSDirectory.loadFSImage(FSDirectory.java:320) > > at > org.apache.hadoop.dfs.FSNamesystem.<init>(FSNamesystem.java:226) > > at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:142) > > at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:134) > > at org.apache.hadoop.dfs.NameNode.main(NameNode.java:585) > > > > I've grep'ed through my edits-file, to see what's wrong. It seems > > the edits-file is missing an OP_MKDIR for > > /user/trank/dotno/segments/20061208154235/parse_data. > > > > Is there a tool for fixing an edits-file, or to put in an OP_MKDIR ? Email conversation with Albert Chen ----------------------------------------------- Hi Dhruba, This happened some time ago so my memory's sketchy, but I'll do my best to answer the questions: > 1. Did you have more than one directory in dfs.name.dir? > 2. Was this a new cluster or was it an existing cluster and was > upgraded to 0.9.0 recently? It happened after we restarted the DFS when upgrading from 0.7 to 0.8. In the process, we also added multiple directories to dfs.name.dir. > 3. Did any unnatural Namenode restarts occur immediately before the > problem started occurring? Not sure about this one. > 1. Will it help to make the fsimage/edit file ascii, so that it can be > easily edited by hand? That's a good idea, but I don't know if the effect on backwards compatibility is worth it. Editing these files is probably not something that most people will do. Maybe some sort of conversion tool that goes from the binary to text and vice versa would be more useful. > 2. Does it make sense for HDFS to automatically create a directory > equivalent to /lost+found? While EditLog processing, if the parent > directory of a file does not exist, the file can go into /lost+found? Yes. At least this way people can start up their DFSs after corruption. > NameNode throws FileNotFoundException: Parent path does not exist on startup > ---------------------------------------------------------------------------- > > Key: HADOOP-745 > URL: http://issues.apache.org/jira/browse/HADOOP-745 > Project: Hadoop > Issue Type: Bug > Components: dfs > Affects Versions: 0.8.0 > Reporter: Nigel Daley > > (with credit to Christian) > The following exception is reported by the namenode upon startup: > 2006-11-22 12:18:17,713 ERROR org.apache.hadoop.dfs.NameNode: > java.io.FileNotFoundException: Parent path does not exist: /foo/bar > at > org.apache.hadoop.dfs.FSDirectory$INode.addNode(FSDirectory.java:186) > at > org.apache.hadoop.dfs.FSDirectory.unprotectedMkdir(FSDirectory.java:731) > at org.apache.hadoop.dfs.FSEditLog.loadFSEdits(FSEditLog.java:254) > at org.apache.hadoop.dfs.FSImage.loadFSImage(FSImage.java:191) > at org.apache.hadoop.dfs.FSDirectory.loadFSImage(FSDirectory.java:321) > at org.apache.hadoop.dfs.FSNamesystem.<init>(FSNamesystem.java:230) > at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:145) > at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:137) > at org.apache.hadoop.dfs.NameNode.main(NameNode.java:585) > Perhaps the 'mkdirs' section > INode inserted = unprotectedMkdir(cur); > if (inserted != null) { > NameNode.stateChangeLog.debug("DIR* FSDirectory.mkdirs: " > +"created directory "+cur ); > fsImage.getEditLog().logMkDir( inserted ); > needs to be synchronized or the unprotectedMkdir method needs to be changed > to allow creating missing parent directories. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
