Running Apache 1.0.2 Primary NN on Ubuntu 11.10, 8 datanodes. When I came in today I ran a stop-all.sh since one of the datanodes was not showing on the dfsnodelist.jsp?whatNodes=LIVE status page. I then ran start-all.sh, and the NN dies on a Null Pointer Exception. I looked at the source but cannot determine what is going. Below is the trace - any explanation of cause or suggestions to get the system back up would be appreciated.
** I have removed the prefix "org.apache.hadoop." from the list below in the interest of saving space. ** -Terry 2012-05-04 09:45:02,046 INFO hdfs.server.namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = xxxx/xxxxxxxx STARTUP_MSG: args = [] STARTUP_MSG: version = 1.0.2 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0.2 -r 1304954; compiled by 'hortonfo' on Sat Mar 24 23:58:21 UTC 2012 ************************************************************/ 2012-05-04 09:45:02,229 INFO metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2012-05-04 09:45:02,244 INFO metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered. 2012-05-04 09:45:02,246 INFO metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2012-05-04 09:45:02,246 INFO metrics2.impl.MetricsSystemImpl: NameNode metrics system started 2012-05-04 09:45:02,483 INFO metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered. 2012-05-04 09:45:02,488 WARN metrics2.impl.MetricsSystemImpl: Source name ugi already exists! 2012-05-04 09:45:02,494 INFO metrics2.impl.MetricsSourceAdapter: MBean for source jvm registered. 2012-05-04 09:45:02,496 INFO metrics2.impl.MetricsSourceAdapter: MBean for source NameNode registered. 2012-05-04 09:45:02,530 INFO hdfs.util.GSet: VM type = 64-bit 2012-05-04 09:45:02,530 INFO hdfs.util.GSet: 2% max memory = 17.77875 MB 2012-05-04 09:45:02,530 INFO hdfs.util.GSet: capacity = 2^21 = 2097152 entries 2012-05-04 09:45:02,530 INFO hdfs.util.GSet: recommended=2097152, actual=2097152 2012-05-04 09:45:02,560 INFO hdfs.server.namenode.FSNamesystem: fsOwner=thealy 2012-05-04 09:45:02,560 INFO hdfs.server.namenode.FSNamesystem: supergroup=supergroup 2012-05-04 09:45:02,560 INFO hdfs.server.namenode.FSNamesystem: isPermissionEnabled=true 2012-05-04 09:45:02,567 INFO hdfs.server.namenode.FSNamesystem: dfs.block.invalidate.limit=100 2012-05-04 09:45:02,567 INFO hdfs.server.namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s) 2012-05-04 09:45:02,749 INFO hdfs.server.namenode.FSNamesystem: Registered FSNamesystemStateMBean and NameNodeMXBean 2012-05-04 09:45:02,765 INFO hdfs.server.namenode.NameNode: Caching file names occuring more than 10 times 2012-05-04 09:45:02,775 INFO hdfs.server.common.Storage: Number of files = 11 2012-05-04 09:45:02,783 INFO hdfs.server.common.Storage: Number of files under construction = 0 2012-05-04 09:45:02,783 INFO hdfs.server.common.Storage: Image file of size 11087 loaded in 0 seconds. 2012-05-04 09:45:02,785 ERROR hdfs.server.namenode.NameNode: java.lang.NullPointerException at hdfs.server.namenode.FSDirectory.addChild(FSDirectory.java:1094) at hdfs.server.namenode.FSDirectory.addChild(FSDirectory.java:1106) at hdfs.server.namenode.FSDirectory.addNode(FSDirectory.java:1009) at hdfs.server.namenode.FSDirectory.unprotectedAddFile(FSDirectory.java:208) at hdfs.server.namenode.FSEditLog.loadFSEdits(FSEditLog.java:626) at hdfs.server.namenode.FSImage.loadFSEdits(FSImage.java:1015) at hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:833) at hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:372) at hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100) at hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388) at hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362) at hdfs.server.namenode.NameNode.initialize(NameNode.java:276) at hdfs.server.namenode.NameNode.<init>(NameNode.java:496) at hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279) at hdfs.server.namenode.NameNode.main(NameNode.java:1288)
