Thanks for you helps, but still could not solve my problem.
On Tue, Mar 18, 2014 at 10:13 AM, Stanley Shi <s...@gopivotal.com> wrote: > Ah yes, I overlooked this. Then please check the file are there or not: > "ls /home/hadoop/project/hadoop-data/dfs/name"? > > Regards, > *Stanley Shi,* > > > > On Tue, Mar 18, 2014 at 2:06 PM, Azuryy Yu <azury...@gmail.com> wrote: > >> I don't think this is the case, because there is; >> <property> >> <name>hadoop.tmp.dir</name> >> <value>/home/hadoop/project/hadoop-data</value> >> </property> >> >> >> On Tue, Mar 18, 2014 at 1:55 PM, Stanley Shi <s...@gopivotal.com> wrote: >> >>> one possible reason is that you didn't set the namenode working >>> directory, by default it's in "/tmp" folder; and the "/tmp" folder might >>> get deleted by the OS without any notification. If this is the case, I am >>> afraid you have lost all your namenode data. >>> >>> *<property> >>> <name>dfs.name.dir</name> >>> <value>${hadoop.tmp.dir}/dfs/name</value> >>> <description>Determines where on the local filesystem the DFS name node >>> should store the name table(fsimage). If this is a comma-delimited >>> list >>> of directories then the name table is replicated in all of the >>> directories, for redundancy. </description> >>> </property>* >>> >>> >>> Regards, >>> *Stanley Shi,* >>> >>> >>> >>> On Sun, Mar 16, 2014 at 5:29 PM, Mirko Kämpf <mirko.kae...@gmail.com>wrote: >>> >>>> Hi, >>>> >>>> what is the location of the namenodes fsimage and editlogs? >>>> And how much memory has the NameNode. >>>> >>>> Did you work with a Secondary NameNode or a Standby NameNode for >>>> checkpointing? >>>> >>>> Where are your HDFS blocks located, are those still safe? >>>> >>>> With this information at hand, one might be able to fix your setup, but >>>> do not format the old namenode before >>>> all is working with a fresh one. >>>> >>>> Grab a copy of the maintainance guide: >>>> http://shop.oreilly.com/product/0636920025085.do?sortby=publicationDate >>>> which helps solving such type of problems as well. >>>> >>>> Best wishes >>>> Mirko >>>> >>>> >>>> 2014-03-16 9:07 GMT+00:00 Fatih Haltas <fatih.hal...@nyu.edu>: >>>> >>>> Dear All, >>>>> >>>>> I have just restarted machines of my hadoop clusters. Now, I am trying >>>>> to restart hadoop clusters again, but getting error on namenode restart. I >>>>> am afraid of loosing my data as it was properly running for more than 3 >>>>> months. Currently, I believe if I do namenode formatting, it will work >>>>> again, however, data will be lost. Is there anyway to solve this without >>>>> losing the data. >>>>> >>>>> I will really appreciate any help. >>>>> >>>>> Thanks. >>>>> >>>>> >>>>> ===================== >>>>> Here is the logs; >>>>> ==================== >>>>> 2014-02-26 16:02:39,698 INFO >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG: >>>>> /************************************************************ >>>>> STARTUP_MSG: Starting NameNode >>>>> STARTUP_MSG: host = ADUAE042-LAP-V/127.0.0.1 >>>>> STARTUP_MSG: args = [] >>>>> STARTUP_MSG: version = 1.0.4 >>>>> STARTUP_MSG: build = >>>>> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r >>>>> 1393290; compiled by 'hortonfo' on Wed Oct 3 05:13:58 UTC 2012 >>>>> ************************************************************/ >>>>> 2014-02-26 16:02:40,005 INFO >>>>> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from >>>>> hadoop-metrics2.properties >>>>> 2014-02-26 16:02:40,019 INFO >>>>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source >>>>> MetricsSystem,sub=Stats registered. >>>>> 2014-02-26 16:02:40,021 INFO >>>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot >>>>> period at 10 second(s). >>>>> 2014-02-26 16:02:40,021 INFO >>>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system >>>>> started >>>>> 2014-02-26 16:02:40,169 INFO >>>>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi >>>>> registered. >>>>> 2014-02-26 16:02:40,193 INFO >>>>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source jvm >>>>> registered. >>>>> 2014-02-26 16:02:40,194 INFO >>>>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source >>>>> NameNode registered. >>>>> 2014-02-26 16:02:40,242 INFO org.apache.hadoop.hdfs.util.GSet: VM type >>>>> = 64-bit >>>>> 2014-02-26 16:02:40,242 INFO org.apache.hadoop.hdfs.util.GSet: 2% max >>>>> memory = 17.77875 MB >>>>> 2014-02-26 16:02:40,242 INFO org.apache.hadoop.hdfs.util.GSet: >>>>> capacity = 2^21 = 2097152 entries >>>>> 2014-02-26 16:02:40,242 INFO org.apache.hadoop.hdfs.util.GSet: >>>>> recommended=2097152, actual=2097152 >>>>> 2014-02-26 16:02:40,273 INFO >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner=hadoop >>>>> 2014-02-26 16:02:40,273 INFO >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup >>>>> 2014-02-26 16:02:40,274 INFO >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: >>>>> isPermissionEnabled=true >>>>> 2014-02-26 16:02:40,279 INFO >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: >>>>> dfs.block.invalidate.limit=100 >>>>> 2014-02-26 16:02:40,279 INFO >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: >>>>> isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), >>>>> accessTokenLifetime=0 min(s) >>>>> 2014-02-26 16:02:40,724 INFO >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered >>>>> FSNamesystemStateMBean and NameNodeMXBean >>>>> 2014-02-26 16:02:40,749 INFO >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file names >>>>> occuring more than 10 times >>>>> 2014-02-26 16:02:40,780 ERROR >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem >>>>> initialization failed. >>>>> java.io.IOException: NameNode is not formatted. >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:330) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288) >>>>> 2014-02-26 16:02:40,781 ERROR >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException: >>>>> NameNode is not formatted. >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:330) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279) >>>>> at >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288) >>>>> >>>>> 2014-02-26 16:02:40,781 INFO >>>>> org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG: >>>>> /************************************************************ >>>>> SHUTDOWN_MSG: Shutting down NameNode at ADUAE042-LAP-V/127.0.0.1 >>>>> ************************************************************/ >>>>> >>>>> =========================== >>>>> Here is the core-site.xml >>>>> =========================== >>>>> <?xml version="1.0"?> >>>>> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> >>>>> >>>>> <!-- Put site-specific property overrides in this file. --> >>>>> >>>>> <configuration> >>>>> <property> >>>>> <name>fs.default.name</name> >>>>> <value>-BLANKED</value> >>>>> </property> >>>>> <property> >>>>> <name>hadoop.tmp.dir</name> >>>>> <value>/home/hadoop/project/hadoop-data</value> >>>>> </property> >>>>> </configuration> >>>>> >>>>> >>>>> >>>>> >>>> >>> >> >