Hi* *Mohit Your fsOwner hdfs should have the permission to access to the /mnt/san1/hdfs/cache/hdfs/dfs/name. So please check the permission of /mnt/ , and sub-directories on the OS. they all need to be read ,writen.
regards 2013/2/28 Mohit Vadhera <[email protected]> > Please find below logs for shutting down the namenode service. Can anybody > check this > > 2013-02-28 02:07:51,752 WARN org.apache.hadoop.hdfs.server.common.Util: > Path /mnt/san1/hdfs/cache/hdfs/dfs/name should be specified as a URI in > configuration files. Please update hdfs configuration. > 2013-02-28 02:07:51,754 WARN org.apache.hadoop.hdfs.server.common.Util: > Path /mnt/san1/hdfs/cache/hdfs/dfs/name should be specified as a URI in > configuration files. Please update hdfs configuration. > 2013-02-28 02:07:51,754 WARN > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Only one image storage > directory (dfs.namenode.name.dir) configured. Beware of dataloss due to > lack of redundant storage directories! > 2013-02-28 02:07:51,754 WARN > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Only one namespace > edits storage directory (dfs.namenode.edits.dir) configured. Beware of > dataloss due to lack of redundant storage directories! > 2013-02-28 02:07:51,884 INFO org.apache.hadoop.util.HostsFileReader: > Refreshing hosts (include/exclude) list > 2013-02-28 02:07:51,890 INFO > org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: > dfs.block.invalidate.limit=1000 > 2013-02-28 02:07:51,909 INFO > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: > dfs.block.access.token.enable=false > 2013-02-28 02:07:51,910 INFO > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: > defaultReplication = 1 > 2013-02-28 02:07:51,910 INFO > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxReplication > = 512 > 2013-02-28 02:07:51,910 INFO > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: minReplication > = 1 > 2013-02-28 02:07:51,910 INFO > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: > maxReplicationStreams = 2 > 2013-02-28 02:07:51,910 INFO > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: > shouldCheckForEnoughRacks = false > 2013-02-28 02:07:51,910 INFO > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: > replicationRecheckInterval = 3000 > 2013-02-28 02:07:51,910 INFO > org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: > encryptDataTransfer = false > 2013-02-28 02:07:51,920 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner = > hdfs (auth:SIMPLE) > 2013-02-28 02:07:51,920 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup = > hadmin > 2013-02-28 02:07:51,920 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled = > true > 2013-02-28 02:07:51,920 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: HA Enabled: false > 2013-02-28 02:07:51,925 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Append Enabled: true > 2013-02-28 02:07:52,462 INFO > org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file names > occuring more than 10 times > 2013-02-28 02:07:52,466 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: > dfs.namenode.safemode.threshold-pct = 0.9990000128746033 > 2013-02-28 02:07:52,467 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: > dfs.namenode.safemode.min.datanodes = 0 > 2013-02-28 02:07:52,467 INFO > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: > dfs.namenode.safemode.extension = 0 > 2013-02-28 02:07:52,469 INFO org.apache.hadoop.hdfs.server.common.Storage: > Storage directory /mnt/san1/hdfs/cache/hdfs/dfs/name does not exist. > 2013-02-28 02:07:52,471 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping NameNode > metrics system... > 2013-02-28 02:07:52,472 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system > stopped. > 2013-02-28 02:07:52,473 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system > shutdown complete. > 2013-02-28 02:07:52,473 FATAL > org.apache.hadoop.hdfs.server.namenode.NameNode: Exception in namenode join > org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: > Directory /mnt/san1/hdfs/cache/hdfs/dfs/name is in an inconsistent state: > storage directory does not exist or is not accessible. > at > org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:295) > at > org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:201) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:534) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:424) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:386) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:398) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:432) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:608) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:589) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1140) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1204) > 2013-02-28 02:08:48,908 INFO org.apache.hadoop.util.ExitUtil: Exiting with > status 1 > 2013-02-28 02:08:48,913 INFO > org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG: > /************************************************************ > SHUTDOWN_MSG: Shutting down NameNode at OPERA-MAST1.ny.os.local/ > 192.168.1.3 > > > On Thu, Feb 28, 2013 at 1:28 PM, Mohit Vadhera < > [email protected]> wrote: > >> Hi Guys, >> >> Namenode switches into safemode when it has low disk space on the root fs >> / i have to manually run a command to leave it >> I have space on other partition. Can I change the path for cache files on >> other partition ? I have below properties . Can it resolve the issue ? If i >> change the path to other directories and restart services I get the below >> error while starting the service namenode. I didn't find anything in logs >> so far. Can you please suggest something ? >> >> <property> >> <name>hadoop.tmp.dir</name> >> <value>/var/lib/hadoop-hdfs/cache/${user.name}</value> >> </property> >> <property> >> <name>dfs.namenode.name.dir</name> >> <value>/var/lib/hadoop-hdfs/cache/${user.name}/dfs/name</value> >> </property> >> <property> >> <name>dfs.namenode.checkpoint.dir</name> >> <value>/var/lib/hadoop-hdfs/cache/${user.name >> }/dfs/namesecondary</value> >> </property> >> <property> >> >> >> Service namenode is failing >> >> # for service in /etc/init.d/hadoop-hdfs-* ; do sudo $service status; done >> Hadoop datanode is running [ OK ] >> Hadoop namenode is dead and pid file exists [FAILED] >> Hadoop secondarynamenode is running [ OK ] >> >> Thanks, >> >> >
