p.s. Also, while starting dfs using bin/start-dfs.sh, I get the following error:
2011-04-13 09:42:31,729 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = inti84.cse.psu.edu/130.203.58.212 STARTUP_MSG: args = [] STARTUP_MSG: version = 0.20.2 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010 ************************************************************/ 2011-04-13 09:42:31,853 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: java.lang.NullPointerException at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:134) at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:156) at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:160) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:175) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:279) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:956) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965) 2011-04-13 09:42:31,854 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at inti84.cse.psu.edu/130.203.58.212 ************************************************************/ 2011-04-13 09:44:03,265 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = inti84.cse.psu.edu/130.203.58.212 STARTUP_MSG: args = [] STARTUP_MSG: version = 0.20.2 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010 ************************************************************/ 2011-04-13 09:44:03,384 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: java.lang.NullPointerException at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:134) at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:156) at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:160) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:175) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:279) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:956) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965) On Wed, Apr 13, 2011 at 9:20 AM, bikash sharma <sharmabiks...@gmail.com>wrote: > Hi, > I need to install hadoop on 16-node cluster. I have a couple of related > questions: > 1. I have installed hadoop on a shared directory, i.e., there is just one > place where the whole hadoop installation files exist and all the 16 nodes > use the same installation. > Is that an issue or I need to install hadoop on each of these nodes in > their local directory separately? > 2. I installed hadoop-0.21 and after following the installation > instructions, when i tried formatting, I get the following error: > > ************************************************************/ > Re-format filesystem in /var/tmp/data/dfs/name ? (Y or N) Y > 11/04/13 09:16:23 INFO namenode.FSNamesystem: defaultReplication = 3 > 11/04/13 09:16:23 INFO namenode.FSNamesystem: maxReplication = 512 > 11/04/13 09:16:23 INFO namenode.FSNamesystem: minReplication = 1 > 11/04/13 09:16:23 INFO namenode.FSNamesystem: maxReplicationStreams = 2 > 11/04/13 09:16:23 INFO namenode.FSNamesystem: shouldCheckForEnoughRacks = > false > 11/04/13 09:16:23 INFO security.Groups: Group mapping > impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping; > cacheTimeout=300000 > 11/04/13 09:16:23 INFO namenode.FSNamesystem: fsOwner=bus145 > 11/04/13 09:16:23 INFO namenode.FSNamesystem: supergroup=supergroup > 11/04/13 09:16:23 INFO namenode.FSNamesystem: isPermissionEnabled=true > 11/04/13 09:16:23 INFO namenode.FSNamesystem: isAccessTokenEnabled=false > accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s) > 11/04/13 09:16:24 INFO common.Storage: Cannot lock storage > /var/tmp/data/dfs/name. The directory is already locked. > 11/04/13 09:16:24 ERROR namenode.NameNode: java.io.IOException: Cannot lock > storage /var/tmp/data/dfs/name. The directory is already locked. > at > org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.lock(Storage.java:617) > at > org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:1426) > at > org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:1444) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1242) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1348) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368) > > 11/04/13 09:16:24 INFO namenode.NameNode: SHUTDOWN_MSG: > /************************************************************ > SHUTDOWN_MSG: Shutting down NameNode at inti79.cse.psu.edu/130.203.58.207 > ************************************************************/ > > 3. I was using before hadoop-0.20, and formatting was working fine. > 4. Also, when i do a bin/start-dfs.sh, am able to see the Namenode, > Datanode up, however on bin/start-mapred.sh, am not able to see Jobtracker > up on the master node, though Tasktracker seems to be up on slaves. > > Before upgrading to Hadoop-0.21, everything was working fine with > hadoop-0.20 including running benchmarks and getting stats. > > Any suggestions in this regard is highly appreciated. > > Thanks, > Bikash >
