Hi,
I need to install hadoop on 16-node cluster. I have a couple of related
questions:
1. I have installed hadoop on a shared directory, i.e., there is just one
place where the whole hadoop installation files exist and all the 16 nodes
use the same installation.
Is that an issue or I need to install hadoop on each of these nodes in their
local directory separately?
2. I installed hadoop-0.21 and after following the installation
instructions, when i tried formatting, I get the following error:
************************************************************/
Re-format filesystem in /var/tmp/data/dfs/name ? (Y or N) Y
11/04/13 09:16:23 INFO namenode.FSNamesystem: defaultReplication = 3
11/04/13 09:16:23 INFO namenode.FSNamesystem: maxReplication = 512
11/04/13 09:16:23 INFO namenode.FSNamesystem: minReplication = 1
11/04/13 09:16:23 INFO namenode.FSNamesystem: maxReplicationStreams = 2
11/04/13 09:16:23 INFO namenode.FSNamesystem: shouldCheckForEnoughRacks =
false
11/04/13 09:16:23 INFO security.Groups: Group mapping
impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping;
cacheTimeout=300000
11/04/13 09:16:23 INFO namenode.FSNamesystem: fsOwner=bus145
11/04/13 09:16:23 INFO namenode.FSNamesystem: supergroup=supergroup
11/04/13 09:16:23 INFO namenode.FSNamesystem: isPermissionEnabled=true
11/04/13 09:16:23 INFO namenode.FSNamesystem: isAccessTokenEnabled=false
accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
11/04/13 09:16:24 INFO common.Storage: Cannot lock storage
/var/tmp/data/dfs/name. The directory is already locked.
11/04/13 09:16:24 ERROR namenode.NameNode: java.io.IOException: Cannot lock
storage /var/tmp/data/dfs/name. The directory is already locked.
at
org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.lock(Storage.java:617)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:1426)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:1444)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1242)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1348)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
11/04/13 09:16:24 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at inti79.cse.psu.edu/130.203.58.207
************************************************************/
3. I was using before hadoop-0.20, and formatting was working fine.
4. Also, when i do a bin/start-dfs.sh, am able to see the Namenode, Datanode
up, however on bin/start-mapred.sh, am not able to see Jobtracker up on the
master node, though Tasktracker seems to be up on slaves.
Before upgrading to Hadoop-0.21, everything was working fine with
hadoop-0.20 including running benchmarks and getting stats.
Any suggestions in this regard is highly appreciated.
Thanks,
Bikash
