Hi Df, Can you get : echo $HADOOP_HOME On Tue, Aug 16, 2011 at 3:49 PM, A Df <[email protected]> wrote:
> See inline > > > > >________________________________ > >From: Steve Loughran <[email protected]> > >To: [email protected] > >Sent: Tuesday, 16 August 2011, 11:08 > >Subject: Re: hadoop cluster mode not starting up > > > >On 16/08/11 11:02, A Df wrote: > >> Hello All: > >> > >> I used a combination of tutorials to setup hadoop but most seems to be > using either an old version of hadoop or only using 2 machines for the > cluster which isn't really a cluster. Does anyone know of a good tutorial > which setups multiple nodes for a cluster?? I already looked at the Apache > website but it does not give sample values for the conf files. Also each set > of tutorials seem to have a different set of parameters which they indicate > should be changed so now its a bit confusing. For example, my configuration > sets a dedicate namenode, secondary namenode and 8 slave nodes but when I > run the start command it gives an error. Should I install hadoop to my user > directory or on the root? I have it in my directory but all the nodes have a > central file system as opposed to distributed so whatever I do on one node > in my user folder it affect all the others so how do i set the paths to > ensure that it uses a distributed system? > >> > >> For the errors below, I checked the directories and the files are there. > Am I not sure what went wrong and how to set the conf to not have central > file system. Thank you. > >> > >> Error message > >> CODE > >> w1153435@n51:~/hadoop-0.20.2_cluster> bin/start-dfs.sh > >> bin/start-dfs.sh: line 28: > /w1153435/hadoop-0.20.2_cluster/bin/hadoop-config.sh: No such file or > directory > >> bin/start-dfs.sh: line 50: > /w1153435/hadoop-0.20.2_cluster/bin/hadoop-daemon.sh: No such file or > directory > >> bin/start-dfs.sh: line 51: > /w1153435/hadoop-0.20.2_cluster/bin/hadoop-daemons.sh: No such file or > directory > >> bin/start-dfs.sh: line 52: > /w1153435/hadoop-0.20.2_cluster/bin/hadoop-daemons.sh: No such file or > directory > >> CODE > > > >there's No such file or directory as > >/w1153435/hadoop-0.20.2_cluster/bin/hadoop-daemons.sh > > > > > >There is, I checked as shown > >w1153435@n51:~/hadoop-0.20.2_cluster> ls bin > >hadoop rcc start-dfs.sh stop-dfs.sh > >hadoop-config.sh slaves.sh start-mapred.sh stop-mapred.sh > >hadoop-daemon.sh start-all.sh stop-all.sh > >hadoop-daemons.sh start-balancer.sh stop-balancer.sh > > > > > > > > > >> > >> I had tried running this command below earlier but also got problems: > >> CODE > >> w1153435@ngs:~/hadoop-0.20.2_cluster> export > HADOOP_CONF_DIR=${HADOOP_HOME}/conf > >> w1153435@ngs:~/hadoop-0.20.2_cluster> export > HADOOP_SLAVES=${HADOOP_CONF_DIR}/slaves > >> w1153435@ngs:~/hadoop-0.20.2_cluster> ${HADOOP_HOME}/bin/slaves.sh > "mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop" > >> -bash: /bin/slaves.sh: No such file or directory > >> w1153435@ngs:~/hadoop-0.20.2_cluster> export > HADOOP_HOME=/home/w1153435/hadoop-0.20.2_cluster > >> w1153435@ngs:~/hadoop-0.20.2_cluster> ${HADOOP_HOME}/bin/slaves.sh > "mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop" > >> cat: /conf/slaves: No such file or directory > >> CODE > >> > >there's No such file or directory as /conf/slaves because you set > >HADOOP_HOME after setting the other env variables, which are expanded at > >set-time, not run-time. > > > >I redid the command but still have errors on the slaves > > > > > >w1153435@n51:~/hadoop-0.20.2_cluster> export > HADOOP_HOME=/home/w1153435/hadoop-0.20.2_cluster > >w1153435@n51:~/hadoop-0.20.2_cluster> export > HADOOP_CONF_DIR=${HADOOP_HOME}/conf > >w1153435@n51:~/hadoop-0.20.2_cluster> export > HADOOP_SLAVES=${HADOOP_CONF_DIR}/slaves > >w1153435@n51:~/hadoop-0.20.2_cluster> ${HADOOP_HOME}/bin/slaves.sh "mkdir > -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop" > >privn51: bash: mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop: > No such file or directory > >privn58: bash: mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop: > No such file or directory > >privn52: bash: mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop: > No such file or directory > >privn55: bash: mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop: > No such file or directory > >privn57: bash: mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop: > No such file or directory > >privn54: bash: mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop: > No such file or directory > >privn53: bash: mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop: > No such file or directory > >privn56: bash: mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop: > No such file or directory > > > > -- Thanks, Shah
