See inline
>________________________________ >From: Steve Loughran <[email protected]> >To: [email protected] >Sent: Tuesday, 16 August 2011, 11:08 >Subject: Re: hadoop cluster mode not starting up > >On 16/08/11 11:02, A Df wrote: >> Hello All: >> >> I used a combination of tutorials to setup hadoop but most seems to be using >> either an old version of hadoop or only using 2 machines for the cluster >> which isn't really a cluster. Does anyone know of a good tutorial which >> setups multiple nodes for a cluster?? I already looked at the Apache website >> but it does not give sample values for the conf files. Also each set of >> tutorials seem to have a different set of parameters which they indicate >> should be changed so now its a bit confusing. For example, my configuration >> sets a dedicate namenode, secondary namenode and 8 slave nodes but when I >> run the start command it gives an error. Should I install hadoop to my user >> directory or on the root? I have it in my directory but all the nodes have a >> central file system as opposed to distributed so whatever I do on one node >> in my user folder it affect all the others so how do i set the paths to >> ensure that it uses a distributed system? >> >> For the errors below, I checked the directories and the files are there. Am >> I not sure what went wrong and how to set the conf to not have central file >> system. Thank you. >> >> Error message >> CODE >> w1153435@n51:~/hadoop-0.20.2_cluster> bin/start-dfs.sh >> bin/start-dfs.sh: line 28: >> /w1153435/hadoop-0.20.2_cluster/bin/hadoop-config.sh: No such file or >> directory >> bin/start-dfs.sh: line 50: >> /w1153435/hadoop-0.20.2_cluster/bin/hadoop-daemon.sh: No such file or >> directory >> bin/start-dfs.sh: line 51: >> /w1153435/hadoop-0.20.2_cluster/bin/hadoop-daemons.sh: No such file or >> directory >> bin/start-dfs.sh: line 52: >> /w1153435/hadoop-0.20.2_cluster/bin/hadoop-daemons.sh: No such file or >> directory >> CODE > >there's No such file or directory as >/w1153435/hadoop-0.20.2_cluster/bin/hadoop-daemons.sh > > >There is, I checked as shown >w1153435@n51:~/hadoop-0.20.2_cluster> ls bin >hadoop rcc start-dfs.sh stop-dfs.sh >hadoop-config.sh slaves.sh start-mapred.sh stop-mapred.sh >hadoop-daemon.sh start-all.sh stop-all.sh >hadoop-daemons.sh start-balancer.sh stop-balancer.sh > > > > >> >> I had tried running this command below earlier but also got problems: >> CODE >> w1153435@ngs:~/hadoop-0.20.2_cluster> export >> HADOOP_CONF_DIR=${HADOOP_HOME}/conf >> w1153435@ngs:~/hadoop-0.20.2_cluster> export >> HADOOP_SLAVES=${HADOOP_CONF_DIR}/slaves >> w1153435@ngs:~/hadoop-0.20.2_cluster> ${HADOOP_HOME}/bin/slaves.sh "mkdir >> -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop" >> -bash: /bin/slaves.sh: No such file or directory >> w1153435@ngs:~/hadoop-0.20.2_cluster> export >> HADOOP_HOME=/home/w1153435/hadoop-0.20.2_cluster >> w1153435@ngs:~/hadoop-0.20.2_cluster> ${HADOOP_HOME}/bin/slaves.sh "mkdir >> -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop" >> cat: /conf/slaves: No such file or directory >> CODE >> >there's No such file or directory as /conf/slaves because you set >HADOOP_HOME after setting the other env variables, which are expanded at >set-time, not run-time. > >I redid the command but still have errors on the slaves > > >w1153435@n51:~/hadoop-0.20.2_cluster> export >HADOOP_HOME=/home/w1153435/hadoop-0.20.2_cluster >w1153435@n51:~/hadoop-0.20.2_cluster> export >HADOOP_CONF_DIR=${HADOOP_HOME}/conf >w1153435@n51:~/hadoop-0.20.2_cluster> export >HADOOP_SLAVES=${HADOOP_CONF_DIR}/slaves >w1153435@n51:~/hadoop-0.20.2_cluster> ${HADOOP_HOME}/bin/slaves.sh "mkdir -p >/home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop" >privn51: bash: mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop: No >such file or directory >privn58: bash: mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop: No >such file or directory >privn52: bash: mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop: No >such file or directory >privn55: bash: mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop: No >such file or directory >privn57: bash: mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop: No >such file or directory >privn54: bash: mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop: No >such file or directory >privn53: bash: mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop: No >such file or directory >privn56: bash: mkdir -p /home/w1153435/hadoop-0.20.2_cluster/tmp/hadoop: No >such file or directory > >
