Hello,
I had a similar problem with hadoop some time ago... by looking at the
logs, I found out that the problem was with resolving hostnames when the
HDFS nodes communicated with each other...
I fixed it by editing my /etc/hosts and adding all the nodes I'm using...
Hope this helps,
Pedro Guedes
Yiqing Jin wrote:
Hi,
I;m new to hadoop and i downloaded hadoop-0.12.3 from apahce website and
installed it to a server with RedHat EL4. I followed the instruction from
wiki page:
1. formatted the namenod,.
2. use hadoop dfs -put to put a txt file to dfs(It's the LICENSE.txt in
hadoop root directory).
3. use hadoop dfs -ls , and the file is listed correctly.
4. try ./hadoop jar ../hadoop-0.12.3-examples.jar grep /user/hadoop/
/user/hadoop/out apache, everything works fine and i got the result.
Next i tried to Starting up a larger cluster, set the ssh and scp a
copy to
another server, formatted the namenode again, started hadoop using
start-all.sh.
Then i tried the example again, but it seems will not finish and just
hanging there:
[EMAIL PROTECTED] bin]$ ./start-all.sh
starting namenode, logging to
/home/hadoop/hadoop-0.12.3/bin/../logs/hadoop-
hadoop-namenode-linux5.out
localhost: starting datanode, logging to /home/hadoop/hadoop-0.12.3
/bin/../logs/hadoop-hadoop-datanode-linux5.out
172.16.201.6: starting datanode, logging to /home/hadoop/hadoop-0.12.3
/bin/../logs/hadoop-hadoop-datanode-linux6.out
localhost: starting secondarynamenode, logging to
/home/hadoop/hadoop-0.12.3
/bin/../logs/hadoop-hadoop-secondarynamenode-linux5.out
starting jobtracker, logging to /home/hadoop/hadoop-0.12.3
/bin/../logs/hadoop-hadoop-jobtracker-linux5.out
172.16.201.6: starting tasktracker, logging to /home/hadoop/hadoop-0.12.3
/bin/../logs/hadoop-hadoop-tasktracker-linux6.out
localhost: starting tasktracker, logging to /home/hadoop/hadoop-0.12.3
/bin/../logs/hadoop-hadoop-tasktracker-linux5.out
[EMAIL PROTECTED] bin]$ ./hadoop jar ../hadoop-0.12.3-examples.jar grep
/user/hadoop/ /user/hadoop/out apache
07/05/11 18:34:49 INFO mapred.InputFormatBase: Total input paths to
process
: 1
07/05/11 18:34:49 INFO mapred.JobClient: Running job: job_0002
07/05/11 18:34:50 INFO mapred.JobClient: map 0% reduce 0%
07/05/11 18:34:56 INFO mapred.JobClient: map 50% reduce 0%
07/05/11 18:34:57 INFO mapred.JobClient: map 100% reduce 0%
07/05/11 18:35:05 INFO mapred.JobClient: map 100% reduce 8%
07/05/11 18:35:06 INFO mapred.JobClient: map 100% reduce 16%
the reduce task just processed to 16%.
So i waited about an hour the job is still here, could some one help
me find
out what i did wrong?
Thanks,
Yiqing
