Hello,
Glad I could help...
I also didn't understand why hadoop uses hostnames even when IP's are
specified in the configuration files... in my (perhaps misinformed)
opinion hadoop should avoid name resolution has much has it can, because
it's just another point of failure... Can anybody explain why this happens?
Pedro Guedes
Yiqing Jin wrote:
thanks for your help Pedro,
i edited my /etc/hosts and added all the nodes as you said, after that
the
program works fine.
i just don't get it why we need to connect using DNS since we already
have
the ip address, and we only get connection failure at program run
time....
that is hard for me to find out if without your help., thank you.
2007/5/11, Pedro Guedes <[EMAIL PROTECTED]>:
Hello,
I had a similar problem with hadoop some time ago... by looking at the
logs, I found out that the problem was with resolving hostnames when the
HDFS nodes communicated with each other...
I fixed it by editing my /etc/hosts and adding all the nodes I'm
using...
Hope this helps,
Pedro Guedes
Yiqing Jin wrote:
> Hi,
> I;m new to hadoop and i downloaded hadoop-0.12.3 from apahce
website and
> installed it to a server with RedHat EL4. I followed the instruction
from
> wiki page:
> 1. formatted the namenod,.
> 2. use hadoop dfs -put to put a txt file to dfs(It's the
LICENSE.txt in
> hadoop root directory).
> 3. use hadoop dfs -ls , and the file is listed correctly.
> 4. try ./hadoop jar ../hadoop-0.12.3-examples.jar grep /user/hadoop/
> /user/hadoop/out apache, everything works fine and i got the result.
>
> Next i tried to Starting up a larger cluster, set the ssh and scp a
> copy to
> another server, formatted the namenode again, started hadoop using
> start-all.sh.
> Then i tried the example again, but it seems will not finish and just
> hanging there:
>
> [EMAIL PROTECTED] bin]$ ./start-all.sh
> starting namenode, logging to
> /home/hadoop/hadoop-0.12.3/bin/../logs/hadoop-
> hadoop-namenode-linux5.out
> localhost: starting datanode, logging to /home/hadoop/hadoop-0.12.3
> /bin/../logs/hadoop-hadoop-datanode-linux5.out
> 172.16.201.6: starting datanode, logging to /home/hadoop/hadoop-0.12.3
> /bin/../logs/hadoop-hadoop-datanode-linux6.out
> localhost: starting secondarynamenode, logging to
> /home/hadoop/hadoop-0.12.3
> /bin/../logs/hadoop-hadoop-secondarynamenode-linux5.out
> starting jobtracker, logging to /home/hadoop/hadoop-0.12.3
> /bin/../logs/hadoop-hadoop-jobtracker-linux5.out
> 172.16.201.6: starting tasktracker, logging to /home/hadoop/hadoop-
0.12.3
> /bin/../logs/hadoop-hadoop-tasktracker-linux6.out
> localhost: starting tasktracker, logging to /home/hadoop/hadoop-0.12.3
> /bin/../logs/hadoop-hadoop-tasktracker-linux5.out
>
> [EMAIL PROTECTED] bin]$ ./hadoop jar ../hadoop-0.12.3-examples.jar grep
> /user/hadoop/ /user/hadoop/out apache
> 07/05/11 18:34:49 INFO mapred.InputFormatBase: Total input paths to
> process
> : 1
> 07/05/11 18:34:49 INFO mapred.JobClient: Running job: job_0002
> 07/05/11 18:34:50 INFO mapred.JobClient: map 0% reduce 0%
> 07/05/11 18:34:56 INFO mapred.JobClient: map 50% reduce 0%
> 07/05/11 18:34:57 INFO mapred.JobClient: map 100% reduce 0%
> 07/05/11 18:35:05 INFO mapred.JobClient: map 100% reduce 8%
> 07/05/11 18:35:06 INFO mapred.JobClient: map 100% reduce 16%
>
> the reduce task just processed to 16%.
> So i waited about an hour the job is still here, could some one help
> me find
> out what i did wrong?
>
> Thanks,
> Yiqing
>
