Thanks a lot for your response David The problem was the rack mapping on the topology.data file. I had originally put hostname -> rack, but hadoop wouldn't take the hostname. so i changed them to ip -> rack, now everything works.
Thanks again, Ben On Thu, Jun 28, 2012 at 11:50 PM, David Rosenstrauch <dar...@darose.net>wrote: > On 06/28/2012 10:46 AM, David Rosenstrauch wrote: > >> On 06/27/2012 11:32 PM, Ben Kim wrote: >> >>> Hi >>> I got my topology script from >>> http://wiki.apache.org/hadoop/**topology_rack_awareness_**scripts<http://wiki.apache.org/hadoop/topology_rack_awareness_scripts> >>> I checked that the script works correctly. >>> >>> But, in the hadoop cluster, all my servers get assigned to the default >>> rack. >>> I'm using hadoop 1.0.3, but had experienced same problem with 1.0.0 >>> version. >>> >>> >>> Yunhong was having the same problem in the past without any resolution. >>> http://mail-archives.apache.**org/mod_mbox/hadoop-common-** >>> user/200807.mbox/%3CPine.LNX.**4.64.0807031453070.28079@bert.** >>> cs.uic.edu%3E<http://mail-archives.apache.org/mod_mbox/hadoop-common-user/200807.mbox/%3cpine.lnx.4.64.0807031453070.28...@bert.cs.uic.edu%3E> >>> >>> >>> *Benjamin Kim* >>> *benkimkimben at gmail* >>> >> >> We've used this script for rack awareness: >> >> #!/bin/sh >> HADOOP_CONF=/etc/hadoop/conf >> while [ $# -gt 0 ] ; do >> nodeArg=$1 >> exec< ${HADOOP_CONF}/topology.data >> result="" >> while read line ; do >> ar=( $line ) >> if [ "${ar[0]}" = "$nodeArg" ] ; then >> result="${ar[1]}" >> fi >> done >> shift >> if [ -z "$result" ] ; then >> echo -n "/default/rack " >> else >> echo -n "$result " >> fi >> done >> >> >> The topology data file looks like so: >> >> 192.168.8.50 /dc1/rack1 >> 192.168.8.70 /dc1/rack2 >> 192.168.8.90 /dc1/rack3 >> ... >> >> HTH, >> >> DR >> > > By the way, don't forget you have to set the > topology.script.file.nameproperty in the core-site.xml on your NameNode. > (And then restart your > NameNode.) i.e.: > > <property> > <name>topology.script.file.**name <http://topology.script.file.name> > </name> > <value>/etc/hadoop/conf/**topology.sh</value> > </property> > > HTH, > > DR > -- *Benjamin Kim* *benkimkimben at gmail*
