I guess what we need is an example of the "script", where do we put it,
and what exactly to fill in the value of the "topology.script.file.name"
entry.
So, I wrote a program called "mydns". I can run the program
./mydns node1.rack1.yahoo.com
It prints "/rack1" to the screen.
Is this correct? Where to put this program? What to fill the the
configuration file?
Thanks
Yunhong
On Mon, 9 Jun 2008, Devaraj Das wrote:
This documentation is for the earlier versions. In 0.17 the way in which
racks are dealt with has changed.
-----Original Message-----
From: Yang Chen [mailto:[EMAIL PROTECTED]
Sent: Sunday, June 08, 2008 8:06 PM
To: [email protected]
Subject: Re: Hadoop topology.script.file.name Form
Rack Awareness
Typically large Hadoop clusters are arranged in *racks* and
network traffic between different nodes with in the same rack
is much more desirable than network traffic across the racks.
In addition Namenode tries to place replicas of block on
multiple racks for improved fault tolerance. Hadoop lets the
cluster administrators decide which *rack* a node belongs to
through configuration variable dfs.network.script. When this
script is configured, each node runs the script to determine
its *rackid*. A default installation assumes all the nodes
belong to the same rack. This feature and configuration is
further described in
PDF<http://issues.apache.org/jira/secure/attachment/12345251/R
ack_aware_HDFS_proposal.pdf>attached
to
HADOOP-692 <http://issues.apache.org/jira/browse/HADOOP-692>.
Hope this will be helpful.
YC
On Sun, Jun 8, 2008 at 9:53 PM, Devaraj Das
<[EMAIL PROTECTED]> wrote:
Hi Iver,
The implementation of the script depends on your setup. The
main thing
is that it should be able to accept a bunch of IP addresses and DNS
names and be able to give back the rackIDs for each. It is a
one-to-one correspondence between what you pass and what
you get back.
For getting the rackID the script could deduce it from the
IP address,
or, query some service (similar to the way dns works, or
some similar
mechanism), or, in the extreme case, read a file that has
the mapping
from IP address to rackId.
Thanks,
Devaraj.
-----Original Message-----
From: ?? [mailto:[EMAIL PROTECTED]
Sent: Friday, June 06, 2008 8:13 AM
To: core-user
Subject: Hadoop topology.script.file.name Form
hi,
I want to setup a hadoop cluster, and I want to make the
cluster to
be Rack Awareness. But I can't find any document about
the form of
topology.script.file.name.
Could anybody give me an example about the form of
topology.script.file.name?
thanks a lot.
iver
2008-06-06
________________________________
