Can you give your machines multiple IP addresses, and bind the grid server to a different IP than the datanode With solaris you could put it in a different zone,
On Sat, May 23, 2009 at 10:13 AM, Brian Bockelman <bbock...@math.unl.edu>wrote: > Hey all, > > Had a problem I wanted to ask advice on. The Caltech site I work with > currently have a few GridFTP servers which are on the same physical machines > as the Hadoop datanodes, and a few that aren't. The GridFTP server has a > libhdfs backend which writes incoming network data into HDFS. > > They've found that the GridFTP servers which are co-located with HDFS > datanode have poor performance because data is incoming at a much faster > rate than the HDD can handle. The standalone GridFTP servers, however, push > data out to multiple nodes at one, and can handle the incoming data just > fine (>200MB/s). > > Is there any way to turn off the preference for the local node? Can anyone > think of a good workaround to trick HDFS into thinking the client isn't on > the same node? > > Brian -- Alpha Chapters of my book on Hadoop are available http://www.apress.com/book/view/9781430219422 www.prohadoopbook.com a community for Hadoop Professionals
