Hi,
I've built a Hadoop cluster from two computers( master and slave), using Hadoop 0.18.2/HBase 0.18.1, if I use big files as input( 5-10 GB) reduce-copy tasks from master to slave is taking too mush time( ~30 minutes) with speed about 0.10 MB/s, despite the fact that master is connected to slave via 1GB switch, and I did /etc/hosts mapping using LAN addresses(10.x.x.x), is there is a way to force hadoop to use ftp for example for copy of files? Or there is some configuration to improve copy files performance? I've tried to copy files with ftp and it works with average speed 50Mb/s. >From reduce task lists: reduce > copy (67 of 69 at 0.89 MB/s) > : task on master reduce > copy (29 of 69 at 0.10 MB/s) > : task on slave Genady
