Hi Suenghwa, It's important to note that changing the dfs.replication config variable does not change the current files in HDFS. You have to use fs -setrep on those files to change their replication count. The replication count is set when the files were created and not modified thereafter unless you explicitly do so.
-Todd On Fri, Jul 17, 2009 at 4:01 PM, Seunghwa Kang <[email protected]> wrote: > I found I forgot to mention my hadoop version. > > I am using 0.19.1. > > Thanks again, > > -seunghwa > > On Fri, 2009-07-17 at 18:57 -0400, Seunghwa Kang wrote: > > Hello, > > > > I am running Hadoop on my 4 nodes system. > > > > Initially, I pick the replication factor of 2, and nearly 100% of map > > tasks run in local up to 3 nodes, but the ratio drops to 80% if I use > > all 4 nodes. > > > > As my nodes have quite high I/O bandwidth (24 disks per node), but > > only > > limited network bandwidth (1 GigE), this really hampers the > > scalability. > > > > Just for test purpose, I increase the replication factor to 4, and > > check > > that input data actually has replication factor of 4 with 'hadoop fs > > -stat %r%n' but find that the ratio is still around 80% for 4 nodes. > > > >
