Re: read from a hdfs file on the same host as client

2014-10-13 Thread Demai Ni
Shivram, many thanks for confirming the behavior. I will also turn on the shortcircuit as you suggested. Appreciate the help Demai On Mon, Oct 13, 2014 at 3:42 PM, Shivram Mani wrote: > Demai, you are right. HDFS's default BlockPlacementPolicyDefault makes > sure one replica of your block is a

Re: read from a hdfs file on the same host as client

2014-10-13 Thread Shivram Mani
Demai, you are right. HDFS's default BlockPlacementPolicyDefault makes sure one replica of your block is available on the writer's datanode. The replica selection for the read operation is also aimed at minimizing bandwidth/latency and will serve the block from the reader's local node. If you want

read from a hdfs file on the same host as client

2014-10-13 Thread Demai Ni
hi, folks, a very simple question, looking forward a couple pointers. Let's say I have a hdfs file: testfile, which only have one block(256MB), and the block has a replica on datanode: host1.hdfs.com (the whole hdfs may have 100 nodes though, and the other 2 replica are available at other datanod