Bharath, You may want to take a look at http://issues.apache.org/jira/browse/HDFS-782 and continue the discussion on that.
On Sun, Oct 16, 2011 at 7:23 AM, Bharath Ravi <bharathra...@gmail.com> wrote: > Hi all, > > I have a question about how HDFS load balances requests for files/blocks: > > HDFS currently distributes data blocks randomly, for balance. > However, if certain files/blocks are more popular than others, some nodes > might get an "unfair" number of requests. > Adding more replicas for these popular files might not help, unless HDFS > explicitly distributes requests fairly among the replicas. > > Looking around, I saw a bit of discussion on this dating to 2009, > Is something like this is done by HDFS today? > > http://www.mail-archive.com/core-user@hadoop.apache.org/msg04357.html > http://grokbase.com/p/hadoop.apache.org/common-dev/2009/02/re-hdfs-load-traffic-balancing/18en5o4z2j4b6eaqnemkp4eppd7q > > Thanks a lot in advance! > -- > Bharath Ravi > -- Harsh J
