blem. Rack Awareness is
described here:
https://issues.apache.org/jira/secure/attachment/12345251/Rack_aware_HDFS_pr
oposal.pdf
- Alex
--
Alexander Lorenz
http://mapredit.blogspot.com
On Jan 5, 2012, at 6:49 PM, Ben Clay wrote:
> Suresh-
> Thanks for the tips, I'll check those functio
ly it sorts the block locations as:
# local node
# local rack node
# random order of remote nodes
See DatanodeManager#sortLocatedBlock(...) and
NetworkTopology#pseudoSortByDistance(...).
You can play around with other policies by plugging in different
NetworkTopology.
On Thu, Jan 5, 2012 at 1:40 PM
Hi-
How does the NameNode handle load balancing of non-local reads with multiple
block locations when locality is equal?
IE, if the client is equidistant (same rack) from 2 DataNodes hosting the
same block, does the NameNode consider current client count or any other
load indicators when de
/dest/path
Ben Clay
rbc...@ncsu.edu
-Original Message-
From: Todd Lipcon [mailto:t...@cloudera.com]
Sent: Tuesday, August 30, 2011 10:45 AM
To: hdfs-user@hadoop.apache.org
Subject: Re: set reduced block size for a specific file
0.21 has some bugs with config option deprecation that you&
his JIRA to add this feature:
> > https://issues.apache.org/jira/browse/HDFS-2293
> >
> > If anyone would like to implement this, I would be happy to
> review it.
> >
> > Thanks a lot,
> > Aaron
> >
> > --
> > Aaron T. Myers
> > Sof
I didn't even think of overriding the config dir. Thanks for the tip!
-Ben
-Original Message-
From: Allen Wittenauer [mailto:a...@apache.org]
Sent: Saturday, August 27, 2011 6:42 PM
To: hdfs-user@hadoop.apache.org
Cc: rbc...@ncsu.edu
Subject: Re: set reduced block size for a specific f
I'd like to set a lowered block size for a specific file. IE, if HDFS is
configured to use 64mb blocks, I'd like to use 32mb blocks for a specific
file.
Is there a way to do this from the commandline, without writing a jar which
uses org.apache.hadoop.fs.FileSystem.create() ?
I tried the f
Hi-
We're working in an environment where we'd like to add a new DataNode and
quickly begin leveraging it, usually after detection of high load at the
rest of our DataNodes.
However, most of the methods I can find regarding placing pre-existing
blocks on a new DataNode involve operations
