You may also check the patch: TableInputFormatFix extends TableInputFormat The way they put it:
// // Attempt to fix the localized nature of table segments. // Compute table splits so that they are processed locally. // Combine multiple splits to avoid the number of splits exceeding numSplits. // Sort the resulting splits so that the shortest ones are processed last. // The resulting savings in network bandwidth are significant (we measured 60%). // Thanks -Yair -----Original Message----- From: Cosmin Lehene [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 26, 2008 5:36 AM To: [email protected] Subject: Re: map/reduce locality It doesn't currently do that. However this seems to be on HBase roadmap. See Data-Locality Awareness The Hadoop map reduce framework does makes a best effort at running tasks on the server hosting the task data after the dictum that its cheaper moving the processing to the data rather than the inverse. HBase needs smarts to assign regions to the region server that is running on the server hosting the regions' data. HBase needs to supply map reduce hints such that the Hadoop framework runs tasks beside the region server hosting the task input. These changes will make for savings in network I/O. http://wiki.apache.org/hadoop/HBase/RoadMaps Regards, Cosmin On 11/26/08 1:32 PM, "David Faitelson" <[EMAIL PROTECTED]> wrote: Hi, Does HBase/Hadoop create map tasks on the same data node that contains the region for the map task? I know that Bigtable does something like that but I could not find any mention of this optimization in the documentation of HBase. Thanks, David
