[ http://issues.apache.org/jira/browse/HADOOP-254?page=all ]
Owen O'Malley updated HADOOP-254:
---------------------------------
Attachment: http-shuffle.patch
This patch introduces the use of http to shuffle the data from maps to reduces.
Using this patch, the shuffle mostly keeps up with the maps during my sort
benchmark.
This patch also allows the task tracker's http servers to try multiple ports
until it finds one that works.
> use http to shuffle data between the maps and the reduces
> ---------------------------------------------------------
>
> Key: HADOOP-254
> URL: http://issues.apache.org/jira/browse/HADOOP-254
> Project: Hadoop
> Type: Improvement
> Components: mapred
> Versions: 0.2.1
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 0.3
> Attachments: http-shuffle.patch
>
> To speed up the shuffle time, I'll use http (via the task tracker's jetty
> server) to send the map outputs.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
