[
https://issues.apache.org/jira/browse/HADOOP-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12563406#action_12563406
]
Amar Kamat commented on HADOOP-2729:
------------------------------------
Some points to note
1) For cached TIPs, we can expect one more split on the same rack and use that
node to host the backup copy (making the slave selection easy)
2) The JT can safely assume that if the cached-map is completed then the backup
copy is also ready and its upto the TT's to perform this backup.
3) Load balancing can be done by the JT for the shuffle phase so that the map
output serving is distributed across the nodes hosting the map output copy.
> Rack level copy of map outputs
> ------------------------------
>
> Key: HADOOP-2729
> URL: https://issues.apache.org/jira/browse/HADOOP-2729
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Amar Kamat
>
> In case of maps taking of lot of time to complete, a rack level copy of the
> map output can be maintained so that incase of master node for that map goes
> down the other copy can be served. This will be useful since re-executing the
> map can be time consuming and rack-level copy is much cheaper than
> network-level copy.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
