[
https://issues.apache.org/jira/browse/MAPREDUCE-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated MAPREDUCE-4893:
-----------------------------------------------
Status: Open (was: Patch Available)
bq. The else if() is a refactoring of existing code. I think its needed
because its checking that the RM did not return an un-requested priority. So we
need to verify that the last remaining priority does not match too.
Your logic is correct but the code isn't. The first condition checks if it is a
normal or a failed map, second one checks if it is a reduce, between them they
cover all valid cases. The third one should be unconditional.
> MR AppMaster can do sub-optimal assignment of containers to map tasks leading
> to poor node locality
> ---------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4893
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4893
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Fix For: 3.0.0
>
> Attachments: MAPREDUCE-4893.1.patch, MAPREDUCE-4893.2.patch
>
>
> Say the MR AppMaster asks the RM for 3 containers on nodes n1, n2 and n3.
> There are 10 node n1-n10 in the same rack. The RM can give it allocated
> containers in the list order n5, n2, n1. The way AM map->container assignment
> happens, the AM will try to assign node local maps to n5, failing which it
> will assign rack local maps to n5. These rack local maps could be node local
> on n2 and n1 and would have been assigned to containers on n1 and n2 if the
> AM had not made an early rack local match for them on n5. This can lead to
> poor locality.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
