[
https://issues.apache.org/jira/browse/MAPREDUCE-4671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13460050#comment-13460050
]
Karthik Kambatla commented on MAPREDUCE-4671:
---------------------------------------------
Thanks Bikas for filing this JIRA.
I think this issue can be addressed in two ways:
# As you suggest, the AM can inform the RM about the container requests that
are no longer needed.
# Alternatively, we can augment the ResourceRequest API to allow the AM to
convey that it needs (1) 'x' of the 'y' containers that it is requesting, and
(2) if the RM can invalidate the remaining ('y' - 'x' ) container requests.
This way, we can avoid (1) the communication overhead, and (2) redundant work
on the RM side.
What do you think?
> AM does not tell the RM about container requests that are no longer needed
> --------------------------------------------------------------------------
>
> Key: MAPREDUCE-4671
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4671
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Bikas Saha
> Assignee: Bikas Saha
> Attachments: MAPREDUCE-4671.1.patch
>
>
> Say the AM wanted a container at hosts h1, h2, h3. After getting a container
> at h1 it should tell RM that it no longer needs containers at h2, h3.
> Otherwise on the RM h2, h3 remain valid allocation locations.
> The AM RMContainerAllocator does remove these resource requests internally.
> When the resource request container count drops to 0 then it drops the
> resource request from its tables but forgets to send the 0 sized request to
> the RM.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
