[
https://issues.apache.org/jira/browse/YARN-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632511#comment-13632511
]
Carlo Curino commented on YARN-45:
----------------------------------
[~bikassaha]
Sounds good, I totally agree with the general spirit. We indeed have code that
exercises the ResourceRequest version of it, thus it is actionable (detailed
question later).
[~tucu00]
bq. Wouldn't be enough just to return the ContainerIds and a flag indicating
that the set is strict or not? The AM can reconstruct all the resources
information if it needs to.
I think it is important to have the "resource-based" version because if the RM
wants a large number of containers back (e.g., 1000) and does not care which
ones,
it would be very wasteful to resolve them on the RM (extra code, extra
compute-cost), send a long detailed list, and have the AM simply aggregate the
resources
ignoring the individuals in the list and return some other containers.
[~bikassaha] so previously very broad question can be rephrased, based on
[~tucu00] review of the patch, more tightly as follows:
Which of the following options you prefer?
# we reuse ResourceRequest of which we use number of containers and Resource
for each container (and for now not use locality or priority, although we might
in the future)
# create a new type that carries *only* number of containers and Resource for
each container
Pros and cons of reusing existing types vs minimalistic approach which you were
pointing out before. I don't have much of a preference (minor leaning towards
1, but either way is fine).
> Scheduler feedback to AM to release containers
> ----------------------------------------------
>
> Key: YARN-45
> URL: https://issues.apache.org/jira/browse/YARN-45
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Reporter: Chris Douglas
> Assignee: Carlo Curino
> Attachments: YARN-45.patch, YARN-45.patch
>
>
> The ResourceManager strikes a balance between cluster utilization and strict
> enforcement of resource invariants in the cluster. Individual allocations of
> containers must be reclaimed- or reserved- to restore the global invariants
> when cluster load shifts. In some cases, the ApplicationMaster can respond to
> fluctuations in resource availability without losing the work already
> completed by that task (MAPREDUCE-4584). Supplying it with this information
> would be helpful for overall cluster utilization [1]. To this end, we want to
> establish a protocol for the RM to ask the AM to release containers.
> [1] http://research.yahoo.com/files/yl-2012-003.pdf
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
