[
https://issues.apache.org/jira/browse/YARN-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628930#comment-13628930
]
Carlo Curino commented on YARN-45:
----------------------------------
Sorry I read only your last comment and answered to that...
Regarding your previous "larger" comment:
- what you propose is somewhat of a combination of 1 and 2 above, where we give
the AM a hint about what would happen at the container level if the pressure
remains. I don't have strong feelings about it, I agree it is easy to do, and
maybe is a good compromised.
- however, I want to be able to maintain the tighter semantics of 1 (in case
the ResourceRequest is not specified in the message), which forces the AM to
preempt exactly the set of containers I am specifying. (now with very
"targeted" ResourceRequest you can in practice
do something similar). This covers use cases like the one I mentioned above.
We are posting more code in YARN-567 YARN-568 and YARN-569, check it out, it
might provide context for this conversation.
> Scheduler feedback to AM to release containers
> ----------------------------------------------
>
> Key: YARN-45
> URL: https://issues.apache.org/jira/browse/YARN-45
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
> Reporter: Chris Douglas
> Assignee: Carlo Curino
> Attachments: YARN-45.patch
>
>
> The ResourceManager strikes a balance between cluster utilization and strict
> enforcement of resource invariants in the cluster. Individual allocations of
> containers must be reclaimed- or reserved- to restore the global invariants
> when cluster load shifts. In some cases, the ApplicationMaster can respond to
> fluctuations in resource availability without losing the work already
> completed by that task (MAPREDUCE-4584). Supplying it with this information
> would be helpful for overall cluster utilization [1]. To this end, we want to
> establish a protocol for the RM to ask the AM to release containers.
> [1] http://research.yahoo.com/files/yl-2012-003.pdf
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
