[
https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13768180#comment-13768180
]
Wangda Tan commented on YARN-1197:
----------------------------------
Here is the tentative plan:
To make it simple, we can start with only support increase container resource
when container state is "ACQUIRED", because when container is running, we need
find a way to inform NM to update the quato of a running process, which will
affect lots of parts.
# Add a field in ResourceRequest, like set/getExistingContainerId() to indicate
the resource will be added to existing container
# When the ask-list submitted to YarnScheduler, it will be normalized:
** When existingContainerId set
*** Check if container with "existingContainerId" in state ACQUIRED, abondon
this request if container not exist OR state not ACQUIRED.
*** Check if capability of this container + asked capability >
scheduler.maximum-allocation-mb, if yes, normalize ask capability
*** Other fields will be ignored except "capability", and the "priority/host"
will use the existing container's priority/host
*** Update resource requests in AppSchedulingInfo
# Add "increase existing container resource" support in LeafQueue of capa/fifo
scheduler, and FSLeafQueue in fair scheduler. We can treat "increase existing
container resource request" as same as other resource request except,
** Only "node-local" available
** Don't create new container
# After we successfully allocated resource to an existing container, we need
refresh its container token, send back to AM. AM can use the new container to
launch processes via NM.
Any thoughts? I wish I could help to move this ahead if we can reach an
agreement on a rough idea. :)
> Support increasing resources of an allocated container
> ------------------------------------------------------
>
> Key: YARN-1197
> URL: https://issues.apache.org/jira/browse/YARN-1197
> Project: Hadoop YARN
> Issue Type: Task
> Components: api, nodemanager, resourcemanager
> Affects Versions: 2.1.0-beta
> Reporter: Wangda Tan
>
> Currently, YARN cannot support merge several containers in one node to a big
> container, which can make us incrementally ask resources, merge them to a
> bigger one, and launch our processes. The user scenario is described in the
> comments.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
