[
https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846064#comment-13846064
]
Sandy Ryza commented on YARN-1197:
----------------------------------
Wanted to post some thoughts on this vs. YARN-1488. YARN-1488 proposes that if
you receive a container on a node you should be able to delegate it to another
container already running on that node, essentially adding the resources of the
container you received to the allocation of the running container. This sounds
a lot like a resource increase. The differences are that:
* With the mechanism proposed in this JIRA, the request is explicitly an
increase and mentions the container you want to add it to. This allows the
scheduler to use special logic for handling increase requests.
* With the mechanism proposed on YARN-1488, a container can be used to increase
the resources of a container from another application.
* With the mechanism proposed here, after satisfying the increase request, the
scheduler is tracking a single larger container, not multiple small ones.
An advantage of treating an increase request the same as a regular container
request is that an application could submit it at the same time. I.e. if I
want a single container with as many resources as possible on node X, I can
request a number of containers on that node, wait for some time period for
allocations to accrue, and then run them all as a single container.
I think the deciding factor might be how preemption functions here. I.e. what
is the preemptable unit - can we preempt a part of a container? Have some
thoughts but will try to organize them more before posting here.
> Support changing resources of an allocated container
> ----------------------------------------------------
>
> Key: YARN-1197
> URL: https://issues.apache.org/jira/browse/YARN-1197
> Project: Hadoop YARN
> Issue Type: Task
> Components: api, nodemanager, resourcemanager
> Affects Versions: 2.1.0-beta
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Attachments: mapreduce-project.patch.ver.1,
> tools-project.patch.ver.1, yarn-1197-v2.pdf, yarn-1197-v3.pdf,
> yarn-1197-v4.pdf, yarn-1197-v5.pdf, yarn-1197.pdf,
> yarn-api-protocol.patch.ver.1, yarn-pb-impl.patch.ver.1,
> yarn-server-common.patch.ver.1, yarn-server-nodemanager.patch.ver.1,
> yarn-server-resourcemanager.patch.ver.1
>
>
> The current YARN resource management logic assumes resource allocated to a
> container is fixed during the lifetime of it. When users want to change a
> resource
> of an allocated container the only way is releasing it and allocating a new
> container with expected size.
> Allowing run-time changing resources of an allocated container will give us
> better control of resource usage in application side
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
