[
https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846075#comment-13846075
]
Bikas Saha commented on YARN-1197:
----------------------------------
I am afraid we seem to mixing things over here. This jira deals with the issue
of increasing and decreasing the resources of an allocated container. There are
clear use cases for it like mentioned in previous comments on this jira. e.g.
having a long running worker daemon increase and decrease its resources
depending on load. We have already discussed at length on this jira on how
increasing a container resource is internally no different than requesting for
an additional container and merging it with an existing container. However the
new container followed by merge is way more complicated for the user and adds
additional complexity to the system (eg how to deal with the new container that
was merged into the old one). This complexity is in addition to the common work
with simply increasing resources on a given container. wrt the user, asking for
a container and being able to increase its resources will give the same effect
as asking for many containers and merging them.
The scenario for YARN-1488 is logically different. That covers the case when an
app wants to use a shared service and purchases that service by transferring
its own container resource to that shared service that itself is running inside
YARN. The consumer app may never need to increase its own container resource.
Secondly, the shared service is not requesting an increase in its own container
resources. So this jira does not come into the picture at all.
I believe we have a clear and cleanly separated piece of useful functionality
being implemented in this jira. We should go ahead and bring this work to
completion and facilitate the creation of long running services in YARN.
wrt doing this in a branch. There are new API's being added here for which
functionality does not exist or is not supported yet. And none of that code
will get executed until clients actually support doing it or someone writes
code against it. So I dont think that any of this is going to destabilize the
code base. I agree that the scheduler changes are going to be complicated. We
can do them in the end when all the plumbing is in place and they could be
separate jiras for each scheduler. Of course, schedulers would want their own
flags to turn this on/off. So its not clear to me what benefits a branch would
bring here but it would entail the overhead of maintenance and lack of test
automation. Does this mean that every feature addition to YARN needs to be done
in a branch? I propose we do this work in trunk and later merge it into
branch-2 when we are satisfied with its stability.
> Support changing resources of an allocated container
> ----------------------------------------------------
>
> Key: YARN-1197
> URL: https://issues.apache.org/jira/browse/YARN-1197
> Project: Hadoop YARN
> Issue Type: Task
> Components: api, nodemanager, resourcemanager
> Affects Versions: 2.1.0-beta
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Attachments: mapreduce-project.patch.ver.1,
> tools-project.patch.ver.1, yarn-1197-v2.pdf, yarn-1197-v3.pdf,
> yarn-1197-v4.pdf, yarn-1197-v5.pdf, yarn-1197.pdf,
> yarn-api-protocol.patch.ver.1, yarn-pb-impl.patch.ver.1,
> yarn-server-common.patch.ver.1, yarn-server-nodemanager.patch.ver.1,
> yarn-server-resourcemanager.patch.ver.1
>
>
> The current YARN resource management logic assumes resource allocated to a
> container is fixed during the lifetime of it. When users want to change a
> resource
> of an allocated container the only way is releasing it and allocating a new
> container with expected size.
> Allowing run-time changing resources of an allocated container will give us
> better control of resource usage in application side
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
