[
https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14556868#comment-14556868
]
MENG DING commented on YARN-1197:
---------------------------------
So to summarize the current dilemma:
Situation:
- A container resource increase request has been granted, and a token has been
issued to AM, and
- The increase action has not been fulfilled, and the token is not expired yet
Problem:
- AM can initiate a container resource decrease action to NM, and NM will
fulfill it and notify RM, and then
- Before the toke expires, AM can still initiate a container resource increase
action to NM with the token, and NM will fulfill it and notify RM
Proposed solution:
- When RM receives a container decrease message from NM, it will first check if
there is an outstanding container increase action (by checking the
ContainerResourceIncreaseExpirer)
- If the answer is no, RM will go ahead and update its internal resource
bookkeeping and reduce the container resource allocation for this container.
- If the answer is yes, RM will skip the resource reduction in this cycle, keep
the resource decrease message in its newlyDecreasedContainers data structure,
and check again in the next NM-RM heartbeat cycle.
- If in the next heartbeat, a resource increase message to the same container
comes, the previous resource decrease message will be dropped.
Not sure if there are better solution to this problem. Let me know if this
makes sense or not.
Thanks,
Meng
> Support changing resources of an allocated container
> ----------------------------------------------------
>
> Key: YARN-1197
> URL: https://issues.apache.org/jira/browse/YARN-1197
> Project: Hadoop YARN
> Issue Type: Task
> Components: api, nodemanager, resourcemanager
> Affects Versions: 2.1.0-beta
> Reporter: Wangda Tan
> Attachments: mapreduce-project.patch.ver.1,
> tools-project.patch.ver.1, yarn-1197-scheduler-v1.pdf, yarn-1197-v2.pdf,
> yarn-1197-v3.pdf, yarn-1197-v4.pdf, yarn-1197-v5.pdf, yarn-1197.pdf,
> yarn-api-protocol.patch.ver.1, yarn-pb-impl.patch.ver.1,
> yarn-server-common.patch.ver.1, yarn-server-nodemanager.patch.ver.1,
> yarn-server-resourcemanager.patch.ver.1
>
>
> The current YARN resource management logic assumes resource allocated to a
> container is fixed during the lifetime of it. When users want to change a
> resource
> of an allocated container the only way is releasing it and allocating a new
> container with expected size.
> Allowing run-time changing resources of an allocated container will give us
> better control of resource usage in application side
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
