MENG DING commented on YARN-1197:

So to summarize the current dilemma:

- A container resource increase request has been granted, and a token has been 
issued to AM, and
- The increase action has not been fulfilled, and the token is not expired yet

- AM can initiate a container resource decrease action to NM, and NM will 
fulfill it and notify RM, and then
- Before the toke expires, AM can still initiate a container resource increase 
action to NM with the token, and NM will fulfill it and notify RM

Proposed solution:
- When RM receives a container decrease message from NM, it will first check if 
there is an outstanding container increase action (by checking the 
- If the answer is no, RM will go ahead and update its internal resource 
bookkeeping and reduce the container resource allocation for this container.
- If the answer is yes, RM will skip the resource reduction in this cycle, keep 
the resource decrease message in its newlyDecreasedContainers data structure, 
and check again in the next NM-RM heartbeat cycle.
- If in the next heartbeat, a resource increase message to the same container 
comes, the previous resource decrease message will be dropped.

Not sure if there are better solution to this problem. Let me know if this 
makes sense or not.


> Support changing resources of an allocated container
> ----------------------------------------------------
>                 Key: YARN-1197
>                 URL: https://issues.apache.org/jira/browse/YARN-1197
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: api, nodemanager, resourcemanager
>    Affects Versions: 2.1.0-beta
>            Reporter: Wangda Tan
>         Attachments: mapreduce-project.patch.ver.1, 
> tools-project.patch.ver.1, yarn-1197-scheduler-v1.pdf, yarn-1197-v2.pdf, 
> yarn-1197-v3.pdf, yarn-1197-v4.pdf, yarn-1197-v5.pdf, yarn-1197.pdf, 
> yarn-api-protocol.patch.ver.1, yarn-pb-impl.patch.ver.1, 
> yarn-server-common.patch.ver.1, yarn-server-nodemanager.patch.ver.1, 
> yarn-server-resourcemanager.patch.ver.1
> The current YARN resource management logic assumes resource allocated to a 
> container is fixed during the lifetime of it. When users want to change a 
> resource 
> of an allocated container the only way is releasing it and allocating a new 
> container with expected size.
> Allowing run-time changing resources of an allocated container will give us 
> better control of resource usage in application side

This message was sent by Atlassian JIRA

Reply via email to