MENG DING commented on YARN-1651:

bq. Do you have any use case in your mind that AM should know about when/if 
container is decreased by NM?
You are right that there doesn't seem to be a compelling use case where AM 
needs to know if container resource has decreased or not in NM. We should keep 
it as is.

bq. I think we only need to remember latest confirmed capacity (including 
decreased, confirmed from NM). In your example, we only need to remember 4G, 
token2 will overwrite token1 and expiration timout will be reset when second 
increase request approved.

So, IIUC, if we only remember last confirmed capacity, using my example, the 
sequence of events will be:

1. Initially a container uses 4GB
2. The first increase request increases it to 6GB (token1)
3. The second increase request increases it to 8GB (token2). The rollback 
resource is still 4GB, and expiration timeout will be reset in RM.
4. AM uses the first token to increase container resource to 6GB in NM
5. NM reports 6GB to RM during heartbeat, and RM will do nothing because of the 
else if (Resources.fitsIn(nmContainerResource, rmContainerResource)) {
      // when rmContainerResource >= nmContainerResource, we won't do anything,
      // it is possible a container increased is issued by RM, but AM hasn't
      // told NM.
6. token2 is never used by AM, and is expired.
7. Expirer now roll back container resource to 4GB in RM.

Now RM and NM has inconsistent view of resource allocation ... Did I miss 

> CapacityScheduler side changes to support increase/decrease container 
> resource.
> -------------------------------------------------------------------------------
>                 Key: YARN-1651
>                 URL: https://issues.apache.org/jira/browse/YARN-1651
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager, scheduler
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-1651-1.YARN-1197.patch

This message was sent by Atlassian JIRA

Reply via email to