Wangda Tan updated YARN-1651:
    Attachment: YARN-1651-4.YARN-1197.patch

Thanks comments! [~mding].

bq. I only mention this because pullNewlyAllocatedContainers() has a check for 
null for the same logic, so I think we may want to make it consistent?
Yes you're correct, updated code, thanks.

bq. So, based on my understanding, if an application has reserved some resource 
for a container resource increase request on a node, that amount of resource 
should never be unreserved in order for the application to allocate a regular 
container on some other node. But that doesn't seem to be the case right now? 
Can you confirm?
Done, now added check to {{getNodeIdToUnreserve}}, will check if a container is 
a increase reservation before cancel it.

bq. I think it will be desirable to implement a pendingDecrease set in 
SchedulerApplicationAttempt, and corresponding logic, just like 
SchedulerApplicationAttempt.pendingRelease. This is to guard against the 
situation when decrease requests are received while RM is in the middle of 
recovery, and has not received all container statuses from NM yet.
I agree the general idea, and we should do the similar thing. However, I'm not 
sure caching in RM is a good idea, potentially a malicious AM can send millions 
of unknown-to-be-decreased-containers to RM when RM started. Maybe it's better 
to cache in AMRMClient side. I think we can do this in a separated JIRA? Could 
you file a new JIRA for this if you agree?

bq. Some nits...

Uploaded ver.4 patch.

> CapacityScheduler side changes to support increase/decrease container 
> resource.
> -------------------------------------------------------------------------------
>                 Key: YARN-1651
>                 URL: https://issues.apache.org/jira/browse/YARN-1651
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager, scheduler
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-1651-1.YARN-1197.patch, 
> YARN-1651-2.YARN-1197.patch, YARN-1651-3.YARN-1197.patch, 
> YARN-1651-4.YARN-1197.patch

This message was sent by Atlassian JIRA

Reply via email to