Wangda Tan commented on YARN-2885:

Thanks for updating, I looked at latest patch.

bq. Given that we will already add the Distributed Scheduling Coordinator in 
the RM (which will be used for the top-k technique, and later for the 
corrective mechanisms in YARN-2888), what about using the same service for 
delegating the AMProtocol wrapper (rather than creating an additional one)?
The major concern of putting distribtued-scheduling-related protocol to the 
AMProtocol wrapper is: It binds two protocols with different purpose to same 

Per my understanding, DistributedSchedulingProtocol is for NM poll 
distributed-scheduling-related information from RM. It could related to 
application's allocation logic or not. For example:
- DistSchedRegisterResponse could be considered as regular heartbeat response 
between LocalRM and centralRM, but now it only gets invoked when an application 
master registers to NM. Same as DistSchedAllocateResponse.

I would still suggest to make an independent DistributedSchedulingProtocol and 
services at NM/RM. Which you can easier control frequency of 
distributed-scheduling-info update, etc.

Another drawback is, now DistributedSchedulingProtocol is in yarn.api, it is 
still visible from user's perspective.

> Create AMRMProxy request interceptor for distributed scheduling decisions for 
> queueable containers
> --------------------------------------------------------------------------------------------------
>                 Key: YARN-2885
>                 URL: https://issues.apache.org/jira/browse/YARN-2885
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Konstantinos Karanasos
>            Assignee: Arun Suresh
>         Attachments: YARN-2885-yarn-2877.001.patch, 
> YARN-2885-yarn-2877.002.patch, YARN-2885-yarn-2877.full-2.patch, 
> YARN-2885-yarn-2877.full.patch, YARN-2885_api_changes.patch
> We propose to add a Local ResourceManager (LocalRM) to the NM in order to 
> support distributed scheduling decisions. 
> Architecturally we leverage the RMProxy, introduced in YARN-2884. 
> The LocalRM makes distributed decisions for queuable containers requests. 
> Guaranteed-start requests are still handled by the central RM.

This message was sent by Atlassian JIRA

Reply via email to