[
https://issues.apache.org/jira/browse/YARN-5478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated YARN-5478:
-----------------------------
Attachment: YARN-5478.2.patch
Had discussions with [~vinodkv]/[~asuresh]/[~kkaranasos]/[~subru] offline.
I think now we generally agree that we should stop investing on the old
ResourceRequest and we should move APIs for new features like
- Allocation tag: YARN-6345.
- Affinity/Anti-affinity: YARN-1042.
- Node attributes: YARN-4902.
To the new ResourceRequest.
We will create YARN-4902 branch and put all API-related changes to the branch
and upcoming features like YARN-1042/YARN-6345 can do cherry-pick from
YARN-4902.
The basic idea of the patch is to make new/old ResourceRequest can be used in
parallel, in the short term they will provide disjoint sets of features: Old
ResourceRequest continues to provide functionalities to specify
capability(resource)/num_containers/priority/locality, etc. And new
ResourceRequest will provide additional features like allocation tag/constraint
specs such as affinity/anti-affinity/node-attribute, etc. And the two will be
linked by their allocation_request_id. And one big advantage of this proposal
is completely backward compatible and no changes need to be made for existing
apps.
And slowly we will move more fields from old ResourceRequest to the new one and
finally deprecate the old one.
>From application's perspective, they can specify the new ResourceRequests in
>both of ApplicationSubmissionContext/AllocationRequest. There's an on-going
>discussion about what the protocol should look like. So I didn't include any
>changes in ApplicationSubmissionContext/AllocationRequest. Will update the
>patch once we reach consensus on that.
For overall API of the patch, please take a look at {{Examples.java}}.
> [YARN-4902] Define Java API for generalized & unified scheduling-strategies.
> ----------------------------------------------------------------------------
>
> Key: YARN-5478
> URL: https://issues.apache.org/jira/browse/YARN-5478
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Attachments: YARN-5478.1.patch, YARN-5478.2.patch,
> YARN-5478.preliminary-poc.1.patch, YARN-5478.preliminary-poc.2.patch
>
>
> Define Java API for application to specify generic scheduling requirements
> described in YARN-4902 design doc.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
