[
https://issues.apache.org/jira/browse/YARN-7292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated YARN-7292:
-----------------------------
Description:
Had discussions with [~templedf], [~vvasudev], [~sunilg] offline. There're a
couple of resource profile related behaviors might need to be updated:
1) Configure resource profile in server side or client side:
Currently resource profile can be only configured centrally:
- Advantages:
A given resource profile has a the same meaning in the cluster. It won’t change
when we run different apps in different configurations. A job can run under
Amazon’s G2.8X can also run on YARN with G2.8X profile. A side benefit is YARN
scheduler can potentially do better bin packing.
- Disadvantages:
Hard for applications to add their own resource profiles.
2) Do we really need mandatory resource profiles such as
minimum/maximum/default?
3) Should we send resource profile name inside ResourceRequest, or should
client/AM translate it to resource and set it to the existing resource fields?
4) Related to above, should we allow resource overrides or client/AM should
send final resource to RM?
was:
Had discussions with [~templedf], [~vvasudev], [~sunilg] offline. There're a
couple of resource profile related behaviors might need to be updated:
1) Configure resource profile in server side or client side:
Currently resource profile can be only configured centrally:
- Advantages:
A given resource profile has a the same meaning in the cluster. It won’t change
when we run different apps in different configurations. A job can run under
Amazon’s G2.8X can also run on YARN with G2.8X profile. A side benefit is YARN
scheduler can potentially do better bin packing.
- Disadvantages:
Hard for applications to add their own resource profiles.
2) Do we really need mandatory resource profiles such as
minimum/maximum/default?
3) Existing feature requires every client has a resource-types.xml in order to
use multiple resource types, should we allow client/AM update supported
resource types via Yarn APIs?
4) Should we send resource profile name inside ResourceRequest, or should
client/AM translate it to resource and set it to the existing resource fields?
5) Related to above, should we allow resource overrides or client/AM should
send final resource to RM?
> Revisit Resource Profile Behavior
> ---------------------------------
>
> Key: YARN-7292
> URL: https://issues.apache.org/jira/browse/YARN-7292
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
> Reporter: Wangda Tan
> Assignee: Wangda Tan
> Priority: Blocker
>
> Had discussions with [~templedf], [~vvasudev], [~sunilg] offline. There're a
> couple of resource profile related behaviors might need to be updated:
> 1) Configure resource profile in server side or client side:
> Currently resource profile can be only configured centrally:
> - Advantages:
> A given resource profile has a the same meaning in the cluster. It won’t
> change when we run different apps in different configurations. A job can run
> under Amazon’s G2.8X can also run on YARN with G2.8X profile. A side benefit
> is YARN scheduler can potentially do better bin packing.
> - Disadvantages:
> Hard for applications to add their own resource profiles.
> 2) Do we really need mandatory resource profiles such as
> minimum/maximum/default?
> 3) Should we send resource profile name inside ResourceRequest, or should
> client/AM translate it to resource and set it to the existing resource
> fields?
> 4) Related to above, should we allow resource overrides or client/AM should
> send final resource to RM?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
