Wangda Tan commented on YARN-7292:


Let me explain a bit:
- Resource profile now can be specified in ContainerRequest:
    public ContainerRequest(Resource capability, String[] nodes, String[] racks,
        Priority priority, long allocationRequestId, boolean relaxLocality,
        String nodeLabelsExpression,
        ExecutionTypeRequest executionTypeRequest, String profile)
The underlying logic is:
Resource actual = Resource.clone(profileMap.get(profile));
for (ResourceInformation info : capability.getResourceInformations()) {
   if (info.getValue() > 0) {
      actual.set(info.getName(), info.getValue());
// Return actual

Do you think we should add a "do-not-override" option to ContainerRequest 
constructor? Application developer pass a zero capability to use profile 
resource, or pass a null profile to use capability. I felt it should be enough.

Since we completely removed profile from ResourceRequest, that means an 
application (like MR) doesn't use AMRMClient can implement their own logic to 
do override. Do you think should we put the logic to a Util class like 
ResourceUtil (Or create a new one like ResourceProfileUtils)?

> Revisit Resource Profile Behavior
> ---------------------------------
>                 Key: YARN-7292
>                 URL: https://issues.apache.org/jira/browse/YARN-7292
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>            Priority: Blocker
>         Attachments: YARN-7292.002.patch, YARN-7292.003.patch, 
> YARN-7292.004.patch, YARN-7292.005.patch, YARN-7292.006.patch, 
> YARN-7292.wip.001.patch
> Had discussions with [~templedf], [~vvasudev], [~sunilg] offline. There're a 
> couple of resource profile related behaviors might need to be updated:
> 1) Configure resource profile in server side or client side: 
> Currently resource profile can be only configured centrally:
> - Advantages:
> A given resource profile has a the same meaning in the cluster. It won’t 
> change when we run different apps in different configurations. A job can run 
> under Amazon’s G2.8X can also run on YARN with G2.8X profile. A side benefit 
> is YARN scheduler can potentially do better bin packing.
> - Disadvantages: 
> Hard for applications to add their own resource profiles. 
> 2) Do we really need mandatory resource profiles such as 
> minimum/maximum/default? 
> 3) Should we send resource profile name inside ResourceRequest, or should 
> client/AM translate it to resource and set it to the existing resource 
> fields? 
> 4) Related to above, should we allow resource overrides or client/AM should 
> send final resource to RM?

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to