[
https://issues.apache.org/jira/browse/YARN-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15623533#comment-15623533
]
Wangda Tan commented on YARN-5587:
----------------------------------
Thanks [~vvasudev],
Took a quick look at the patch, some questions/comments:
1) Are Changes of ClusterNodeTrack necessary?
2) ProfileCapability.toResource:
- Result of the {{if (resourceProfileMap != null)}} will be overwritten by {{if
(capability.getProfile..)}}
- Should order of the two if.. be exchanged?
- Do we have a method to copy a Resource instead of using for... to clone
ResourceInformation
3) ResourceRequest / ProfileCapability / Resource
IIUC, one of getProfileCapability and getCapability will be selected based on
RM_RESOURCE_PROFILES_ENABLED. And
RMServerUtils#convertProfileToResourceCapability handles this logic to set
which is the effective capability. I have some questions/comments:
- How to handle cluster level profile update. For example, application request
a "small" container and waiting to be allocated. Before the container
allocated, admin updates capability of the "small" profile (can admin do things
like this?). In this case, should we allocate container with new configured
resource of "small" profile?
3) getProfileCapability.getProfileCapabilityOverride get preferred even if
RM_RESOURCE_PROFILES_ENABLED is false, not sure if it is the best solution.
Here's my suggestion about API design:
It looks like ProfileCapability#getProfileCapabilityOverride will be only used
by application (admin won't set this field), if this is true, in existing YARN,
we have two fields for capability -- ProfileCapability and Capability. Could we
treat the Capability as ProfileCapability#getProfileCapabilityOverride? This
approach has following benefits:
- Avoid putting application-required-only field to the common object
ProfileCapability which will be used by application and admin.
- Less option to application: application only need to set 2 fields
(capability, profile) instead of 3 (capability, profile, profile.override)
- It will also simplify some service/client logics such as RemoteRequestsTable.
> Add support for resource profiles
> ---------------------------------
>
> Key: YARN-5587
> URL: https://issues.apache.org/jira/browse/YARN-5587
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
> Reporter: Varun Vasudev
> Assignee: Varun Vasudev
> Labels: oct16-hard
> Attachments: YARN-5587-YARN-3926.001.patch,
> YARN-5587-YARN-3926.002.patch, YARN-5587-YARN-3926.003.patch,
> YARN-5587-YARN-3926.004.patch, YARN-5587-YARN-3926.005.patch,
> YARN-5587-YARN-3926.006.patch, YARN-5587-YARN-3926.007.patch
>
>
> Add support for resource profiles on the RM side to allow users to use
> shorthands to specify resource requirements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
