[
https://issues.apache.org/jira/browse/YARN-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated YARN-5587:
--------------------------------
Attachment: YARN-5587-YARN-3926.006.patch
Thanks for the review [~asuresh]! I've uploaded a new patch to address your
comments.
{quote}
We create a special ProfileCapability with profile name NONE where user
only specifies the capability (Resource). This would be equivalent to our old
Resource
We replace the capability(Resource) argument in both the ContainerRequest
and ResourceRequest with ProfileCapability. Maybe for backward compatibility,
we have 1 constructor that takes Resource and 1 that takes ProfileCapability.
We then convert a Resource to ProfileCapability(NONE, Resource) in the
AMRMClient.
{quote}
Fixed. I've left the old Resource interfaces for backwards compatibility but
internally everything is getting converted to a ProfileCapability. Instead of
NONE, I'm setting the profile to "default". On the RM side, the default profile
is used and overriden with the Resource values.
{quote}
In the rest of the AMRMClient codebase, including the RemoteRequestsTable}
}we just use {{ProfileCapability
{quote}
Fixed.
{quote}
Hmmm... Actually, I think we have to deal with one other issue in the
AMRMClient:
Once a Container is returned by the RM, the AMRMClient uses the
getMatchingRequests() api to retrieve matching ContainerRequests and remove it.
This implies that we need to have an equivalence relationship from a Resource
to a ProfileCapability, else the matching won't work. Thus I think we might
have to normalize all "named" ProfileCapabilities to a either a Resource or a
NONE ProfileCapability (with the override being the complete resource)
Either that, or we have to ensure that these new requests are tagged with
unique 'allocationRequestIds', for which we already have a separate
getMatchingRequests that matches ContainerRequests to Containers only based on
the allocationRequestId.
{quote}
I missed this - thanks for pointing it out. I added a function to convert
profiles back to the Resource values to make the functionality work.
> Add support for resource profiles
> ---------------------------------
>
> Key: YARN-5587
> URL: https://issues.apache.org/jira/browse/YARN-5587
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
> Reporter: Varun Vasudev
> Assignee: Varun Vasudev
> Labels: oct16-hard
> Attachments: YARN-5587-YARN-3926.001.patch,
> YARN-5587-YARN-3926.002.patch, YARN-5587-YARN-3926.003.patch,
> YARN-5587-YARN-3926.004.patch, YARN-5587-YARN-3926.005.patch,
> YARN-5587-YARN-3926.006.patch
>
>
> Add support for resource profiles on the RM side to allow users to use
> shorthands to specify resource requirements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
