[
https://issues.apache.org/jira/browse/YARN-7972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383369#comment-16383369
]
Weiwei Yang commented on YARN-7972:
-----------------------------------
Uploaded v5 patch, a consolidated patch over v4. Some helper message for
reviewers: with help of {{AllocationTagNamespace}},
{{AllocationTagNamespaceType}}, the process of accessing tags in PCUtil will
look like
1. Parse a valid namespace from a target key
{code}
AllocationTagNamespace namespace = AllocationTagNamespace.parse(targetKey);
{code}
2. Evaluate the namespace against a certain target applications. This will set
an application ID set implicitly to the namespace, so make it easier to further
work with tags manager on cardinality checks.
{code}
// self
namespace.evaluate(ImmutableSet.of(currentAppId));
// app-id: no eval step needed
{code}
3. Retrieve the scope of the namespace and pass that to tags manager for
cardinality checks
{code}
// Right now we only support self and app-id namespaces, so scope returns a
single app ID set
namespace.getNamespaceScope()
{code}
I have done various of checks in {{PlacementConstraintsUtil}} to ensure it only
processes what we currently supported.
[~asuresh], [~leftnoteasy], appreciate if you can help to review. Once this can
get in, I can work on the refactor of AllocationTagsManager to support other
forms of namespaces.
Thanks a bunch!
> Support inter-app placement constraints for allocation tags by application ID
> -----------------------------------------------------------------------------
>
> Key: YARN-7972
> URL: https://issues.apache.org/jira/browse/YARN-7972
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Priority: Major
> Attachments: YARN-7972.001.patch, YARN-7972.002.patch,
> YARN-7972.003.patch, YARN-7972.004.patch, YARN-7972.005.patch
>
>
> Per discussion in [this
> comment|https://issues.apache.org/jira/browse/YARN-6599focusedCommentId=16319662&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16319662]
> in YARN-6599, we need to support inter-app PC for allocation tags.
> This will help to do better placement when dealing with potential competing
> resource applications, e.g don't place two tensorflow workers from two
> different applications on one same node.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
