[
https://issues.apache.org/jira/browse/YARN-8002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398195#comment-16398195
]
Weiwei Yang commented on YARN-8002:
-----------------------------------
Hi [~leftnoteasy]
I spent a few hours consolidating the patch based on your comments,
{quote}I'm fine with keep the logics here, but here I just want to make sure
that we don't do any unnecessary aggregations for single.
{quote}
Make sense. In {{AllocationTagsManager}}, I have updated
{{aggregateAllocationTagsByNode}} and {{aggregateAllocationTagsByRack}} to make
sure there is no unnecessary aggregation for single app ID case.
{quote}More specifically, following call need to be avoided
{quote}
I tried, but I did not find out a better neat way to handle this. This
consistent model was suggested by [~asuresh] in earlier discussion, no matter
what namespace a parser returns to me, I call evaluate to ensure its target
applications is properly interpreted. Otherwise we will end up with a lot of
if-else.
However, I have made some improvements. If you take a look at
{{AllocationTagNamespace}}, now only classes need evaluation step overrides the
{{evaluate()}} method now. Hope that looks better now.
{quote}For AllocationTags, I'm fine with keep the name, but I suggest to move
it to scheduler.constraint package...
{quote}
Make senses. I have moved
* AllocationTagNamespace
* AllocationTags
* Evaluable
to {{org.apache.hadoop.yarn.server.resourcemanager.scheduler.constraint}}
package. Only left {{AllocationTagNamespaceType}} in the api package.
{quote}AllocationTags#fromScope... I suggest to make explicit names to this.
{quote}
Make sense. I've done so.
{quote}we should keep as less public interfaces as possible for ATM
{quote}
I have removed following public methods since they are only used by UT now,
{code:java}
public long getNodeCardinalityByOp(NodeId nodeId, ApplicationId applicationId,
Set<String> tags, LongBinaryOperator op);
public long getRackCardinalityByOp(String rack, ApplicationId applicationId,
Set<String> tags, LongBinaryOperator op);
{code}
{quote}about not-self
{quote}
I discussed with [~asuresh] offline, he mentioned this is required in their
use-case. I suggest we do keep this namespace type for now, given the approach
we are using now, it is trivial to add or remove the support of this namespace.
Please take a look at the v3 patch. Let know your thoughts, thanks!
> Support NOT_SELF and ALL namespace types for allocation tag
> -----------------------------------------------------------
>
> Key: YARN-8002
> URL: https://issues.apache.org/jira/browse/YARN-8002
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Priority: Major
> Attachments: YARN-8002.001.patch, YARN-8002.002.patch
>
>
> This is a continua task after YARN-7972, YARN-7972 adds support to specify
> tags with namespace SELF and APP_ID, like following
> * self/<tag>
> * app-id/<appid>/<tag>
> this task is to track the work to support 2 of remaining namespace types
> *NOT_SELF* & *ALL* (we'll support app-label later),
> * not-self/<tag>
> * all/<tag>
> this will require a bit refactoring in {{AllocationTagsManager}} as it needs
> to do some proper aggregation on tags for multiple apps.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
