[
https://issues.apache.org/jira/browse/YARN-8002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395878#comment-16395878
]
Wangda Tan edited comment on YARN-8002 at 3/12/18 9:00 PM:
-----------------------------------------------------------
Thanks [~cheersyang],
bq. The approach in this patch adds the flexibility to let it easy to get
cardinality on a set of application IDs regardless what namespace it is. The
extra computation will only happen when we need to compute PC against app-tag
(or other namespace that evals to a set of app IDs), not every time. And with
the extra cost you mentioned (which I don't think it is very bad). What you
suggested saves this cost, but it will be hard to support not-self, app-ids(a
set of IDs) or possibly app-regex etc. I'd like to hold on re-implementing this
until every one agrees we are not going to support those.
I completely understand benefits here, I think we all agree that we should only
aggregate results when necessary (for app-ids, not-self, etc.). I'm fine with
keep the logics here, but here I just want to make sure that we don't do any
unnecessary aggregations for single.
More specifically, following call need to be avoided:
{code}
TargetApplications ta = new TargetApplications(currentAppId,
atm.getAllApplicationIds());
namespace.evaluate(ta);
{code}
For {{AllocationTags}}, I'm fine with keep the name, but I suggest to move it
to {{scheduler.constraint}} package. I think it should not belong to API
package.
{{AllocationTags#fromScope}}: First I don't suggest to use the "scope" name
since we already have it inside {{TargetConstraint}}. I suggest to make
explicit names to this. Such as
{code}
createSingleAppAllocationTags(ApplicationId, Set<String> tags)
createGlobalAllocationTags(Set<String> tags).
In the future, we can add
createAppsAllocationTags(...)
createAppLabelAllocationTags(...)
{code}
This makes code more readable and less implicit rules in parameters.
Similarly,
For {{AllocationTagNamespace}}, I saw it is used by some api packages, from
what I can see, the only code usage is toString. We should move it to
{{scheduler.constraint}} as well.
Instead of doing this, I would prefer to add a simple Java enum to api package
(such as AllocationTagTypes or AllocationTagNamespaceTypes), and use it in the
AllocationTagNamespaces class.
was (Author: leftnoteasy):
Thanks [~cheersyang],
bq. The approach in this patch adds the flexibility to let it easy to get
cardinality on a set of application IDs regardless what namespace it is. The
extra computation will only happen when we need to compute PC against app-tag
(or other namespace that evals to a set of app IDs), not every time. And with
the extra cost you mentioned (which I don't think it is very bad). What you
suggested saves this cost, but it will be hard to support not-self, app-ids(a
set of IDs) or possibly app-regex etc. I'd like to hold on re-implementing this
until every one agrees we are not going to support those.
I completely understand benefits here, I think we all agree that we should only
aggregate results when necessary (for app-ids, not-self, etc.). I'm fine with
keep the logics here, but here I just want to make sure that we don't do any
unnecessary aggregations for single.
More specifically, following call need to be avoided:
{code}
TargetApplications ta = new TargetApplications(currentAppId,
atm.getAllApplicationIds());
namespace.evaluate(ta);
{code}
For {{AllocationTags}}, I'm fine with keep the name, but I suggest to move it
to {{scheduler.constraint}} package. I think it should not belong to API
package.
{{AllocationTags#fromScope}}: First I don't suggest to use the "scope" name
since we already have it inside {{TargetConstraint}}. I suggest to make
explicit names to this. Such as
{code}
createSingleAppAllocationTags(ApplicationId, Set<String> tags)
createGlobalAllocationTags(Set<String> tags).
In the future, we can add
createAppsAllocationTags(...)
createAppLabelAllocationTags(...)
{code}
This makes code more readable and less implicit rules in configs.
Similarly,
For {{AllocationTagNamespace}}, I saw it is used by some api packages, from
what I can see, the only code usage is toString. We should move it to
{{scheduler.constraint}} as well.
Instead of doing this, I would prefer to add a simple Java enum to api package
(such as AllocationTagTypes or AllocationTagNamespaceTypes), and use it in the
AllocationTagNamespaces class.
> Support NOT_SELF and ALL namespace types for allocation tag
> -----------------------------------------------------------
>
> Key: YARN-8002
> URL: https://issues.apache.org/jira/browse/YARN-8002
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Priority: Major
> Attachments: YARN-8002.001.patch, YARN-8002.002.patch
>
>
> This is a continua task after YARN-7972, YARN-7972 adds support to specify
> tags with namespace SELF and APP_ID, like following
> * self/<tag>
> * app-id/<appid>/<tag>
> this task is to track the work to support 2 of remaining namespace types
> *NOT_SELF* & *ALL* (we'll support app-label later),
> * not-self/<tag>
> * all/<tag>
> this will require a bit refactoring in {{AllocationTagsManager}} as it needs
> to do some proper aggregation on tags for multiple apps.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
