[
https://issues.apache.org/jira/browse/YARN-8002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393974#comment-16393974
]
Wangda Tan commented on YARN-8002:
----------------------------------
Thanks [~cheersyang] for the contexts, suggestions from a quick glance:
1) AllocationTags:
The naming is a bit confusing, from the name it looks like
List<AllocationTag>. However it is actually for cardinality query. So how about
calling it like "AllocationTagsFilter" or "AllocationTagsQuery"?
2) About ALL evaluator:
Is it really necessary (since we already have special handling for cluster
level tags). And some related classes like
{{AllocationTagsManager#getAllApplicationIds}}.
I think we should be able to eliminate, correct?
3) As I mentioned offline, I'm not sure if NOT_SELF is necessary, unless it is
must-to-have, I don't suggest to add it.
4) About existing aggregation approach:
Currently it takes list of application id from the eval result. Which is a
O(N) operation (N = #apps from the definition). And it uses addtional O(N)
space as well. (Since we need to copy map in some cases).
Alternatively, if we can assume:
- Each app has only 1 app-label.
- And no NOT_SELF namespace.
We can actually finish all get Cardinality / update Cardinality in O(1) and
space space complexity is also O(1).
(Explanation, for each application update, it will update 6 maps:
- App_Id node/rack.
- App_label node/rack.
- Global node/rack.
Basically, it will be 3 * (#node-scope), node-scope includes
node/rack/failure-zone, etc. which will still be better than O(#apps) *
(#node-scope) for every retrive operation.)
And retrieve cardinality for ALL/App_id/App_label will be just O(1).
5)
{code:java}
public long getNodeCardinalityByOp(NodeId nodeId, ApplicationId applicationId,
Set<String> tags, LongBinaryOperator op)
public long getRackCardinalityByOp(String rack, ApplicationId applicationId,
Set<String> tags, LongBinaryOperator op)
{code}
Can be removed, we should keep as less public interfaces as possible for ATM
> Support NOT_SELF and ALL namespace types for allocation tag
> -----------------------------------------------------------
>
> Key: YARN-8002
> URL: https://issues.apache.org/jira/browse/YARN-8002
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Priority: Major
> Attachments: YARN-8002.001.patch, YARN-8002.002.patch
>
>
> This is a continua task after YARN-7972, YARN-7972 adds support to specify
> tags with namespace SELF and APP_ID, like following
> * self/<tag>
> * app-id/<appid>/<tag>
> this task is to track the work to support 2 of remaining namespace types
> *NOT_SELF* & *ALL* (we'll support app-label later),
> * not-self/<tag>
> * all/<tag>
> this will require a bit refactoring in {{AllocationTagsManager}} as it needs
> to do some proper aggregation on tags for multiple apps.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
