[ https://issues.apache.org/jira/browse/YARN-8002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393974#comment-16393974 ]
Wangda Tan commented on YARN-8002: ---------------------------------- Thanks [~cheersyang] for the contexts, suggestions from a quick glance: 1) AllocationTags: The naming is a bit confusing, from the name it looks like List<AllocationTag>. However it is actually for cardinality query. So how about calling it like "AllocationTagsFilter" or "AllocationTagsQuery"? 2) About ALL evaluator: Is it really necessary (since we already have special handling for cluster level tags). And some related classes like {{AllocationTagsManager#getAllApplicationIds}}. I think we should be able to eliminate, correct? 3) As I mentioned offline, I'm not sure if NOT_SELF is necessary, unless it is must-to-have, I don't suggest to add it. 4) About existing aggregation approach: Currently it takes list of application id from the eval result. Which is a O(N) operation (N = #apps from the definition). And it uses addtional O(N) space as well. (Since we need to copy map in some cases). Alternatively, if we can assume: - Each app has only 1 app-label. - And no NOT_SELF namespace. We can actually finish all get Cardinality / update Cardinality in O(1) and space space complexity is also O(1). (Explanation, for each application update, it will update 6 maps: - App_Id node/rack. - App_label node/rack. - Global node/rack. Basically, it will be 3 * (#node-scope), node-scope includes node/rack/failure-zone, etc. which will still be better than O(#apps) * (#node-scope) for every retrive operation.) And retrieve cardinality for ALL/App_id/App_label will be just O(1). 5) {code:java} public long getNodeCardinalityByOp(NodeId nodeId, ApplicationId applicationId, Set<String> tags, LongBinaryOperator op) public long getRackCardinalityByOp(String rack, ApplicationId applicationId, Set<String> tags, LongBinaryOperator op) {code} Can be removed, we should keep as less public interfaces as possible for ATM > Support NOT_SELF and ALL namespace types for allocation tag > ----------------------------------------------------------- > > Key: YARN-8002 > URL: https://issues.apache.org/jira/browse/YARN-8002 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager > Reporter: Weiwei Yang > Assignee: Weiwei Yang > Priority: Major > Attachments: YARN-8002.001.patch, YARN-8002.002.patch > > > This is a continua task after YARN-7972, YARN-7972 adds support to specify > tags with namespace SELF and APP_ID, like following > * self/<tag> > * app-id/<appid>/<tag> > this task is to track the work to support 2 of remaining namespace types > *NOT_SELF* & *ALL* (we'll support app-label later), > * not-self/<tag> > * all/<tag> > this will require a bit refactoring in {{AllocationTagsManager}} as it needs > to do some proper aggregation on tags for multiple apps. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org