[
https://issues.apache.org/jira/browse/YARN-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16301091#comment-16301091
]
Panagiotis Garefalakis edited comment on YARN-7653 at 12/22/17 8:15 AM:
------------------------------------------------------------------------
Thanks for the comments [~asuresh] !
Regarding NPE - the issue is addressed in the latest versions of the patch.
bq. What if a node goes down ?
In that case, I believe we have to follow the lifecycle of the affected
containers and purge the tags as they become unavailable. Maybe using the
particular RMContainer states: COMPLETED, EXPIRED, RELEASED, KILLED
bq. Would we ever need a tag -> nodes mapping ?
It is a valid point - the main reason for the extra mapping is to avoid
iterating through all the applicationIDs (as keys) and return the aggregated
counts. Even the algorithm implementation we would iterate through Nodes not
ApplicationIDs - so we would have to do the extra iterations to retrieve .i.e.:
a global count of tag "mapreduce" across all applications
Periodic cleaning could be part of another Jira I agree.
was (Author: pgaref):
Thanks for the comments [~asuresh] !
Regarding NPE - the issue is addressed in the latest versions of the patch.
bq. What if a node goes down ?
In that case, I believe we have to follow the lifecycle of the affected
containers and purge the tags as they become unavailable. Maybe using the
particular RMContainer states: COMPLETED, EXPIRED, RELEASED, KILLED
bq. Would we ever need a tag -> nodes mapping ?
It is a valid point - the main reason for the extra mapping is to avoid
iterating through all the applicationIDs (as keys) and return the aggregated
counts. Even the algorithm implementation we would iterate through Nodes not
ApplicationIDs - so we would have to do the extra iterations to retrieve a
global count of tag "mapreduce" across all applications for example.
Periodic cleaning could be part of another Jira I agree.
> Node group support for AllocationTagsManager
> --------------------------------------------
>
> Key: YARN-7653
> URL: https://issues.apache.org/jira/browse/YARN-7653
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Panagiotis Garefalakis
> Assignee: Panagiotis Garefalakis
> Attachments: YARN-7653-YARN-6592.001.patch,
> YARN-7653-YARN-6592.002.patch, YARN-7653-YARN-6592.003.patch
>
>
> AllocationTagsManager currently supports node and cluster-wide tag
> cardinality retrieval.
> If we want to support arbitrary node-groups/scopes for our placement
> constraints TagsManager should be extended to provide such functionality.
> As a first step we need to support RACK scope cardinality retrieval (as
> defined in our API).
> i.e. how many "spark" containers are currently running on "RACK-1"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
