[
https://issues.apache.org/jira/browse/YARN-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16300873#comment-16300873
]
Arun Suresh commented on YARN-7653:
-----------------------------------
Thanks for the patch [~pgaref].
It looks mostly good. One thing I did notice with the original patch is how do
we handle Node removals. Currently tags are removed only when a container is
completed. What if a node goes down ? Would we ever need a tag -> nodes mapping
?
I assume in the algorithm, you would generally iterate over nodes/racks and get
the cardinality - in which case, the issue above is somewhat moot (since you
will probably never ask the tags manager to "give me all nodes with this tag").
Although, might be nice to purge entries from the maps once in a while. We can
do this is in a follow-up JIRA if required.
> Node group support for AllocationTagsManager
> --------------------------------------------
>
> Key: YARN-7653
> URL: https://issues.apache.org/jira/browse/YARN-7653
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Panagiotis Garefalakis
> Assignee: Panagiotis Garefalakis
> Attachments: YARN-7653-YARN-6592.001.patch,
> YARN-7653-YARN-6592.002.patch
>
>
> AllocationTagsManager currently supports node and cluster-wide tag
> cardinality retrieval.
> If we want to support arbitrary node-groups/scopes for our placement
> constraints TagsManager should be extended to provide such functionality.
> As a first step we need to support RACK scope cardinality retrieval (as
> defined in our API).
> i.e. how many "spark" containers are currently running on "RACK-1"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
