[
https://issues.apache.org/jira/browse/YARN-7800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16337191#comment-16337191
]
Wangda Tan commented on YARN-7800:
----------------------------------
Thanks [~cheersyang] for sharing the thoughts.
I spent some time thinking about the problem, it looks like more complicated
when we have more labels.
For example:
{code}
req1: foo -> (anti-affinity) bar
req2: bar -> (anti-affinity) bar
req3: bar -> (anti-affinity) xyz
{code}
If somebody wants to use "bar" as source tag, "foo" and "xyz" need to be added
to avoid invalidate other's allocation. I roughly think this look like an
undirected graph, nodes are tags, any anti-affinity request adds an edge
between two tags, and if a request has "x" as source tag, it has to include all
directly connected nodes of "x". This graph keeps changing: At any time new
request could be added and the old request could be removed. If you think more,
you can find cardinality request could be even more complicated, we may need to
add weight to the edge and I don't know what it should look like.
> Bind node constraint once a container is proposed to be placed on this node
> ---------------------------------------------------------------------------
>
> Key: YARN-7800
> URL: https://issues.apache.org/jira/browse/YARN-7800
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: RM
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Priority: Major
>
> We found when there is circular dependency between multiple scheduling
> requests, allocation decisions made by placement constraint algorithm might
> be conflicting with related tags. More describing of the issue please refer
> to YARN-7783.
> To solve this issue, a possible solution is to bind *node constraint*. If the
> algorithm wants to place any new container to this node, except for checking
> if it satisfies the placement constraint, it also check if it satisfies the
> node constraint. For example
> 1) "foo", anti-affinity with "foo"
> +Implies node constraint:+ on each node, it cannot have more than 1 foo tags
> 2) "bar", anti-affinity with "foo"
> +Implies node constraint:+ on each node, it cannot have both "bar" and "foo"
> tags
> With such constraint, it works like
> * req2 is placed on any of nodes, e.g n2, +a node constraint[1] is added to
> n2 that constrains this node cannot have both "bar" and "foo" tags+
> * when the algorithm wants to place req1 on n2, it checks if its placement
> constraint is satisfied. It should be as there is no foo container on this
> node yet.
> * Then the algorithm checks if the node constraint is satisfied. It is not
> because it violate node constraint [1].
> This avoids to do additional re-attempt like what was done in YARN-7783.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
