[
https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16589171#comment-16589171
]
Naganarasimha G R commented on YARN-7863:
-----------------------------------------
Thanks [~cheersyang], for some clarifications but at the same time was
discussing with Sunil too.
{quote} PC doesn't affect any logic how scheduler selects requests, it is still
how it is handled now. A PC is simply checked twice when 1) creating an
allocation proposal on a node and 2) at commit phase against a specific node.
{quote}
I am not specifying that changes in this Jira alone introducing anything
implications but my point was, earlier when partitions was introduced it was
easy to determine pending resources per partition per queue in the earlier api
using resource request. Now with the New api introduced with PC(not just from
this Jira alone) there is no way to find out for a given queue how much pending
resources are there in each partition it can access. This is because PC can
have an Partition OR'd with Allocation TAG or with this Jira OR'd with
Attributes. Impact of this would be that Cluster admin will not be able to plan
resources for the partition per queue. And also i am not able to envisage the
scenario where in partition needs to be OR'd with Allocation tags or Attributes.
{quote}No, not necessarily. Just to keep the changes clean and incremental, we
can allow this form for now. Because this is the spec we used for distributed
shell. Since we don't have an --allocationTags argument. A "foo=3" is the only
way to specify allocation tags right now.
{quote}
Agree, My bad but the reason i got confused is all the parsing logic for the
Distributed shell's expression is present in "hadoop-yarn-api" project which
more so implies that this the API to be used by other apps too.
{quote}I want to reinforce the phase-target (for the merge), we want
node-attributes can be integrated into PC and support simple operation "=" and
"!=".
{quote}
Yes I concur with you on this, as long as we are able to clearly capture for
others what kind of Java API needs to be used to specify Node attributes.
[~sunilg],
Could there be a simple test case or example which captures how to write a java
api where in node attributes can be specified for a given SchedulingRequest
i.e. without any of these expression DSL's ?
Also i could not see a test case where in CS is handling scheduling of
containers with PC having attributes. So that modifications in
PlacementConstraintsUtil is tested.
> Modify placement constraints to support node attributes
> -------------------------------------------------------
>
> Key: YARN-7863
> URL: https://issues.apache.org/jira/browse/YARN-7863
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Sunil Govindan
> Assignee: Sunil Govindan
> Priority: Major
> Attachments: YARN-7863-YARN-3409.002.patch,
> YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch,
> YARN-7863-YARN-3409.005.patch, YARN-7863-YARN-3409.006.patch,
> YARN-7863-YARN-3409.007.patch, YARN-7863-YARN-3409.008.patch,
> YARN-7863.v0.patch
>
>
> This Jira will track to *Modify existing placement constraints to support
> node attributes.*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
