[
https://issues.apache.org/jira/browse/YARN-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15723856#comment-15723856
]
Naganarasimha G R commented on YARN-3409:
-----------------------------------------
Thanks [~wangda] for the quick feedback,
bq. I still think constraint is one special kind of label, we should reuse
existing APIs:
IMHO It depends on how we conclude on whether we require *constraint type*,
lets further discuss on it and then come back to this. and further i cross
verified, my bad we support replace nodelabel mappings and adding cluster node
labels through REST.
bq. Do we really need define type of the constraint?
In the example which i had given software version x.y.z it always doesn't have
the numerical digits to be padded for example {{r2 and r19}}, so ascii
comparison might fail, this is one such example we had faced earlier, but based
on scenario it can vary, but if we have a custom type and define how to compare
it would be better to adapt anything in future.
bq. I think we can do late-binding it.
May be value for the node's constraint is given as {{Long}} and in the
expression we give it as {{Double}} or vice versa, then evaluation will lead to
run time failures right ? or for each evaluation we need to check and define
what is the better match or how to compare two different kinds of values. Late
binding would be *too much costly/expensive* when each request has a constraint
expression. As we need to almost validate for such scenarios for each node
against the constraint expression of a resource request in each schedule cycle.
In the approach which i have specified in the patch its almost static
comparison of long to long, or double to double, or type to type, as the type
is predetermined type for the Node's value and the RHS value of the expression.
bq. sometimes we need compare it (like version > 2.3.4.5 and version < 2.3.4.9)
and sometimes we need do string much (like version ~ 2.3.4.*)
bq. Maybe we can support "like" operator as well, it will compare string with
given regex.
Agree i too was thinking about supporting regex for the the String Constraint
type. further it would be better to have for constraint Name too, which will be
helpful for Boolean type.
bq. We only need support one set of syntax (for example, equal is "==", but
"eq" will not be a valid operation name)
Yeah this is not a basic requirement, it would have been required if we
directly use it this expression in REST api's but we usually use as part of
ResourceRequest object, but just thought in case we want to support dynamic
Constraint expression modification for a RR in future.
I would like to get more feedback on the constraint type from others too, hope
to get early feedback
cc [~kkaranasos] [~devaraj.k] [~cchen317] [~grey] and others in the
subscriber's list.
> Add constraint node labels
> --------------------------
>
> Key: YARN-3409
> URL: https://issues.apache.org/jira/browse/YARN-3409
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: api, capacityscheduler, client
> Reporter: Wangda Tan
> Assignee: Naganarasimha G R
> Attachments: Constraint-Node-Labels-Requirements-Design-doc_v1.pdf,
> YARN-3409.WIP.001.patch
>
>
> Specify only one label for each node (IAW, partition a cluster) is a way to
> determinate how resources of a special set of nodes could be shared by a
> group of entities (like teams, departments, etc.). Partitions of a cluster
> has following characteristics:
> - Cluster divided to several disjoint sub clusters.
> - ACL/priority can apply on partition (Only market team / marke team has
> priority to use the partition).
> - Percentage of capacities can apply on partition (Market team has 40%
> minimum capacity and Dev team has 60% of minimum capacity of the partition).
> Constraints are orthogonal to partition, they’re describing attributes of
> node’s hardware/software just for affinity. Some example of constraints:
> - glibc version
> - JDK version
> - Type of CPU (x86_64/i686)
> - Type of OS (windows, linux, etc.)
> With this, application can be able to ask for resource has (glibc.version >=
> 2.20 && JDK.version >= 8u20 && x86_64).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
