[
https://issues.apache.org/jira/browse/YARN-7854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349752#comment-16349752
]
LiangYe edited comment on YARN-7854 at 2/2/18 4:02 AM:
-------------------------------------------------------
In k8s，
[Namespace|https://github.com/kubernetes/community/blob/master/contributors/design-proposals/architecture/namespaces.md]
is a mechanism to partition resources created by users into logically named
group.
The
[Namespace|https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/]
provides a unique scope for:
# named resources (pods, services, replication controllers, etc. to avoid
basic naming collisions, {color:#ff0000}*now not include nodes and
persistantVolume*{color})
# delegated management authority to trusted users
# ability to limit community resource consumption
In k8s, _Labels_ are key/value pairs that are attached to objects, such as
pods, nodes. Valid label keys have two segments: an optional prefix and name,
separated by a slash ({{/}}). If the prefix is omitted, the label Key is
presumed to be private to the user. Automated system components (e.g.
{{kube-scheduler}}, {{kube-controller-manager}}, {{kube-apiserver}},
{{kubectl}}, or other third-party automation) which add labels to end-user
objects must specify a prefix. The {{kubernetes.io/}} prefix is reserved for
Kubernetes core components.
In k8s, according to different object's labels, there is different affinity,
e.g. [node labels——node
affinity|https://github.com/kubernetes/community/blob/master/contributors/design-proposals/scheduling/nodeaffinity.md],
[pod labels——pod
affinity|https://github.com/kubernetes/community/blob/master/contributors/design-proposals/scheduling/podaffinity.md].
Especially [node taints——pod
toleration|https://github.com/kubernetes/community/blob/master/contributors/design-proposals/scheduling/taint-toleration-dedicated.md].
In yarn, according to YARN-3409 YARN-6592 [#YARN-7800] and so, node
attribute is like k8s node labels, allocation tags to container is like k8s pod
labels but in k8s pod labels not delivery to nodes for namespace isolation,
node constraint is some kind of k8s taints and tolerations.
So to this Jira issue, to make up prefix for node attributes from different
sources is a kind of namespace isolation.
For now,
# Centralized: attributes set by users (admin or normal users) can use a
prefix like {color:#ff0000}yarn_rm_admin/{color}
# Distributed: attributes collected by a certain attribute provider on each NM
can use a prefix like {color:#ff0000}yarn_nm/{color}
# System: some built-in attributes in yarn, set by yarn internal components,
e.g scheduler like {color:#ff0000}yarn_system/{color}
then,
In RM merge from these sources, no need to handle the conflict cases.
In placement constraints for node attributes should specify prefix to indicate
the sources/namespaces, or use yarn_nm/ prefix by default.
[~cheersyang] [~leftnoteasy] [~Naganarasimha] any comment? Thanks.
was (Author: lambertye):
In k8s，
[Namespace|https://github.com/kubernetes/community/blob/master/contributors/design-proposals/architecture/namespaces.md]
is a mechanism to partition resources created by users into logically named
group.
The
[Namespace|https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/]
provides a unique scope for:
# named resources (pods, services, replication controllers, etc. to avoid
basic naming collisions, {color:#FF0000}*now not include nodes and
persistantVolume*{color})
# delegated management authority to trusted users
# ability to limit community resource consumption
In k8s, _Labels_ are key/value pairs that are attached to objects, such as
pods, nodes. Valid label keys have two segments: an optional prefix and name,
separated by a slash ({{/}}). If the prefix is omitted, the label Key is
presumed to be private to the user. Automated system components (e.g.
{{kube-scheduler}}, {{kube-controller-manager}}, {{kube-apiserver}},
{{kubectl}}, or other third-party automation) which add labels to end-user
objects must specify a prefix. The {{kubernetes.io/}} prefix is reserved for
Kubernetes core components.
In k8s, according to different object's labels, there is different affinity,
e.g. [node labels——node
affinity|https://github.com/kubernetes/community/blob/master/contributors/design-proposals/scheduling/nodeaffinity.md],
[pod labels——pod
affinity|https://github.com/kubernetes/community/blob/master/contributors/design-proposals/scheduling/podaffinity.md].
Especially [node taints——pod
toleration|https://github.com/kubernetes/community/blob/master/contributors/design-proposals/scheduling/taint-toleration-dedicated.md].
In yarn, according to YARN-3409 YARN-6592 [#YARN-7800] and so, node
attribute is like k8s node labels, allocation tags to container is like k8s pod
labels but in k8s pod labels not delivery to nodes for namespace isolation,
node constraint is some kind of k8s taints and tolerations.
So to this Jira issue, to make up prefix for node attributes from different
sources is a kind of namespace isolation.
For now,
# Centralized: attributes set by users (admin or normal users) can use a
prefix like {color:#FF0000}yarn_rm_admin/{color}
# Distributed: attributes collected by a certain attribute provider on each NM
can use a prefix like {color:#FF0000}yarn_nm/{color}
# System: some built-in attributes in yarn, set by yarn internal components,
e.g scheduler like {color:#FF0000}yarn_system/{color}
then,
In RM merge from these sources, no need to handle the conflict cases.
In placement constraints for node attributes should specify prefix to indicate
the sources/namespaces, or use yarn_nm/ prefix by default.
[~cheersyang] [~leftnoteasy] [~Naganarasimha] any comment? Thanks.
> Attach prefixes to different type of node attributes
> ----------------------------------------------------
>
> Key: YARN-7854
> URL: https://issues.apache.org/jira/browse/YARN-7854
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: RM
> Reporter: Weiwei Yang
> Assignee: LiangYe
> Priority: Major
>
> There are multiple types of node attributes depending on which source it
> comes from, includes
> # Centralized: attributes set by users (admin or normal users)
> # Distributed: attributes collected by a certain attribute provider on each
> NM
> # System: some built-in attributes in yarn, set by yarn internal components,
> e.g scheduler
> To better manage these attributes, we introduce the prefix (namespace)
> concept to the an attribute. This Jira is opened to figure out how to attach
> prefixes (automatically/implicitly or explicitly) to different type of
> attributes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
