[
https://issues.apache.org/jira/browse/YARN-7854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349832#comment-16349832
]
Weiwei Yang edited comment on YARN-7854 at 2/2/18 6:09 AM:
-----------------------------------------------------------
Hi [~LambertYe]
Thanks for the investigation. Following was from an earlier discussion
bq. Prefix will help in segregating the attributes provided by different
sources and is a DNS name, like
nm.yarn.io/docker-supported,nm.yarn.io/container-executor-type, *.yarn.io is
limited to be defined by system instead of admin using centralized API.
So we need to set prefix name with DNS format.
bq. In RM merge from these sources, no need to handle the conflict cases.
This might be still necessary. [~Naganarasimha] raised his concern this morning
about, for example, nm1 reports an attribute A:STRING:V1, nm2 reports an
attribute A:FLOAT:V2, is this allowed? He suggested we need some
cross-validation in RM on types. [~Naganarasimha], could you please comment
your idea?
bq. In placement constraints for node attributes should specify prefix to
indicate the sources/namespaces, or use yarn_nm/ prefix by default.
I agree. We need to define such behaviors. A rough thought is that in first
phase, we can ignore namespace check, that means an user is able to see all
attributes (this is related to constraints so we can visit this later). User
needs to specify the prefix while using it (so it is able to get a match),
otherwise a default prefix is added which belongs to this user's namespace (I
don't think we should use yarn_nm as default prefix for this case).
Thanks
was (Author: cheersyang):
Hi [~LambertYe]
Thanks for the investigation. Following was a from an earlier discussion
bq. Prefix will help in segregating the attributes provided by different
sources and is a DNS name, like
nm.yarn.io/docker-supported,nm.yarn.io/container-executor-type, *.yarn.io is
limited to be defined by system instead of admin using centralized API.
So we need to set prefix name with DNS format.
bq. In RM merge from these sources, no need to handle the conflict cases.
This might be still necessary. [~Naganarasimha] raised his concern this morning
about, for example, nm1 reports an attribute A:STRING:V1, nm2 reports an
attribute A:FLOAT:V2, is this allowed? He suggested we need some
cross-validation in RM on types. [~Naganarasimha], could you please comment
your idea?
bq. In placement constraints for node attributes should specify prefix to
indicate the sources/namespaces, or use yarn_nm/ prefix by default.
I agree. We need to define such behaviors. A rough thought is that in first
phase, we can ignore namespace check, that means an user is able to see all
attributes (this is related to constraints so we can visit this later). User
needs to specify the prefix while using it (so it is able to get a match),
otherwise a default prefix is added which belongs to this user's namespace (I
don't think we should use yarn_nm as default prefix for this case).
Thanks
> Attach prefixes to different type of node attributes
> ----------------------------------------------------
>
> Key: YARN-7854
> URL: https://issues.apache.org/jira/browse/YARN-7854
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: RM
> Reporter: Weiwei Yang
> Assignee: LiangYe
> Priority: Major
>
> There are multiple types of node attributes depending on which source it
> comes from, includes
> # Centralized: attributes set by users (admin or normal users)
> # Distributed: attributes collected by a certain attribute provider on each
> NM
> # System: some built-in attributes in yarn, set by yarn internal components,
> e.g scheduler
> To better manage these attributes, we introduce the prefix (namespace)
> concept to the an attribute. This Jira is opened to figure out how to attach
> prefixes (automatically/implicitly or explicitly) to different type of
> attributes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
