[
https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16558728#comment-16558728
]
Sunil Govindan edited comment on YARN-7863 at 7/26/18 6:35 PM:
---------------------------------------------------------------
Thanks [~Naganarasimha] and [~cheersyang] for comments
A quick summary based on an offline discussion:
{noformat}
DS Example:
bin/yarn org.apache.hadoop.yarn.applications.distributedshell.Client -jar
share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1.0-SNAPSHOT.jar
-shell_command Sleep
-shell_args 10
-num_containers 2
-master_memory 200
-container_memory 200
-placement_spec
OR(IN,python=true:NOTIN,java=1.8):IN,os!=centos:NOTIN,env=prod,dev
{noformat}
For DS, I think its better we correlate with placement_spec for now for better
easiness in review and api compatibility. A refactor improvement could be done
later to improvise this inline with native service approach.
Proposal for jira for later time is
{{-placement_spec (python=true OR java!=1.8) AND (os!=centos) AND (env NOTIN
(prod,dev))}}
Now to expand more on below approach in patch,
{noformat}
-placement_spec
OR(IN,python=true:NOTIN,java=1.8):IN,os!=centos:NOTIN,env=prod,dev{noformat}
we could see this like
*put me to a node where it has node-attribute python=true.*
OR
*dont put me in a node where it has node-attribute java=1.8*
AND
*put me to a node where it has node-attribute os is not centos.*
AND
*dont put me in a node where it has attributes env with values prod or dev*
Also attaching a patch based on this including comments from Naga. I ll add
test case in next patch.
was (Author: sunilg):
Thanks [~Naganarasimha] and [~cheersyang] for comments
A quick summary based on an offline discussion:
{noformat}
DS Example:
bin/yarn org.apache.hadoop.yarn.applications.distributedshell.Client -jar
share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1.0-SNAPSHOT.jar
-shell_command Sleep
-shell_args 10
-num_containers 2
-master_memory 200
-container_memory 200
-placement_spec
OR(IN,python=true:NOTIN,java=1.8):IN,os!=centos:NOTIN,env=prod,dev
{noformat}
For DS, I think its better we correlate with placement_spec for now for better
easiness in review and api compatibility. A refactor improvement could be done
later to improvise this inline with native service approach.
Proposal for jira for later time is
{{-placement_spec (python=true OR java!=1.8) AND (os!=centos) AND (env NOTIN
(prod,dev))}}
Now to expand more on below approach in patch,
{noformat}
-placement_spec
OR(IN,python=true:NOTIN,java=1.8):IN,os!=centos:NOTIN,env=prod,dev{noformat}
we could see this like
*put me to a node where it has node-attribute python=true.*
OR
*dont put me in a node where it has node-attribute java=1.8*
AND
*put me to a node where it has node-attribute os is not centos.*
AND
*dont put me in a node where it has attributes env with values prod or dev*
Also attaching a patch based on this including comments from Naga.
> Modify placement constraints to support node attributes
> -------------------------------------------------------
>
> Key: YARN-7863
> URL: https://issues.apache.org/jira/browse/YARN-7863
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Sunil Govindan
> Assignee: Sunil Govindan
> Priority: Major
> Attachments: YARN-7863-YARN-3409.002.patch,
> YARN-7863-YARN-3409.003.patch, YARN-7863.v0.patch
>
>
> This Jira will track to *Modify existing placement constraints to support
> node attributes.*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
