[jira] [Comment Edited] (YARN-8007) Support specifying placement constraint for task containers in SLS
[ https://issues.apache.org/jira/browse/YARN-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389557#comment-16389557 ] Weiwei Yang edited comment on YARN-8007 at 3/7/18 1:37 PM: --- Hi [~yangjiandan] Thanks for initiating this, it makes a lot of sense to help test scheduler perf with placement constraints, There are a few general comments to your patch # Instead of using {{SourceTags}} abstraction, lets follow the public API in scheduling request, which is to provide a Set of source tags and a PlacementConstraint. I think you will need 2 configs, allocation_tags: foo,bar placement_constraint: notin,node,foo this way we can honor number of tasks in the config. # I don't think we need to add placement constraint field in {{ContainerSimulator}}, a placement constraint is request level, not container level. # Have you tried to run this and how this works out? Thanks was (Author: cheersyang): Hi [~yangjiandan] Thanks for initiating this, it makes a lot of sense to help test scheduler perf with placement constraints, There are a few general comments to your patch # Instead of using {{SourceTags}} abstraction, lets follow the public API in scheduling request, which is to provide a Set of source tags and a PlacementConstraint. I think you will need 2 configs, allocation_tags: foo,bar placement_constraint: notin,node,foo this way we can honor number of tasks in the config. # I don't think we need to add placement constraint field in ContainerSimulator, a placement constraint is request level, not container level. # Have you tried to run this and how this works out? Thanks > Support specifying placement constraint for task containers in SLS > -- > > Key: YARN-8007 > URL: https://issues.apache.org/jira/browse/YARN-8007 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler-load-simulator >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > Attachments: YARN-8007.001.patch > > > YARN-6592 introduces placement constraint. Currently SLS does not support > specify placement constraint. > In order to help better perf test, we should be able to support specify > placement for containers in sls configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8007) Support specifying placement constraint for task containers in SLS
[ https://issues.apache.org/jira/browse/YARN-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389181#comment-16389181 ] Jiandan Yang edited comment on YARN-8007 at 3/7/18 9:03 AM: - Because currently MR specify placement constraints in the form of data locality, there is no need to specify placement constraint in MR job. I plan to implement specifying placement constraints in *generic/stream type* of SYNTH JSON format (introduced by YARN-7732) . [~cheersyang], [~leftnoteasy] Please feel free to share your opinions, thanks! was (Author: yangjiandan): Because currently MR specify placement constraints in the form of data locality, there is no need to specify placement constraint in MR job. I plan to implement specifying placement constraints in *generic/stream type* of SYNTH JSON format (introduced by YARN-7732) . [~cheersyang] [~leftnoteasy] Please feel free to share your opinions, thanks! > Support specifying placement constraint for task containers in SLS > -- > > Key: YARN-8007 > URL: https://issues.apache.org/jira/browse/YARN-8007 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler-load-simulator >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > > YARN-6592 introduces placement constraint. Currently SLS does not support > specify placement constraint. > In order to help better perf test, we should be able to support specify > placement for containers in sls configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8007) Support specifying placement constraint for task containers in SLS
[ https://issues.apache.org/jira/browse/YARN-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389181#comment-16389181 ] Jiandan Yang edited comment on YARN-8007 at 3/7/18 7:43 AM: - Because currently MR specify placement constraints in the form of data locality, there is no need to specify placement constraint in MR job. I plan to implement specifying placement constraints in *generic/stream type* of SYNTH JSON format (introduced by YARN-7732) . [~cheersyang] [~leftnoteasy] Please feel free to share your opinions, thanks! was (Author: yangjiandan): Because currently MR specify placement constraints in the form of data locality, there is no need to specify placement constraint in MR job. I plan to implement specifying placement constraints in *generic/stream type* app in SLS introduced by YARN-7732. [~cheersyang] [~leftnoteasy] Please feel free to share your opinions, thanks! > Support specifying placement constraint for task containers in SLS > -- > > Key: YARN-8007 > URL: https://issues.apache.org/jira/browse/YARN-8007 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler-load-simulator >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > > YARN-6592 introduces placement constraint. Currently SLS does not support > specify placement constraint. > In order to help better perf test, we should be able to support specify > placement for containers in sls configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8007) Support specifying placement constraint for task containers in SLS
[ https://issues.apache.org/jira/browse/YARN-8007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389181#comment-16389181 ] Jiandan Yang edited comment on YARN-8007 at 3/7/18 7:37 AM: - Because currently MR specify placement constraints in the form of data locality, there is no need to specify placement constraint in MR job. I plan to implement specifying placement constraints in *generic/stream type* app in SLS introduced by YARN-7732. [~cheersyang] [~leftnoteasy] Please feel free to share your opinions, thanks! was (Author: yangjiandan): Because currently MR specify placement constraints in the form of data locality, there is no need to specify placement constraint in MR job. I plan to implement specifying placement constraints in *stream type* app in SLS introduced by YARN-7732. [~cheersyang] [~leftnoteasy] Please feel free to share your opinions, thanks! > Support specifying placement constraint for task containers in SLS > -- > > Key: YARN-8007 > URL: https://issues.apache.org/jira/browse/YARN-8007 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler-load-simulator >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > > YARN-6592 introduces placement constraint. Currently SLS does not support > specify placement constraint. > In order to help better perf test, we should be able to support specify > placement for containers in sls configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org