[jira] [Commented] (YARN-8013) Support application tags when defining application namespaces for placement constraints
[ https://issues.apache.org/jira/browse/YARN-8013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16425955#comment-16425955 ] Hudson commented on YARN-8013: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13924 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13924/]) YARN-8013. Support application tags when defining application namespaces (kkaranasos: rev 7853ec8d2fb8731b7f7c28fd87491a0a2d47967e) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/MockRMApp.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/constraint/TestAllocationTagsNamespace.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/constraint/TargetApplicationsNamespace.java * (delete) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/constraint/AllocationTagNamespace.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/constraint/TestPlacementConstraintsUtil.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/constraint/PlacementConstraintsUtil.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/constraint/AllocationTagsManager.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/constraint/TargetApplications.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestSchedulingRequestContainerAllocation.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/AllocationTagNamespaceType.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/placement/SingleConstraintAppPlacementAllocator.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/constraint/TestAllocationTagsManager.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/constraint/AllocationTags.java > Support application tags when defining application namespaces for placement > constraints > --- > > Key: YARN-8013 > URL: https://issues.apache.org/jira/browse/YARN-8013 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Weiwei Yang >Assignee: Weiwei Yang >Priority: Major > Fix For: 3.2.0, 3.1.1 > > Attachments: YARN-8013.001.patch, YARN-8013.002.patch, > YARN-8013.003.patch, YARN-8013.004.patch, YARN-8013.005.patch, > YARN-8013.006.patch, YARN-8013.007.patch > > > YARN-1461 adds the concept of *Application Tags* to Yarn applications. In > particular, a user is able to annotate application with multiple tags to > classify apps. We can leverage this to define application namespaces based on > application tags for placement constraints. > Below is a typical use case. > There are a lot of TF jobs running on Yarn, and some of them are consuming > resources heavily. So we want to limit number of PS on each node for such BIG > players but ignore those SMALL ones. To achieve this, we can do following > steps: > # Add application tag "big-tf" to these big TF jobs > # For each PS request, we add "ps" source tag and map it to constraint > "{color:#d04437}notin, node, tensorflow/ps{color}" or > "{color:#d04437}cardinality, node, tensorflow/ps{color}{color:#d04437}, 0, > 2{color}" for finer grained controls. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8013) Support application tags when defining application namespaces for placement constraints
[ https://issues.apache.org/jira/browse/YARN-8013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424848#comment-16424848 ] Weiwei Yang commented on YARN-8013: --- Hi [~kkaranasos] The conflict was because YARN-8002 was not committed to branch-3.1. I have just cherry-picked YARN-8002 to branch-3.1, now there should have no conflicts anymore. Could you please try again? Thanks. > Support application tags when defining application namespaces for placement > constraints > --- > > Key: YARN-8013 > URL: https://issues.apache.org/jira/browse/YARN-8013 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Weiwei Yang >Assignee: Weiwei Yang >Priority: Major > Attachments: YARN-8013.001.patch, YARN-8013.002.patch, > YARN-8013.003.patch, YARN-8013.004.patch, YARN-8013.005.patch, > YARN-8013.006.patch, YARN-8013.007.patch > > > YARN-1461 adds the concept of *Application Tags* to Yarn applications. In > particular, a user is able to annotate application with multiple tags to > classify apps. We can leverage this to define application namespaces based on > application tags for placement constraints. > Below is a typical use case. > There are a lot of TF jobs running on Yarn, and some of them are consuming > resources heavily. So we want to limit number of PS on each node for such BIG > players but ignore those SMALL ones. To achieve this, we can do following > steps: > # Add application tag "big-tf" to these big TF jobs > # For each PS request, we add "ps" source tag and map it to constraint > "{color:#d04437}notin, node, tensorflow/ps{color}" or > "{color:#d04437}cardinality, node, tensorflow/ps{color}{color:#d04437}, 0, > 2{color}" for finer grained controls. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8013) Support application tags when defining application namespaces for placement constraints
[ https://issues.apache.org/jira/browse/YARN-8013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16424801#comment-16424801 ] Konstantinos Karanasos commented on YARN-8013: -- [~cheersyang], can you please post a patch for branch-3.1, too? There are a few merge conflicts due to the renames and I want to make sure we are doing introducing any errors. > Support application tags when defining application namespaces for placement > constraints > --- > > Key: YARN-8013 > URL: https://issues.apache.org/jira/browse/YARN-8013 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Weiwei Yang >Assignee: Weiwei Yang >Priority: Major > Attachments: YARN-8013.001.patch, YARN-8013.002.patch, > YARN-8013.003.patch, YARN-8013.004.patch, YARN-8013.005.patch, > YARN-8013.006.patch, YARN-8013.007.patch > > > YARN-1461 adds the concept of *Application Tags* to Yarn applications. In > particular, a user is able to annotate application with multiple tags to > classify apps. We can leverage this to define application namespaces based on > application tags for placement constraints. > Below is a typical use case. > There are a lot of TF jobs running on Yarn, and some of them are consuming > resources heavily. So we want to limit number of PS on each node for such BIG > players but ignore those SMALL ones. To achieve this, we can do following > steps: > # Add application tag "big-tf" to these big TF jobs > # For each PS request, we add "ps" source tag and map it to constraint > "{color:#d04437}notin, node, tensorflow/ps{color}" or > "{color:#d04437}cardinality, node, tensorflow/ps{color}{color:#d04437}, 0, > 2{color}" for finer grained controls. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org